I'm going to address the overweight comment right away. There are overweight people doing all sports. Just because you see someone ride a horse who happens to be over weight, doesn't mean anything. If you saw an overweight person play football (both footballs) or baseball or whatever, you wouldn't suddenly think that it's not hard to play. You'd just assume that that person happens to be overweight. It also depends on how serious of a rider that person is. If they only ride once a week, and that is their only exercise, they probably will be overweight. Riding everyday is extremely expensive. Whether you pay someone to ride their horse, or have your own horse, or you lease a horse, riding every day can cost you thousands a month.
It's also important to consider the difference between sport and recreation. A person can shoot hoops but never play basketball, they can play catch but never play baseball. Similarly, someone can get on a horse and dink around without actually choosing a riding related sport.
But I have not seen an Olympic rider who is obese. I think where people also go wrong is that some Olympic riders are much older than the contestant of other Olympic sports. This isn't because the sport is easy, it is because the sport takes years to get a grip of. I've been riding for over six years and I know that I'm still getting the basics down. And not because I'm inept, because I ride with people who have ridden longer than I, rather because knowledge in the sport is measured in years and decades.
Show Jumping: At it's very basics, show jumping is a test of strength, dexterity and speed. The riders and horses jump over obstacles in an arena as fast as they can without knocking down poles. If they knock down poles or the horse refuses to jump, they receive time penalties. There is also a time limit, and penalties for going over the time limit.
Dressage: Dressage at it's basics is a dance. It is a routine done to music. The riders tell the horses what to do without any noticeable movement, so it seems like the horse is reading the rider's mind. The movements of the horse replicate how they move when they are at play in the field without a rider on their backs.
Eventing: Eventing involves three events, Show Jumping, Dressage, and Cross Country. So it really requires a well rounded horse. Cross Country is a show of stamina. It is similar to show jumping, however it takes place along a vast outdoor track and the jumps resemble more natural looking objects. Cross country courses also involve multiple routes that riders can choose between.
Riding is a mixture of art and sport, much like dancing. The only difference is that dancing is considered an art and riding is considered a sport (by some people). (Also I suppose gymnastics applies as well.) But they have a lot in common:
-They both take years to learn, at least in my opinion.
-They require the whole body.
-The point of them is to look as easy as possible and be graceful.
-Appearance matters during competitions.
-The Judging at competitions is very particular and harsh.
-It is both a team sport and a singular sport at the same time.
I wish everyone would try to ride a horse a few times in their adult lives. I remember the first time I rode a horse I was extremely sore and I barely did anything. I feel if people just went on a trail ride and came away from that sore, when all they did was walk or trot around, and still woke up the next morning sore, that they might respect the sport riders who actually pull their weight in the ride. After all, you can't just sit on a horse and expect them to do everything. Look at the show jumping picture, her butt's not on the horse at all. That is not because it's a big jump, that is little jumps too. (To be honest, when you just sit on the horse as they go over the jump, it's very unpleasant to them and they start to hate you.).
So that is my two cents worth. (Trust me, I could give you five dollars worth but I think it'd be a bit tiresome). I think everyone should ride a horse at some point in their lives. Horses and humans have so much history. I know many people are afraid of horses, but I think there is a lot that horses can teach people. Plus then people might realize that horseback riding is a hardcore sport!