I dont really know the science of it to the extent that u guys have stated so maybe Im full of shit but heres my spiel:
This is based on shit that I've learned as a college/pro athlete and from usa nutrition talks
To lose one pound you have to burn about 3500 calories.
Now whatever you eat produces your fuel which is made readily available in your glycogen stores. When you dont use these energy stores they are stored in the body as fat. so if you eat 5000 calories and only use 2000 the remaining 3500 gets stored as fat (one lb). Americans are typically so fat cuz we eat so goddamn much that its hard to burn off 10000 calories worth of intake when you sit behind a desk all day.
Now this is where shit can get messy...
Protein is not a favorable source of energy because its primarily used by the body to repair shit. When you eat a high protein low everything else you body goes into your fat stores for fuel--unless youre one fat motherfucka. This will also dip into you protein intake because fuck you thats why--kidding: doing all physical activity requires a bit of protein intake for fuel.
However as your body fat decreases and you continue to exercise your body can become catabolic. The fuel has to come from somewhere and what happens is the body starts eating itself--your muscle. And trust me that shit is super wack (and burns).
How to burn fat:
During short bust exercises (anaerobic: sprinting, weight lifting, jumping--basically quick movements that last shorter than 9 seconds) your body will use your readily available glycogen/fuel that is typically from what you have eaten recently. When it is done with that it will change over to your fat stores for energy/fuel and is change over largely an inefficient process. But for the most part these motions happen too quickly to really access your fat stores.
To access fat stores you need more indurance based exercises (aerobic: cycling, running, swimming, exercises that last longer than 9 seconds) your body will go to your readily available stores and determine that it needs a big source of fuel and access your fat stores.
How it effects your physical appearance is more of a genetic thing imo ("that shit will got straight to your hips", "thunder thighs", etc). My self for instance: 7.5% body fat but it feels and looks like its all in my stomach (maybe im being dramatic).
This is all my theory now:
I think the reason you see endurance based athletes typically with leaner frames is from years for training by accessing your fat stores for fuel/energy. The body is a marvelous thing and I believe that endurance athletes have adapted to accessing fat stores almost directly for tens of years. And since changing from a more aerobic energy system to an aerobic energy system is inefficient for body the aerobic system stays "on" continually accessing your fat stores for energy in similar situations where a normal person's body would access energy aerobically (burning readily available fuel).
My logic is based on a conversation I had with my weight coach years ago about drinking--if you havent figured it out from phelps: swimmers got the club turnin up on a tuesday. Swimming is a heavy aerobic sport even for sprinters. Because of this our body's fuel burning system is dominantly the aerobic system that accesses fat stores. And to be efficient that energy system just stays in the "on" position. Because it stays "on" we are able to drink our asses off and our bodies quickly burn through the beer calories as they are stored. My coach explained it to me this way and topped it off saying that swimmers oven/fire is always burning and consuming fuel even when it really doesnt need to. Other sports are largely anaerobic (basketball, football, track sprints, etc.) and dont burn stored energy as quickly or efficiently.
Lastly there are sports/exercises that are in the middle (rowing, middle distance running, decathalon, skiing, snowboarding, etc) and even in anaerobic exercises there are parts where you switch between energy systems (basketball played over a long time, decathalon, parts of football, etc).
Clearly its possible to train your body to stay in the "on" possition but I'm not sure how long that adaption takes to occur. It most certainly can be turned off quickly though--Ive seen olympians BALOON once they stop swimming...
life comes at you fast...
Hope this helps. If someone already addressed what I just said my bad--I was typing this while watching the walking dead.
edit: there are empty calories (french fries, white rice, shit like that) that do nothing for you. I think someone mentioned it above