Heavier lines are easier to cast poorly, particularly when one buys a rod that is too stiff for one's skill level.
The creep in line weights reflects the market demand, which in turn is driven by guys who need a bandaid for their casting flaws.
When you can't feel your line 'loading' your rod you need a better backcast (and maybe a lighter reel), not a heavier line.
Likewise, the inability to cast a short line well on a stiff rod reflects too large a casting arc (and likely bad tracking) rather than a problem 'loading the rod'.
The whole concept of the need to "load" a rod to make a good cast is misleading. The rod acts like both a lever and a spring. The spring component contributes only about 15-25% of the energy transmitted to the line. The main benefit of "loading" a rod is that it makes the rod bend. The bend, in turn, helps keep the tip on a straight line path as the butt is accelerated in an arc.....hence transmitting force to the line directed towards a target rather than around an arc. Provided you accelerate a rod smoothly it will load itself correctly for the line that is on it. If you focus on trying to load it, however, things quickly go bad.
Hummm..... I'll have to think on that one. I like where you are going with that, but I'm not sure it covers the entire picture.
Energy transfer is the key. It's mostly on the shoulder of the caster's ability to manage slop and allowing the line to travel as true to very straight horizontal and vertical planes, but there is also a process of accumulating potential energy through rod loading, minor stretch in the line, casting drifting and such, allowing that perfect stop and then transferring and releasing that energy in the opposite direction and the timing of the perfect release and feathering control the shooting line to allow it to be place at that exact targeted place.
You're right, overloading a rod can lead to the cast failing, but then again, if distance required, underloading can also cause issues. Yes as you mentioned, tip travel is another key and how straight and true it travels according to how much or little the rod and rod tip flexes to maintain that straight rod tip path, as well as finding that sweet spot to not over or under power it to keep that tip in that perfect straight path (all all distances and at all phases of the cast, which is real all about getting to know and understand what the rod wants and likes to behave and then you giving it the right amount of thrust). Yes loading a tip flexible enough to cause the line to travel straight is one process, however, casting techniques can also make that happen in part. So I think it's the culmination of all those things (along with other items) that make it happen like it is effortless and flawless.
One other thing to remember to the rest of you reading this.... The Affta fly line weight chart is gauged on a 30ft length of the front side of the shooting head (less the tip). Today, there are a lot of heads longer and some even shorter than that (also heavbier and some lighter) and with that being said, one thing I see in casting errors and people trying to reach out to those far distance shots by aerializing too much of the shooting head outside of the tip top guide (far past that 30ft mark) while false casting (we'll just call it "carrying too much line"). If 30ft of the front part of a fly line head weighs out at 8wt on your 8wt rod @ the first 30ft of your head and you are carrying 40 and even 50ft up in the air and then trying to shoot it with one last powerful haul, then it can cause the rod to collapse (overload or over bend) and therefore take the loaded energy right out of the rod, slow the speed of the line down significantly and even throw an arch in your fly line and you end up with a mess on the business end of your flyline. Sure some rods built today need that extra weight bend deep enough at that right moment to unlock it's potential (say a heavier line or an extra 10ft or so out of the tip top guide). But most inexperience guys that really don't know their own rod yet will have issues making it work right.
Again, I didn't want to get in too deep in this subject. But there are many things happening there.
What I try to tell people is just to make sure your flies are properly sized to the fly line you are throwing them with. Then try to get a line closer to a true to weight line, unless they are still learning and the rod is too fast or stiff for them. Then LEARN proper casting strokes and techniques, keeping the line tracking straight horizontally and vertically, slowing down your casting strokes, waiting for the loops to completely unfurling out before starting your opposite stroke not carrying too much line, not overpowering your casting strokes or final shoot and then making your target 3ft above the target on the water, allowing your fly to reach that point before dropping your rod tip.
One final thing to note.... It's my opinion from doing this for decades (my son will say "a minute"), it you have a fair to good rod (doesn't have to be the best and most expensive rod out there), a very good flyline suited for your fishing conditions (don't skimp out on this one) and have really, really good casting techniques that you learned and practiced well, then you will have all the success and fun you need without buying some $2-$3k outfit. I see all these people chasing the latest and most expensive rods and reels, thinking that will make them a better caster, only to see them end up as a "Meh..." caster and fly fishermen and it doesn't put more fish to hand than a $300-$500 outfit. I promise you, as you guys learn (those reading this thread) about what it take to cast good..
Ted