If the sub is flopping around like a fish out of water when your playing higher notes then you are unloading, unloading will typically be caused by 1 of 3 things or a combination of those things. Too much power, to much port, or playing out of tuning frequency.
I personally don't like the port area by wattage method because things just like this happen, and that is why I prefer the per cube method. It only seems to work good in more of your SPL style builds where your only playing 1 frequency. You get to a point where too much is too much and you start to lose output. Ideally you want to take the sub(s) cone area, xmax, enclosure size, and applied power into consideration when figuring out port area.
For a free calculator WinISD is great there is a graph for port velocity on there, its worth downloading and learning, or if you need something more basic Torres works decently well too but wont give you any port velocity numbers.
Port velocity isn't that bad as long as you stay under 10-12% speed of sound. Depending on the setup and how the sub(s) and port are oriented you may not hear port noise at all even if you have a really high port velocity. For example a a box in the trunk with everything firing towards the rear of the vehicle.