Yeah it seems like going up in size isn't actually worth it. Sure you gain +4 strength but large has a -1 penalty to attack rolls and the penalties to your attack rolls and AC get worse the bigger you go. Doesn't actually seem to be any real tradeoff either. You hit harder but you hit less often and you become a bigger, easier to hit target too.