As has been said, most people don’t have a clue what big/freaky is, as if it’s something that happens by accident from going to the gym more than twice a week.
Most men, let alone women, don’t realise that in order to look good at a certain weight, you’re likely gona have to be 30-40lbs heavier than your target weight first, then diet down. A woman automatically thinks getting softer = bad. So while she may not feel that 15lbs of more muscle is bad AFTER it happens, she may not like the idea of your stomach getting bigger initially.
Also, it’s how it affects your personality that your woman care about the most. They hate the idea of you looking good and “knowing it” (as if as soon as you’ve reached your goals, you’re gona want a better model than them). They also hate a guy who “loves himself” more than anyone else (muscles on a guy = “I love myself” to a woman). So in effect, they are saying they don’t want you to become an arrogant prick who won’t part with the mirror haha.