It’s a balancing act where you’ll never be happy. Whenever I cut weight and get lean, I appreciate how much more athletic I feel, but I note a lack of strength. When I add some weight and get a little fluffy, I get stronger in training but more sluggish.
Your best bet is to set some actual athletic and strength markers and compare those to see if it’s for real or just psychological. Set up agility drills or carry medleys or something and see if your time is declining. If it is, you actually ARE getting less athletic, and that means you need to prioritize it. If not, then it’s just in your head.
Also, if you ever try to get changed in the back of a car, you’ll pick light and lean over big and jacked everytime.