Untrue, unquestioned negative thoughts are the problem, not your body. Fat is not a feeling. "I feel fat" is generally code for something else - fear, insecurity, hopelessness, anxiety. You can be bigger and not feel that way, or smaller and still feel it. It's not a problem solved by being a certain weight/size.
How we perceive our body is often related to how we've been treating ourselves in general.
I always 'feel fat' when my eating is not how I want it to be. When I am doing well with my 'Happy Eating' I have better body image, even if there has been no physical change to my body.