Yes, I would alter the pattern. If there is extra fabric with no place to go, it’ll stick out. If the rest of the bodice fits ok and it’s just at the top, it could mean that you have narrow shoulders.