IPhone 4 sound only works using headphones?

The sound works when I have my headphones plugged in, but when I take them out there's no sound or even a bar to show sounds volume. When I hit the sound volume buttons on the side it just shows a picture of a speaker, but no volume bar.
How can I fix this?

Try resetting it by holding down the power on button and the home button at the same time until it restarts.

No bueno, it didn't do a thing

Turn off the mute button that's what I had to do