How to delete a song from my iPhone 4S?

Before the new software updates from Apple, I could just swipe the song on my iPhone and press delete when it came up, but now that function doesn't work and I haven't a clue how to delete songs from it.

I can delete them in my iTunes library but they still stay on my iPhone which is really quite annoying.

Anyone have any suggestions?

You can only delete them on itunes.