How to keep deleted songs off of my iphone?

I have an iPhone 4S and when I don't want a song in my music anymore, I just swipe the song and delete it. The problem occurs when I hook my phone up to my iTunes. All the songs I've deleted from my phone are put back on. I don't want the songs off of my iTunes, I just want them off of my phone for good. I don't want them synced back onto my phone. What do I need to do?

What I do when deleting songs off my iPhone 5 is going into iTunes to delete the songs by dragging them in the trash. It should delete the song from iTunes too. Make sure your phone is plugged in while doing so.