How to delete songs/albums on iphone?

I deleted them off of my iTunes but even when I click do not transfer songs from iphone to itunes and sync it, the songs are still left on my iphone. Is there a way to delete them by syncing it to my computer or do I have to manually delete it on my phone? (Note: the deleted songs have the icloud logo next to them, I guess meaning that they were deleted but I can download them again, but they still play when my phone is playing music.)

Swipe to left of the song on playlist and you will see a red square with the word Delete on it

Here is a quick guide that will walk you through how you can delete songs from themusci app