How to remove old iphone updates?

I need more space on my phone for music. I have the iPhone 5s, and only installed the first update or two that came out since I don't want it to take up 2gb in space. How can I remove all updates without completely restoring the phone?

You can't remove them once installed. They become part of the 'brains' of the phone's firmware. You can downgrade but it is risky and takes jailbreaking.
But also note the update doesn't consume 2gb. Read the fine print. It needs 2gb to download and install itself. It needs room to set up shop and get under construction. The actual size when it's done shrinks back to only adding roughly 700kb to the phone's brains. Bottom line, do a backup sync of your phone on iTunes. Remove enough stuff to give the update it's working room. Once the update is complete, sync all your info back. You really won't notice the missing kb's difference when you're done. Best luck.