How to get past the the iPhone setup on an iPhone 4?

I recently got a new phone but still wanted to use my old iphone 4, for music. There was a lot of extra clutter that I wanted to clear off of the phone, and so I did a factory restore. However, now I can't get past setup since it does not have a SIM card.

Could just borrow one to activate it

You need a SIM card installed when you activate an iPhone. You can remove it again after activation, but it has to be there while you are activating it.