How to change my battery for my iPhone 4s when it doesn't stay off?

When ever I turn my iPhone off it turns right back on, it's like a quick reboot. I tried everything, restoring it to deleting apps and restarting it. Nothing works so I'm over trying to keep it off, I just want to know how is it possible to change my battery without turning the phone completely off. Because it won't.

Mmm the phone eventually would run out of charge… When that happen in sure it won't turn on!

You will not damage the phone by unplugging the battery when you open the case you will see the connector just be sure this is the first thing you do as messing with other things will damage them