Can I take the sim card out of my att iphone and use it in a tmobile iphone?

Just bought a 32gb iPhone 4s from someone who's been using T-Mobile. Can I transfer my current SIM card from my 16gb iPhone 4 from AT&T and basically use this new iPhone as my own?
Do I have to call either of the companies and ask them to do anything for compatibility reasons?

The phone would need to be sim free or unlocked for it to work

Nope. The T-Mobile iPhone in its current state will only work with T-Mobile SIMs. It would have to be unlocked in order for that to be possible.