You don't have to remove the stock, but you might want to- it makes the receiver extension easier to turn.
You need to remove the castle nut, which may well be staked on (I believe Colt stakes theirs). That is, look closely at where the castle nut touches the end plate- you'll probably see some dimpling of the end plate into recesses on the nut- that's to keep it from loosening up under recoil. You may need to dremel that metal out of there (it's going to ugly up the castle nut a bit) in order to loosen the thing. I might recommend replacing the castle nut too, and then restake it when you retighten.
Just be careful when you do get it off; unscrewing the receiver extension can launch the buffer stop pin into hard to access places in the room, and removing the end plate will expose the takedown pin detent spring, allowing it to fall out (along with the very hard to find once it's lost detent).