That sounds correct - registration (for tax purposes) should be required for whichever country you're residing in.

Before you leave the US, check with a US business lawyer about maintaining registration in the US: see if it's necessary, see if it's possible to do and yet not pay taxes to both countries, see what you have to do to "close down" your US registration so that, if you move back, you can re-register without running into any penalties or problems. When you register in the UK, check about the same thing. You don't need to know all the details, just what you'll need to do regarding your business registration if you re-relocate back to the US for a long period of time.