I'm not sure you understand.If I moved it would be to completely ex-patriate the US and expunge them from my tax liability.
Short of renouncing your American citizenship, there's no real route to cutting them out of your obligation to pay them. I've not lived in the US for almost a decade and still get to pay US and local taxes when my income levels tip high enough to trigger US taxes responsibilities.
If you did want to renounce your US citizenship (which I wouldn't really suggest, especially if you're relocating to less stable portions of the globe) you'll be needing first to naturalize in Costa Rica. Which from a glance looks like it will require a 20 year residency for a non-Spanish applicant or marriage to a citizen.
While some of people think moving overseas will solve all their problems, having a practical idea of what is and isn't possible with a move is going to make decisions much easier.