While using a clean phone with nothing on it sounds like a good plan, it also looks very suspicious and can attract more attention. So take that into account when traveling regardless of the destination. Just like anonymization on your browser, the goal is to blend in, not stand out.
Honestly, it depends a lot on where you’re coming from and what you look like. And to be frank, it’s a little overblown to be paranoid enough to go that far, unless you are a high profile person of interest like a journalist or someone involved in politics. If you’re just an ordinary bloke going on vacation or attending a conference, you will be fine with basic precautions.