It depends. I lived in Florianópolis, Brazil for most of my life and I don't think I would drink from the tap there. Not that the water was dirty, but it was just not recommended. We always had a filter that would clean the tap water.
Then I moved to Poznan, Poland, and it really depends. In theory the tap water is safe to drink, but not necessarily the pipes of most old buildings. In that case we also have a filter that cleans the tap water.
We've recently moved a little away from the city in a place built recently with new pipes. We still use the same water filter as before, but I can drink from the tap and I do it often especially at night when I just take water from the bathroom tap :p