This is entirely destination-specific, so the honest framework is: find out which category your country falls into, because the world splits roughly three ways. In some regions, tap water in cities is excellent and drinking it is completely normal. In others, the water leaving the treatment plant is fine but aging pipes, rooftop tanks, and intermittent supply make quality unreliable by the time it reaches the faucet, so residents themselves drink filtered or bottled water. And in some places the water simply isn't potable for anyone.
Useful signals: what locals do is the single best indicator; if every kitchen has a filter unit or a delivery of large bottles, follow suit. Hotels in non-potable countries usually provide bottled water, and restaurants aimed at foreigners use purified water and ice.
If you're settling long term in a filter country, a proper home filtration unit quickly beats buying bottles on cost, waste, and convenience. Newcomers' stomachs also sometimes react to different water minerals even where it's safe, so easing in isn't paranoia.