Tap water safety is genuinely country-specific, so this is one topic where you must check for your destination rather than generalize. In much of Europe, North America, Japan, Australia, and similar places, tap water is treated to drinking standard and locals drink it straight from the tap without a second thought. In many other regions, locals themselves rely on filtered or bottled water, and following their lead is the sensible move.
A useful signal is simply watching what residents do. If every household has a filter unit or a water delivery jug, take the hint. Even where water is officially safe, old building pipes can affect taste and quality, so many people use a basic filter jug for comfort.
One thing newcomers often forget: even perfectly safe water can unsettle your stomach for a week or two simply because its mineral profile differs from what your body knows. That is adjustment, not contamination.