Tap water safety splits the world roughly in three. In most of Western Europe, North America, Japan, Australia, New Zealand and a few other countries, tap water is perfectly drinkable. In much of Asia, Africa and Latin America, locals themselves drink filtered or bottled water, and visitors should absolutely do the same, since your gut lacks the local microbial tolerance even where residents manage fine. And there is a middle band where water is technically treated but old pipes make quality unreliable.
Where tap water is unsafe, extend the caution logically: skip ice unless you know it is made from purified water, which reputable restaurants and hotel chains generally use; brush your teeth with bottled or filtered water for at least your first weeks; and rinse fruit you will eat unpeeled with safe water. Cooking is more forgiving, since a proper rolling boil kills the pathogens that matter, so boiled tea, coffee and cooked food are fine.
Long-term expats usually install a home filter rather than buying endless bottles, which is cheaper and kinder to the environment. When in doubt in a new place, just watch whether locals drink from the tap.