If I'm standing in line at a store and someone is talking on their cell phone, I don't mind.   It's no more annoying or distracting than if they were talking to another person who was physically there.    This whole sort of nascent taboo is silly and baseless.

I suppose the only way it would make a difference whether the other is physically there or not is if one would rather hear two sides of the conversation instead of one side, because one enjoys spying on strangers' conversations.