Is Calm down rude?

To the person receiving those words, “calm down” the tone come across as condescending. Even if you try to sound very subdued and matter-of-fact. If youre the person saying “calm down” to someone whos visibly upset or distraught, youre subtly acting as if you have the upper hand.