It's also all in the phrasing. "Don't worry" uttered by a Fayoumis child is going to make my mother (chicken people: imagine a diminutive Silver Penciled Wyandotte) start worrying hardest.
This is especially true when "Don't worry!" immediately follows up a piece of neutral to pleasant news, because it implies that there's something to worry about. I'm sure that anyone who's ever heard me talking to my mother can imagine the smile I've got when I say it, and the tone of voice... for that matter, anyone who's ever seen the Azzgrin doesn't have to imagine the smile...