First, I like to correct the mistake in your question:
you cannot use "calm ponder", because "ponder" can only be used as verb. Therefore, the correct way shall be "calmly ponder". By the way, you can sometimes see "calm Ponder", here "Ponder" is people's name, so it capitalized "P". Therefore, your question shall be the difference between
" calmly ponder " and " calm down ".
"ponder" is more in the brain (logical thinking), "calm down" is more of psychological or emotional.
He calmly pondered the question before he answered.
The team calmly pondered their chances of success.
We calmly pondered whether we could afford the trip.
After the fight, both men need to calm down (cool off).
It took a while for things to calm down (settle down) again.
After a heated argument, both parties have finally calmed down.
The above sentences, you CANNOT interchange " calmly ponder" with "calm down" or vice versa.