And he said to them, "Thus says the LORD God of Israel, 'Put your sword on your side each of you, and go to and fro from gate to gate throughout the camp, and each of you kill his brother and his companion and his neighbor.'"
Moses returns after the people have rebelled. As a sign of the seriousness of their sin, and of the anger of God at their direct offence against him, God commands the Levites (those who hadn't worshipped the calf) to go forth and slay the people who were still in the grip of their folly.
They went through and killed thousands as judgement for their sin and to help shock the rest of the people back into their right minds. It seems very harsh in today's world, but the events were of such impact that the message not to worship false idols has been preserved for us today.
If God had merely "slapped them on the wrist with a wet bus ticket" (punishment of zero impact) there would be no lasting effect. We wouldn't know just how much God hates such evil and we would feel that we could get away with it too. Instead, all who read this account should develop a healthy concern from doing so, that they would remain faithful to the true God.