And he gave to Moses, when he had finished speaking with him on Mount Sinai, the two tablets of the testimony, tablets of stone, written with the finger of God.
God has always considered his written word to be important. The most important part would be that which was written by God's own hand. Here we have the commandments which he gave to Moses, written with his finger in stone.
The weight of the stone tablets was also a reminder of sorts. Not so much of the burden of those commands but of the importance and authority (a different sort of weight) of what was written.
To be written in stone is to be very difficult to erase. Consider how hard it is to remove painted graffiti from a wooden desk in comparison to something scratched into the surface. Many a person has come back to such graffiti after many years and sees their message unchanged. Now consider how much more lasting a message written in stone would be!
God goes further though. In his New Testament covenant, God writes on our hearts so that we cannot ignore his will for us.