Psalm 51 is called "A prayer of repentance".
Its heading says it is a psalm of David from when Nathan the prophet reprimanded him after he had gone in unto Bathsheba. It's a cry of despair, full of deep repentance and the plea to be cleansed from the sins he has committed. The psalm shows a clear pattern through its words; by openly acknowledging the sin the speaker gets more understanding about the way of the Lord and about life. The psalm also shows a growing assurance and trust in the Lord and his acceptance of this cry.
This psalm, at a deeper level, also describes Jesus’ prayer to overcome the sins of his human nature which he took upon himself in becoming human. The Lord, unlike David, didn't give into temptations – John 8:46 has the Lord asking, “Which of you convicts me of sin?” – so His prayer is to have the strength to withstand the tendencies inherent in his human nature. The Lord was severely tempted in his humanity.
For us, this psalm is one we can relate to and feel is our own prayer when we see the ways in which our thoughts rise up out of our self-love or our proprium (our selfhood). True repentance is not remorse for some wrong we have done, although this can be part of our spiritual progress; it has far more to do with us giving our attention to thoughts and moods which suddenly rise up and plague our intention to be true to the Lord. (Divine Providence 121, 122)
This psalm mentions things which have to do with the Lord and things which have to do with us. Words like "mercy, loving kindness, just, judge, washing, presence, and good pleasure" speak about the Lord, while "transgressions, iniquities, evils, and sins" speak about us. The overall idea is that the Lord longs to give to us what is his, so that we know he is there with us and that he completely understands our human frailty. In being what we are like in ourselves, we will keep going astray. We continually need to bring ourselves before the Lord to be restored.
The Lord’s mercy is the His wish to save us and regenerate us, so that we will be drawn - and wish to be drawn - to heaven, which is where the Lord is. Mercy is not instant in its activity, but it is perpetual. (See Heaven and Hell 522)
‘Washing’ is another divine quality, but we need to be careful in understanding it. It is not the Lord washing our wrongs away as if they had never been done. Our wrongs may need to be brought back to us, not to taunt us, but to remind us of our need of the Lord and our wish to do no more wrongdoing. The true idea of washing is for us to ‘wash ourselves’ and to feel we have been washed, so that we can go forward stronger than we were. (See Divine Providence 151 and Apocalypse Explained 475.5)
‘Transgressions’ are evils we might do which come from a perverted understanding. (Arcana Caelestia 9156)
‘Iniquities’ are evils we do intentionally because we have twisted our thinking to justify doing them (Apocalypse Explained 475)
‘Evils’ are what we do because we are born with an imperfect human nature. (See The New Jerusalem and its Heavenly Doctrine 79 near the end)
‘Sins’ are evils we do intentionally because a love of evil has taken hold of us. (Arcana Caelestia 5726)
The phrase “Purge me with hyssop” means being cleansed by external truths or truths which we see and which confront us, demanding that we correlate our life to them. (Arcana Caelestia 7918)
The phrase “Let the bones which you have broken, rejoice” is describing the appearance to us that during temptation the Lord has broken us, whereas the very opposite is true, that the Lord is defending us and being our bones. So in truth, we may rejoice. ‘Bones’ spiritually mean truths because bones support the body and truths the spirit. (Arcana Caelestia 3812.8)
Overall, this is a psalm which, for us, can help us move on from seeing the number and range of our imperfections through to understanding, appreciating and acting on the truth that the Lord can and will lead us to restoration and wholeness if we go to Him.