Goats, as in Leviticus 16:21, 22, signify faith. Because man is regenerated by the Lord by the truth of faith, and consequently, his sins are removed and cast into hell, therefore it is said that "Aaron made the goat bear upon him all the iniquities of the children of Israel unto the land of separation, or into the wilderness." From correspondences, a goat signifies the natural man. The goat which was sacrificed, as in Leviticus 16:5-10, signifies the natural man regarding a part purified, and the goat which was sent into the wilderness regarding the natural man not purified.

(References: Apocalypse Explained 730)

714. That 'every clean beast' means affections for good is clear from what has been stated and shown already about beasts in 45, 46, 142, 143, 246. The reason affections are meant by this is that a human being regarded in himself and his proprium is no different from a beast. His [five] senses are very similar, appetites very similar, natural desires very similar, as well as all his affections being very similar. His good and even his best loves are very similar, such as loving companions of his own species, loving his offspring, and loving his married partner. To this extent there is no difference at all. He is a human being and superior to beasts however in that he has an inner life which neither does nor can exist with beasts. That life is the life of faith and love deriving from the Lord, and unless that life were present in the smallest characteristic which a human being shares in common with beasts he would not be in any way different. Take just one example - love towards companions. If he were to love them merely for his own sake without anything more heavenly or more divine within such love, he could not then be called a human being, for the same is true of beasts. Similarly with every other example that could be taken. Consequently if the life of love from the Lord were not present in his will, and the life of faith from the Lord in his understanding, he would in no way be human. Through the life he has from the Lord he lives on after death, for the Lord takes him to Himself; and in this way he can be with angels in His heaven and live for ever. And even when someone lives like a wild animal, loving nothing whatever but himself and things regarding himself, still the Lord's mercy, being Divine and Infinite, is so great that He does not abandon him but by means of angels continually breathes His life into him. And even though someone does not receive that life otherwise than as a wild animal the Lord still makes it possible for him to think, reflect, and understand whether something in private or public life, or something worldly or bodily is good or bad, and so whether it is true or false.