Goats, as in Leviticus 16:21, 22, signify faith. Because man is regenerated by the Lord by the truth of faith, and consequently, his sins are removed and cast into hell, therefore it is said that "Aaron made the goat bear upon him all the iniquities of the children of Israel unto the land of separation, or into the wilderness." From correspondences, a goat signifies the natural man. The goat which was sacrificed, as in Leviticus 16:5-10, signifies the natural man regarding a part purified, and the goat which was sent into the wilderness regarding the natural man not purified.

(Odkazy: Apocalypse Explained 730)

715. Since the most ancient people knew and when in self-abasement acknowledged that they were nothing but beasts and wild animals, and that the Lord alone enabled them to be human beings, they not only used to liken whatever resided with themselves to beasts and birds but also called them such. Things of the will they compared to beasts and called them beasts, and those of the understanding they compared to birds and called them birds. They differentiated however between good affections and evil affections. Good affections they compared to lambs, sheep, kids, he-goats, young she-goats, rams, young bulls, and oxen, because they were good and gentle creatures, and also because they had a use in life in that they could be eaten and men could clothe themselves with their skins and wool. These are chiefly the clean beasts. But the evil and savage ones, which also have no use in life, are unclean beasts.