Our five senses correspond to spiritual affections, or the attraction we feel toward spiritual things. The sense of taste, according to the Writings, corresponds to the affection for knowing, or the affection for being wise. This makes sense; we often immediately know and judge the quality of a thing by its taste. When people taste things in the Bible, it means they are seeking wisdom and enlightenment, and when food tastes good it shows the delight they feel in being wise.

(References: Arcana Coelestia 4404, 4792, 4793 [1-3], 4794, 8522)