Feelings of joy and rejoicing flow from our affections, not from our thoughts. Some people might argue that that's not true, that you can rejoice over learning a new idea or having a new thought -- but really you're rejoicing because you have an affection for that idea, or even simply an affection for growing wise (something very strong in men in particular). So when the Bible talks about rejoicing, exulting, or feeling joy, it is talking about an excitement of the affections -- either good or evil. Often the Bible offers this idea in pairings -- "exult and rejoice," for instance, or "rejoice and be glad." Generally this means that the spiritual event is stirring both the affection we have for truth, for thoughts, and the affection we have for doing what is good.