All this could happen because you came out of the closet. So it's no surprise that people would keep their sexuality hidden. They are afraid of not being accepted by their family, their friends, or anyone that they may cross paths with. But it's unhealthy for them to keep their secret hidden from others, especially from those that they care about.
So how does coming out of the closet relate to Wicca? Being Pagan isn't too different from being homosexual; it's not mainstream, and that can lead to a lot of misinterpretation and discrimination. People don't understand what it means to be Pagan or gay, or they may simply hate Pagans or gays because they are different and are "outsiders." I've known both Pagans and gays who have been attacked or shunned because they are open about their respective religion or sexuality.
Which is why the phrase "coming out of the broom closet" has been used to describe someone who hasn't come out and openly said that they are Pagan (it combines the original phrase on sexuality with the witches-on-brooms idea). I myself am in the broom closet, although I have told a few people about being Wiccan. Just simply being "different" than everyone else is difficult; if you keep it secret then it'll burn inside of you until you tell someone, but if you are open about it then you need to be careful and prepare yourself for people's reaction and the stigma they've associated with what makes you "different." Unless you've gone through it before, you don't know what it's like.