Out of the mainstream doesn't necessary suggest wrongness. However, it also doesn't suggest not so.
More importantly, it suggests that the majority do not consider it normal, or correct. Thus, relatively speaking, it is not correct. So, using the phrase "out of the mainstream" may sounds not accusing, but it is ... , relatively speaking.
Absolutely speaking, wrongness or correctness of something is regardless of if it is out of the mainstream or not. E.g. the Sun was once thought to be revolving around the Earth.
Yet, statistically and evidently speaking, most out of the mainstream stuffs are wrong. This also implies that some of them are right.
Basically, being out of the mainstream, or not, doesn't imply wrongness nor correctness.
LGBT is out of the mainstream, and I say it is also wrong.