I don’t know that that’s true, maybe you just met some shitty breakcore fans. I’m not a fan but I do listen to some idm music which can be breakcore-adjacent at times.
I'm something of a breakcore fan myself, I can't say there aren't individualists amongst the fans or artists, of course, but it's like more genres in that regard. (Also depends on what you mean by breakcore I think cause there's quite the dispute between oldschool breakcore fans - stuff like Venetian Snares, Alec Empire's "The Destroyer", etc. - and new music being labelled 'breakcore' but which is mostly kinda shitty drum'n'bass with creepy anime cover art).