Star Trek celebrates the diversity of humanity. The extremes of genetic engineering and (on the other side of the spectrum, perhaps) the Borg are symbolic of the corruption of that diversity.
For an in-universe explanation, I suppose you could just look at the degree to which cybernetics are tolerated. Rutherford-level cybernetics? No problem! Borg Queen-level cybernetics? Helm, warp nine, full reverse!
I would argue the bynars are not doing it voluntarily either though. According to phlox in ENT, right after birth a surgeon removes a certain part of the newborn‘s brain, replacing and connecting it with their central computer.
And I would argue a literal newborn cannot give consent to this. The only difference to the Borg appears to be the larger degree of individuality, though that seems questionable too, considering how interlinked they are in the TNG episode.
Prime directive. The Federation can apply such ideals to their own citizens and to foreigners who ask for help, but not to foreign citizens who don't ask. The Borg try to assimilate Federation citizens, so the Federation has a valid cassus belli for war. The Bynars aren't attacking any other species, so the prime directive doesn't apply.
Generally even ex-Borg, like Seven of Nine, aren’t disliked or mistrusted because of their remaining cybernetic implants. It’s almost always about their connection too, and implied complicity in, horrors and genocide.