In spirit I agree with you, but I can imagine a scenario in which someone ended up with a group of people who aren't explicitly evil but do exploit employees and end up helping their "friend" who doesn't exploit people to become a billionaire, either to ease their own conscience or for any number of selfish reasons. The person ends up as a billionaire and doesn't get rid of it in their life for whatever reasons (people usually like to appease people they know personally)
It's mostly just a thought experiment, the existence of a good billionaire, but it's technically possible for sure, even if not actually possible.
It's interesting as a thought experiment because there's no real world example of this. Which I guess is the gotcha OP was going for, but kinda fumbled.
Arguably hoarding the wealth for yourself (and even your immediate family), never mind how you accumulated it, is still not "good". It's indirectly oppressive to collect a bunch of money, while many suffer, and say "noone else is touching this, it's mine".
Yeah I still find it hard to digest that someone with a conscience actually made that much money in the first place. I'd love to see how he arrived at this decision, and if he could convince others too.