That's what I fucking did. Reddit was shit not because of karma, but because of hivemind and owners. Lemmy is not protected from that either, but at least users potentially have more power here
You can, but there's absolutely no reason to think the community population will ignore it, once it has its own value the karma whores will arise. People literally sell high karma accounts for real money. You can't ignore the karma farmers when they are all around you and you can't ignore the fact that it shapes real user behavior toward hive-mind dribble.
Keep karma to the Gods, it is not for mortals to toy with.