2y ago

Viewing lemmy posts by all tends to be dominated by a few communities

I've noticed that there are a few communities that tend to dominate when viewing all. Some days it gets to where looking at all isn't very different than just looking at Memes@lemmy.ml or 196@lemmy.blahaj.zone.

Before someone says "you can just block communities you don't want to see," it's not that I never want to see them, it's that I want to be able to have a view that shows me what is new and popular in a wide variety of communities. I appreciate seeing a few good memes in my feed. The problem is when that's all I see. Changing the sort from active to hot or top x days doesn't have much effect on which communities dominate, so that isn't the solution either.

"You can just subscribe to communities you like". True, but that has the effect of narrowing what I see. I'd like a view that showed me new things I never thought to subscribe to.

Lemmy devs - if you are reading this - it would be nice to have a feed that limited the number of posts showing up from any particular community. It could be a simple cutoff of 2 or 3 posts, or maybe some sort of weighting function to cause additional posts from the same community to appear lower in the sort order for that feed.

I'd love to hear what devs and other users think about this.

Edit: To everyone saying "just sort be new" - yes, that has its uses, but it only solves part of the problem. I'd like a feed that shows me what is new and popular, but from more than just one or two communities.

104 comments

I read something the other day that they’re working on a new sorting algorithm that would limit it to the top few posts from each community within a given time frame. Specifically to address this issue.
No idea on timeframe or further details, or if I even summarized it correctly lol.
- Yes, they are working on a so called "best" sorting: https://github.com/LemmyNet/lemmy/pull/3378
  And yes, they are working on "multi": https://github.com/LemmyNet/lemmy/issues/818

This has already been mentioned a few times. Just find the relevant issue on GitHub and give it an upvote.
The rank of a post in the aggregated feed should be inversely proportional to the size of the community #1026
Weighted Community Subscription for Customized Feed #3518
Balance Instance Scores Based on Monthly Active Users #3642
- There's also a pull request https://github.com/LemmyNet/lemmy/pull/3378
  
  Lgtm!
  
  There's also a comment by Nutomic (a main dev) which shows they are aware of threads like this:
  There are a lot of user complaints that posts from the same few large communities are always at the top.

Idk it seems like a problem that will sort itself out as Lemmy grows, and artificially limiting how many posts from a community can reach the front page seems like a suboptimal solution that’s going to have unintended consequences down the line.
- I'd agree if the coms in question were niche but so far in my experience they never are. When it comes to communities dominating All, it's always bottom-of-the-barrel memes and porn. The posts from non-garbage communities that show up are usually by themselves
  
  Everyone’s definition of bottom of the barrel will be different, and nobody’s personal content preferences should be forced on the community as a whole. If you really dislike those communities that much you can block them.
- Doesn't Reddit totally weight by size of community tho? Not saying we should just ape the old site but I suspect it's actually necessary for smaller communities to grow naturally.
  
  I think there are better ways to highlight smaller communities and grow them more organically, like a community dedicated to new and small communities (sorry if I fucked up that link, I'm new here) could highlight a new community each day worthy of our attention. Reddit used to have a subreddit of the day.
  Right now the number one thing federated social media needs is just more users. I worry they'll feel discouraged if they stop seeing the content that gets the most upvotes right now.

If a community has grown a lot, unfortunately it takes up all the posts on the homepage. If I were a developer, I would sort by weighted success. For example, if the "x" community has 1000 subscribers and the post gets 100 likes, it has ten percent success. If the other 'y' community has 100 people and gets 11 likes, it has 11 percent success. This overrides the post in community x because the post in community y is more successful. This is the logic of 'weighted success'. With this logic, a better ranking formula can be created.
- I figured that's what Id see sorting by New Comments but that hasn't actually been the case. Any new post or post with a new comment gets pushed up to the top immediately and doesn't even take into account votes, but even refreshing the page all day doesn't catch a lot of small communities unless they are also attracting attention.
- The thing is, a 10% upvoted post on a 10,000 people community is more popular than a 90% upvoted post on a 1000 people community - those 10,000 people in the former community are 10,000 people interested enough in that kind of thing to subscribe, whilst only 1000 people are interested enough in the other kind of thing.
  So it does make sense to put the former higher up in the global page when sorted by popularity because globally that post was more popular.
  However I do think there should be someway to as a user push down posts from certain communities without outright blocking the whole thing: maybe som throttling-down based on the rate of posts per time (i.e. the upvote threshold for posts from a community to come out in All depends on the number of otherwise qualifying posts in the last X days/hours, thus explicitly targetting the "flooding with posts" itself) rather than the straight count of upvotes or the proportion of upvotes that you suggest.
  That said in the meanwhile I'm really tempted to block the more generic meme communities.
  
  General popularity is not a good metric IMO. If I like a community, then it shouldn't matter if a million people like or it's only me and my cousin. If the community likes the content, I want to see it.
  It's trust between the members of a community.
  However, weighted sorting is not a solution too, upvotes counts are not linear. Maybe, quantile sort?

Another, related issue to this is that I find the three frontpage categories (All/Local/Subscribed) to be too blunt of a sorting method.
I would really love if we had the option to create subcategories to our Subscribed feed, so we could group related communities and end up with different frontpages for News/Sports/Memes etc.
- I too would like a 'custom feed' functionality.
- Being able to filter by language would also be super helpful.
  Im sure the Swedish gaming community is great, some lovely Svens and Ingas but I dont speak swedish. Being able to autofilter subs by default language would be awesome.
- https://github.com/LemmyNet/lemmy/issues/818
- That's a really great idea!
- Isn't that what reddit's "multi Reddit" was all about?
  
  Yes. It's been discussed on the Lemmy GitHub so it's in the works, but that's all I know.
  
  I always hated multi-reddit because it never worked the way I wanted it to.
  I wanted it to give me an r/all setup for a specific sets of subreddits. What it did was just give posts from every subreddit in the list, including ones that hadn't had posts in months. It was just trash to try to use.

I generally use “New Comments” as my sorting and it’s a little bit better, but still the same spammy communities end up on top
- Memes, memes as far at the eye can see
  
  So many memes
  I don’t hate it yet, since they’re the memes of my childhood/teen years, so tbh I find them funnier than the post 2016/18 bullshit

I'd love some kind of per community bias adjustment even for subscribed communities. Like, I don't really want to remove them cause memes are great, but because !memes@lemmy.world and 196 post so often my subscribed feed is pretty dominated by them no matter how I sort it.
For "All" some kind of adjustment based on subscribers makes sense, but I don't even know if that's possible given the way Lemmy works. Maybe a "show me less" button that moves the same bias adjustment just for communities you're not subscribed to?

Note that this is one of the advantages of having an account on a smaller and/or more focused instance or having multiple accounts.
All "Alls" are not the same. Actually, the "All" displayed on a given instance is everything local to that instance and everything from other instances to which someone on that instance has subscribed. So if nobody from that instance has subscribed to a particular community on another instance, then for all intents and purposes, it just doesn't exist. Even on "All".
Granted that it's somewhat unlikely for an instance to not have someone somewhere along the way subscribe to some notably popular community, it is possible, and the smaller and more focused the instance is, the more likely it is.
- Thanks for teaching me. I can see that causing challenges down the road.
  For example, I'm always on the lookout for all things quilting. If someone names their quilting community "Fabric Hordes" (not impossible, just look at phenomenon like r/animetitties) it wouldn't come up in my explicit searches, and is very unlikely to be sought out or found by others in my instance.
  
  Right, but there are lots of ways around that.
  There's already been a fair amount of demand for some method to group communities by interest, so it's essentially guaranteed that somebody is going to provide some way to do that, and likely multiple somebodies are going to figure out multiple ways.
- Ah. I had noticed that lemmy.world's all seemed different than lemm.ee's, which in turn was also different than kbin. That's good to know
- So, would it be wise and helpful for a mod or bot per instance to subscribe to as many communities as possible to help the instance's feed?
  
  Hell no.
  How does that "help" their feed? What possible benefit could there be in using a bot to subscribe willy-nilly to every community out there, no matter how shitty it is?
  I mean - if some instance owner wants to do that, that's their choice, and I guess there are people out there who would like the resulting instance filled to the brim with every bit of garbage that exists anywhere in the fediverse, so it's safe to assume that somebody will do it sooner or later. Personally, I think the idea is repulsive though.
  Maybe I wasn't clear enough in that other post - I think that the fact that each instance has a different "All" depending on what the members there have subscribed to is a good thing. It means that different instances have different feels, and over time, as they get more established, that's going to be even more the case.
  So for instance, a notably tech-oriented instance is going to end up displaying pretty much every tech-oriented community on the fediverse on its All because somebody on the instance will have subscribed to it, pretty much no matter what it is, AND at the same time, all of the stuff nobody's interested in just won't be there at all, because nobody bothered to subscribe to it in the first place.
  Granted that that's not going to appeal to people who want to be flooded with every bit of garbage on the entire fediverse when they click All, but they can just go away and sign up with some other instance that gives them what they want. Which I'm sure is exactly what the people who sought out a tech-oriented instance in the first place would prefer anyway.

This was an issue Reddit used to have circa 2015. Front page was all League of Legends posts, and then it was all The_Donald posts. Then Reddit screwed up their algorithm and it was literally 100% The_Donald posts.
- I don't need another t_d vietnam flashbacks man, don't jinx it.
- Fun fact. r/leagueoflegends was one of the most filtered subreddits from r/all at one point.
- The important takeaway here is that it took a long time before it was actually good. They had to try a bunch of different sorting algorithms before they found one that really worked and let you see your small subs just as much as your big ones.
  It might take a while here too unfortunately.

I don't know about anyone else's experience, but I've noticed that any time I click into a post view and then back out, I'm taken to the first page of posts, no matter if I was 2 or 3 pages on. If the redirect respected where it came from instead of going to home, that could reduce the impact of post-order sorting. Also if the list pager had more options than 'prev' and 'next' (maybe a few numbered pages between or beyond) I could get beyond that 3rd page without getting there feeling like an illustration of the schlemiel the painter's problem
- I found a workaround to that problem, just do a page refresh after going back from a post to the list of posts.
- The first page problem happens to me, too.

Fyi the devs aren't reading this (and probably won't be before long, since they are busy just coding a lot of features). Best place to ask for this is on the issue tracker (first check if it hasn't been asked before), even better implement it yourself if you can!

I'm right there with you - I'm making an app called flemmy, and I have 12 more tasks on my list before I'm putting it on the play store - should be this weekend at the latest. Iphone build shouldn't be that far behind. I can also make a desktop build if anyone wants it, but right now have no intentions to host a site myself - I strongly feel the data the app collects shouldn't leave your device
Version 1 is about creating something close to the Reddit apps I used to use, and it's there - just needs a little more polish (and to let you post... I'm more of a commenter, so I forgot that was a thing for an embarrassingly long time)
I can support all sorts of filters, from keywords to hiding specific posts to "snoozing" communities. I can also save your place when you change sort methods or accounts - it's what I always wanted for Reddit.
Also, I have support for redirecting links - Twitter to nitter, YouTube to pipe, etc.
Version 2 is going to focus on your feed. Already I connect to multiple servers (it's a real headache, but the foundation is there), so next is stitching feeds together and custom feed algorithms. What you mention is at the top of my list - a way to tweak the feed based on all sorts of filters.
Ideally, I want it to adapt to you - using upvotes and comments to tweak your feed. All on your phone - it's amazing what you can do on a phone when you're not interested in data collection
I just made !flemmy@Lemmy.world, I'll post some screenshots when I need a longer break
- Awesome, that you develop tools for lemmy! Thank you.
  A question: Did you consider to publish your app on the F-Droid-Store? I only use apps from there and the only usable app for lemmy is jerboa. An alternative to that would be awesome, as it has quite a lot of problems.

Browse by New. Bam! Problem solved! Stop being slaves of the Algo. Actively search for stuff you like while also actively block those you don't like.
- Newest comments works best for me as it gives a nice mix of big and small posts

Better solution: don't use Lemmy as your feed, use a feed reader (RSS). There are per-channel feeds that you can sort and filter using parameters.
Doing things this way will also help create the open web we all want to see, where "forum" is not a synonym for "Reddit" or "Lemmy", where you can also follow the goings-on in other places and not miss anything.
- Yeah, I really think it's important to not see Lemmy as one singular community, or a lot of important use cases will go ignored.

yes! yes! yes! I have the same problem.
My suggestion is to add a view with subscriptions ordered by the selected criterion (i.e. new/active/hot/top) and below each community there is vertical list with posts from that community sorted.
This would allow see what's up with the communities we follow and then jump in those communities if we find anything interesting.

I see a wider variety of active topics by sorting All by New Comments.
- Hmm I might need to try that with the front-page. I had noticed it seemed to just be the same topics daily.

A quick, but a little dirty solution for this, would be communities having “tags” in their metadata. This wouldn’t prevent spam, or an accumulation of four trillion tags, but you could easily add “only these tags,” or “not these tags,” to any feed. User objects have metadata that is used like this (as the “bot” flag) already. I’m just familiar enough with the code to know it wouldn’t be a slam dunk, but it’s also not a breaking change or re-write!
- Tags would be great, and a much better way of controlling your feed than blocking communities or instances. Just because I usually don't want to see memes or shitposts doesn't mean I'm never in the mood for them. It could also potentially help people who don't want porn in their feed but want to keep non-porn NSFW visible.
- Maybe they could even integrate with Mastodon's hashtags

I like to use "Hot" instead of active as it seems to fetch posts from a more diverse set of communities
I've also heard that view by new comments is solid as well
- Top from the past 6 or 12 hours works better than Hot IMO. It feels more like the traditional reddit front page.
  
  Thanks for the suggestions, I'll try that
- This you? https://lemmy.fmhy.ml/comment/1180416 it's a direct copy/paste of that comment plus one sentence
  
  I'm really confused lol, that is not me I have this account and one on beehaw.
  How strange I had a different comment but I edited it and it changed to that one ...
- Btw, It's already possible to block all bot posts
  
  Ohhh is that what unchecking the "show bot account" does I thought it removed the badge indicating a user was a bot account. Thanks!

I think there should still be a strictly chronological feed that isn't algorithmic, but blocking the bots that import a lot of low quality content from reddit would be easier than blocking the individual subs.
- I'm definitely not suggesting that any of the existing feeds need to be done away with. They all have their uses.
- Can't you just sort by new?
  
  That just gives you what's new. I'm looking for what is new and popular. I usually browse by top 6 hours.
  
  I thought OP was talking about sorting by new but I was mistaken.

@dryguy The algorithm needs to be improved. It needs to adjust for the number of boosts/favorites that large communities get. You should be seeing posts from all of the subs you are subscribed too and not just the most popular ones.

I'm with you 100%. I did find that searching by hot comes out a little better. But I think I'm finally ready to just block the meme and the rule subs.
Also there's just not as much content here as. Reddit. That will eventually change,

Sorting by new works pretty well for me. There's still the couple of them showing up, but there's a lot more variety to what I see that way.

My people
- Rule
  
  Rule

Most communities I'm in just seem to post about Reddit and its soooo boring, like get a life and make original content

I think this problem is relegated to being registered to larger instances. On one hand you get lots to choose from when it comes to your local feed, similar to how Mastodon works, on the other hand you have the problem you just mentioned. I know you bring up the problem of narrowing what you see, but if you're going to be on a big instance, there really is no other way than to subscribe to those communities you have a specific interest in. I'm on a smaller instance and I just take the time to go through communities that I may have an even fleeting interest in and subscribe to them, avoiding larger generic communities. This is the same strategy I employ when it comes to my news feed as well. I only subscribe to RSS feeds from sites I have an interest in.

I fully agree. On reddit i would use the all frontpage to find new communities. Here it doesnt work.

As more people join these kinds of things will change and evolve. Hopefully the site infrastructure will adapt to it as well.

What if you make an alt account that blocks big prominent communities when sorting through /all so you can view the top posts from smaller communities?
- But then you have to change accounts, which I understand isn't difficult, though it's still something that needs to be done proactively by the user. Would be nice to just have a better content ranking algorithm that gives smaller communities a chance.

Someone from reddit should quit and recreate their algorithm for front page.
- No, thanks. Their front page is rage baity shit

104 comments