Can you explain how federation works on protocol level?
Let's say we have lemmy instances A, B, C.
alice from A makes a post "Hello, world" to B. What happens? How is it processed on servers A, B, C and how do users from A, B, C receive her post?
You're viewing a single thread.
The easiest way to explain it is that the instances have no native ability to crawl other instances for communities or content. For all intents and purposes, a fresh Lemmy server is on an island and all other instances are their own island until someone builds a bridge to them.
The ability of an instance to receive content is dependent on the subscriptions users add to the database. Once the instance is aware of these other places it will begin checking them for updates and you'll see them regularly whether you interact with them or not.
This goes completely against what the average person is expecting and causes a lot of confusion.
Piefed instances now do have a form of this for instance admins to populate new instances.
Admins can:
-pull the lemmyverse data and subscribe to a bunch of communities at once
or
-target a single lemmy or mbin instance, get the list of communities that instance hosts, and subscribe to a bunch of communities on that instance.
Both have some tunable settings to allow admins control over how many communities are followed.
Its not an end-user thing, but it should help with setting up new instances and them not being so 'empty'.
edit: typo
That sounds like a much better implementation of community discovery.
instances have no native ability to crawl other instances for communities or content
That's not quite true. They don't do it automatically or routinely, but a user can cause a server to read a post from another server by putting its URL into the search box. This can be useful for an end user to manually address a federation glitch.
Here's a concrete example. I was trying to post a comment via lemmy.world, but lemmy.world sits behind Cloudflare, and Cloudflare flagged its content as potentially malicious. I then posted that comment via my own Mastodon server, but push federation to lemmy.world also failed, for the same reason. I could, however cause lemmy.world to pull the comment using the search.
This goes completely against what the average person is expecting and causes a lot of confusion.
But this is only true if the user looks at the All feed, correct?
But this is only true if the user looks at the All feed
It impacts what content is available to users at all. The All feed is just the visual representation of what's actively federating.
Let's say you join a new instance for whatever reason with no outside awareness of how the fediverse works. If you try to search the instance for "sportball" and get zero results the natural assumption is going to be that there are no communities and no interest in that topic. The user has no idea that lemmyserver5000.com has a sportball community with thousands of users because no one with those interests ever did the work to get the content flowing in a way that they could access it intuitively. It's a poor design IMO.
The reason I brought it up has more to do with starting a new instance or using a smaller instance. Communities that the instance isn't aware of (via someone previously subscribing) won't show up at all which causes places to appear non-existent or dead by default. Someone trying a federating website for the first time isn't going to know this, so to them, that's all the fediverse has to offer.
OK, I see that problem. In fact I remember having the same issue myself. (Presumably this will create a secondary confusion problem for "All" subscribers, who will see the content of their feed gradually expand without explanation as other users subscribe to other foreign servers, correct? Whatever, I don't care much about them, someone who subscribes to "All" apparently doesn't know what they want anyway!)
So the optimal solution here would be for each instance to preemptively connect to a whitelist of known foreign communities, perhaps? Or maybe each instance could regularly ping other servers in order to update its search database with popular communities.
It's a poor design if what you want to do is emulate a centralized social media service.
But maybe we should stop trying to do that.
Maybe.
But I'd counter that it's prohibitive to growth. People aren't used to turning up at a domain name only to find out 90% of the content can't be accessed without jumping through a bunch of hoops.
Does that mean that an "all" view is "onl"y all of the subscriptions/places people from my server have?
That's quite interesting.
And thanks!
Does that mean that an "all" view is "onl"y all of the subscriptions/places people from my server have?
Correct.
Note that many instances either have a bot subscribed to other communities to force federation, or use something like https://lemmy-federate.com
Note that many instances either have a bot subscribed to other communities to force federation, or use something like https://lemmy-federate.com/
FWIW this approach can be helpful but is flawed in its own ways.
Firstly, since not all instances participate you still aren't getting the "complete" fediverse so to speak. This becomes less of an issue as more instances join the bot program, but it's another step that roadblocks what should be an easy and organic process.
Secondly, the bot can pose a potential security risk depending on how it's configured. If you use it to federate in both directions you're subject to malicious actors spinning up tons of new communities on instances that don't restrict user registration. This will in turn hammer the database an instance uses for EVERYTHING and eventually causes slow downs, crashes, etc. The solution to this is to only seed your communities outwardly but if everyone only does that the bot is rather useless...
I don't have a solution for any of this, I'm just pointing out some rather frustrating problems this platform has in its current state.
Well, you can always defederate if an instance starts abusing it. Not that much different to the normal flow, really.
you can always defederate if an instance starts abusing it
Sure, but potentially after at least one of the instances subscribed to the bot goes down and someone realizes what's happening. It's incredibly easy to overwhelm a small server's database just by subscribing to a lot of communities the normal way. The difference here is potentially any instance federating the bot in both directions is susceptible to this.
Not that much different to the normal flow, really.
The impact across the fediverse vs just one instance would be the main difference. Plenty of people are using that bot having no real idea of what it's doing.
That's just a part of the learning process, IMO. My instance crashed many times, I've fixed it every time and now it's better than before. And I don't think I've had my last fuck up with the instance.
And that's fine for you, I'm not knocking the experimenting and learning process. That was the whole reason I spun up an instance myself.
What I'm saying is that to the other users that would be impacted by these things, it sucks. People are patient to a point but the fediverse has a lot of odd quirks that make it more difficult than it should be to use for a lot of people. Things have gotten better in the last year or so but it still feels like we're asking people to know more than they should have to just to figure out that Lemmy isn't empty. Many people will get frustrated and leave long before they start making excuses for a site they don't know anything about.
It's easy to sit around proclaiming that reddit sucks but the fact of the matter is that it's easy to use and everything they have to offer is covered under one domain. Again, I don't have the solution to these things for Lemmy, but we can't deny that this platform is harder to use than most and a lot of people aren't going to handle that well.
