Conversation

Notices

1. Embed this notice
   Christine Lemmer-Webber (cwebber@octodon.social)'s status on Saturday, 09-Nov-2024 01:19:26 JST Christine Lemmer-Webber

   • Clairement crevée

   @Claire @eramdam Well I mean, Bluesky's ATproto recently has made the rounds as the "more decentralized" protocol over ActivityPub, and clearly a lot of people on my Bluesky feed right now seem to think it's more decentralized, and it does have one major improvement which is that it uses content-addressed posts (I don't think the DID methods currently actually do their job though the goal is good, I can expand on that at a future time)

   Which is what's leading me to look more into it, in which ways is it more decentralized in practice? Is it even particularly decentralized in practice? I'm trying to gain a technical understanding.

   • clacke repeated this.
   • Embed this notice
     Christine Lemmer-Webber (cwebber@octodon.social)'s status on Saturday, 09-Nov-2024 01:24:38 JST Christine Lemmer-Webber

     in reply to

     • Clairement crevée

     @Claire @eramdam the DID stuff I would like to piece apart but that is a thread of its own

     I was involved in early DID standardization work. I ultimately don't think this approach to DIDs solves what it thinks it's solving, but that's for later

   • Embed this notice
     Clairement crevée (claire@social.sitedethib.com)'s status on Saturday, 09-Nov-2024 01:24:43 JST Clairement crevée

     in reply to

     @cwebber @eramdam disparate repos of content-addressed objects that are truly decentralized (up to DID schemes), gathered by relays/AppViews which do not have to be centralized (but have to be huge nodes to be useful for Bluesky's purposes of a public microblogging thing)

     the web DID scheme is about as decentralized as current-world ActivityPub i guess (with the nice addition of it being easier to move some details around), and the PLC one is… a very centralized placeholder?

   • Embed this notice
     bryan newbold (bnewbold@social.coop)'s status on Saturday, 09-Nov-2024 04:15:51 JST bryan newbold

     in reply to

     • Clairement crevée

     @cwebber @Claire @eramdam I have a longer post on this, and our progress, on my personal blog:
     https://bnewbold.net/2024/atproto_progress/

     clacke likes this.
   • Embed this notice
     bryan newbold (bnewbold@social.coop)'s status on Saturday, 09-Nov-2024 04:15:56 JST bryan newbold

     in reply to

     • Clairement crevée

     @cwebber @Claire @eramdam
     unbundling and composition: ability to swap components w/o swapping full stack.

     credible exit: ability for new interoperable providers to enter network. public data must be accessible w/o permission, and schemas/APIs declared

     control identity: whole network/proto doesn't need to be individualist, but identity is personal

     easy to build new apps: don't build for the old modalities, enable new modalities. accomodate opinionated devs.

     clacke likes this.
     Christine Lemmer-Webber repeated this.
   • Embed this notice
     bryan newbold (bnewbold@social.coop)'s status on Saturday, 09-Nov-2024 04:16:02 JST bryan newbold

     in reply to

     • Clairement crevée

     @cwebber @Claire @eramdam don't think the goal w/ atproto is to be "more decentralized" in the abstract. we (team) had worked on SSB and dat, which were radically decentralized/p2p but hard to work with and grow. would not supplant "the platforms".

     atproto came out of identifying the *minimum* necessary decentralization properties, and ensuring those are strongly locked in. we basically settled on:

     clacke likes this.
     Christine Lemmer-Webber repeated this.
   • Embed this notice
     bryan newbold (bnewbold@social.coop)'s status on Saturday, 09-Nov-2024 17:20:13 JST bryan newbold

     in reply to

     • Clairement crevée

     @cwebber @Claire @eramdam
     I think a bunch about this post about the history of mp3 piracy and "minimum viable decentralization":
     https://web.archive.org/web/20180725200137/https://medium.com/@jbackus/resistant-protocols-how-decentralization-evolves-2f9538832ada

     (though it wasn't directly influential on atproto design, and Backus has since pulled the post)

   • Embed this notice
     Nemo_bis 🌈 (nemobis@mamot.fr)'s status on Saturday, 09-Nov-2024 17:20:23 JST Nemo_bis 🌈

     in reply to

     • bryan newbold

     @bnewbold Thank you Bryan, this is extremely helpful!

     I hope to see multiple #BlueSky relays soon (incentives unclear: https://neuromatch.social/@jonny/113364719373034539). I worry about the climate costs of many full copies.

     One accidental design feature in #Mastodon is how an instance serves as "relay" with a cache of posts and media caused haphazardly by whatever happens to federate. This is messy but flexible. https://masto.host/re-mastodon-media-storage/ Instances can share deduplicated object storage. https://jortage.com/

     #FediMeta

   • Embed this notice
     Nemo_bis 🌈 (nemobis@mamot.fr)'s status on Saturday, 09-Nov-2024 18:40:17 JST Nemo_bis 🌈

     in reply to

     • Luca Sironi

     @luca It's the opposite. The amount of media to be cached per user grows less than linearly with the amount of users on an instance. A single-user instance has the highest level of storage waste. An instance with a million users (like mastodon.social) already mirrors pretty much everything so they would have little benefit from deduplication.

     You don't need to guess, we already have actual numbers. Check the figures of Jortage members.

   • Embed this notice
     Luca Sironi (luca@sironi.tk)'s status on Saturday, 09-Nov-2024 18:40:19 JST Luca Sironi

     in reply to

     • Nemo_bis 🌈
     • bryan newbold
     @nemobis @bnewbold
     
     very interesting, but if we bring decentralization to the extreme of almost personal server, up to say 100 users, which is a thing activitypub allow to do with bare minimum hw/network...
     
     there is no need to have a remote storage for caching media.
     
     A server with reasonably few users, download just a fraction of media from the fediverse, and don't have to keep them forever.
     
     We are chatting, not mirroring the whole internet.
     If you see something on internet, a beautiful picture, a long article, and you like it, you save it, you don't expect it to find it online next year
   • Embed this notice
     Nemo_bis 🌈 (nemobis@mamot.fr)'s status on Saturday, 09-Nov-2024 18:47:07 JST Nemo_bis 🌈

     in reply to

     • Luca Sironi

     @luca We're saying the same thing but I'm not sure about the details. Perhaps the optimal size for storage is closer to 1000 users rather than 100. I'm not sure I'd call 2 TiB of storage (the average in the Jortage pool) "minimal". It's still much better than 100 (what they'd need if each member of the pool mirrored everything).

     Recently #mamotfr started purging old media and posts with the extremely broken https://github.com/mastodon/mastodon/discussions/19260 . It's not clear why, perhaps limits in PostgreSQL scaling.

   • Embed this notice
     Nemo_bis 🌈 (nemobis@mamot.fr)'s status on Saturday, 09-Nov-2024 19:00:57 JST Nemo_bis 🌈

     in reply to

     • Luca Sironi

     @luca A single-user Pleroma instance is indeed a very different matter because Pleroma relies on hotlinking from remote instances. Only other Pleroma users or people who visit your instance directly will see your own media served by your server. I see it through Mamot's cache.

   • Embed this notice
     Luca Sironi (luca@sironi.tk)'s status on Saturday, 09-Nov-2024 19:00:59 JST Luca Sironi

     in reply to

     • Nemo_bis 🌈
     @nemobis
     
     It's all very clever, i really appreciate all attempts of optimizing storage, that's the only way big and medium instances can survive.
     
     My point is, my single user instance has the highest level of storage waste but it's already there, paid.
     How many people will see a picture because it's cached on my pleroma for a month ?
     
     Of course this works because one user is one, i receive daily just the amount of media i'm able to consume
     
     What if all users were standalone ?
     The most wasted internet resource is residential landline bandwidth
   • Embed this notice
     Nemo_bis 🌈 (nemobis@mamot.fr)'s status on Sunday, 10-Nov-2024 05:00:51 JST Nemo_bis 🌈

     in reply to

     • bryan newbold

     @bnewbold Interesting! IIRC one of the early Twitter papers had great findings on how they handled sorting. Broadcasting is easier.

     I can easily believe a single centralised relay is more efficient (as long as it fits the biggest machine you can get). A few dozens relays sounds reasonable; 10k (the number of Mastodon servers) less so. Relays would need to be pooled resources. A managed service à la masto.host would offer just one, shared by hundreds/thousands of customers.

   • Embed this notice
     bryan newbold (bnewbold@social.coop)'s status on Sunday, 10-Nov-2024 05:00:53 JST bryan newbold

     in reply to

     • Nemo_bis 🌈

     @nemobis at least for us, now, at current scale, for the bsky app, "reads" are more expensive (in kilowatts and silicon) than "writes". I don't think this would be particularly more efficient if network distributed the load more to many smaller nodes (vs having big API servers).

     it is possible to "shard" parts of network for things like search queries. https://yacy.net/ might be relevant. I don't think that helps with efficiency though?

   • Embed this notice
     bryan newbold (bnewbold@social.coop)'s status on Sunday, 10-Nov-2024 05:00:54 JST bryan newbold

     in reply to

     • Nemo_bis 🌈

     @nemobis atproto relays currently mirror only "records", not media blobs, so size isn't too crazy.

     we think a degree of duplication and mirroring is good/healthy for the network. similar to having multiple copies of git repo checkouts. but a few dozen full network copies is probably plenty?

     resource/climate-wise, what we see in our infra is that "reverse chron" timelines, and to some degree notification tracking, are expensive (much more than relay). ironically "algo feeds" are cheaper?

   • Embed this notice
     Nemo_bis 🌈 (nemobis@mamot.fr)'s status on Sunday, 10-Nov-2024 05:08:28 JST Nemo_bis 🌈

     in reply to

     • bryan newbold

     @bnewbold Sharding metadata across database is challenging. No idea how to solve that problem. It would be fascinating to compare the resource efficiency of the DB servers at a large Mastodon instance like mastodon.social vs. hundreds of small ones across 20 DB servers like on masto.host. Any DB researchers in the room? 😇