Conversation

Notices

Embed this notice
feld (feld@friedcheese.us)'s status on Saturday, 26-Oct-2024 12:03:43 JST feld
- pistolero
@p I spent so many hours fighting with IPFS I think it's a dead end. You can even find a stale MR where I upgraded the storage backend from Badger to Badger2 (because the default filesystem store sucks) and it didn't help much. Nobody working on Kubo has any interest in fixing the storage scalability problems it seems. To use it for fedi we'd really want the garbage collection to work and not spin 100% CPU and IO for hours but that's what it does when you get enough data in there 🤬

In conversation about 12 days ago from friedcheese.us permalink
- Embed this notice
  feld (feld@friedcheese.us)'s status on Saturday, 26-Oct-2024 12:40:11 JST feld
  in reply to
  - pistolero
  @p lol they did what? Crazy, especially because it supports like 20 different hash formats. Which is annoying when running a proxy because you have to script generation of a blacklist to stop every possible hash for that same file.
  
  In conversation about 12 days ago permalink
- Embed this notice
  pistolero (p@fsebugoutzone.org)'s status on Saturday, 26-Oct-2024 12:40:12 JST pistolero
  in reply to
  
  @feld
  
  > I spent so many hours fighting with IPFS I think it's a dead end.
  
  Yeah, that's my conclusion, too. It should be great but is not even good. FSE had the start of a patch but I didn't get too far actually integrating it. (
  https://git.freespeechextremist.com/gitweb/?p=fse;a=commit;h=2dec0a138612eebe79f757b5bc04810b81d65951 )
  > Nobody working on Kubo has any interest in fixing the storage scalability problems it seems.
  
  I think it's a little past them, if I'm being honest. They could probably get away with just integrating tkrzw.
  
  > To use it for fedi we'd really want the garbage collection to work and not spin 100% CPU and IO for hours but that's what it does when you get enough data in there 🤬
  
  Yeah, you wanna see a real horror, look at the directory where it stores the data. I don't know if it's still doing that but it was storing different permutations of the same hash as different files last I checked. So the Qm$whatever or the bafy$whatever, those were just stored as separate files instead of normalizing the hash.
  
  8MB is too large for you to get fortuitous collisions, it's not a very good chunk size for transfer, just the entire thing is like a parody.
  In conversation about 12 days ago permalink
  Attachments
  1. Untitled attachment
- Embed this notice
  feld (feld@friedcheese.us)'s status on Saturday, 26-Oct-2024 12:50:42 JST feld
  in reply to
  - pistolero
  @p Unfamiliar with trkzw; I was thinking they should build on LMDB.
  
  In conversation about 12 days ago permalink
- Embed this notice
  silverpill (silverpill@mitra.social)'s status on Saturday, 26-Oct-2024 23:40:57 JST silverpill
  in reply to
  @frogzone @jeffcliff @p @feld Performance issues with IPFS were there from the beginning. Many hoped that Protocol Labs will fix them but that didn't happen, perhaps due to the inherent limitations of the protocol, or due to incompetence.
  In any case, this is not a "controlled demolition". There was a shift to NFTs and then Filecoin around 2020, but I don't think that affected IPFS in a negative way
  
  In conversation about 12 days ago permalink
- Embed this notice
  frogzone@wizard.casa's status on Saturday, 26-Oct-2024 23:40:58 JST frogzone
  in reply to
  @feld @p @silverpill @jeffcliff ok, very interesting to read these thoughts on the degraded state of IPFS and how unfit it currently is for use with fedi (in browser as an add-on and as a server).
  it fits with my thesis that foss on m$github is getting slowly compromised, by the shell accounts that the corporate state have on that weaponized platform.... a controlled platform, at best, and a platform that deanonymizes devs, which is a way to control them at the very least.
  Id consider going back to 2021 and looking at their codebase then, and comparing it to today, there were some very prominent independent media folk talking about IPFS back then and so i suspect that THAT is when the controlled demolition may have started. My understanding is once the corporates have their sights on a codebase they force the programmers to accept bad tradeoffs. Trade-offs exist everywhere, and permutations of tradeoffs can be chosen ad-infinitum. my learned experience is that foss on mshithub tends to not improve in ways that would allow it to compete with corposhit, especially in the #ux department.
  to accelerate :butterfedyC: fediverse centric adaptations, just over a week ago i wrote about having #ipfs on another repo and i now, having read both your comments i think it may need to indeed be a fork. a week ago i suggested the acronym DCN (decentralised content network) a tongue-in-cheek reference/reversal of CDN....
  so yeah if anyone wants to run with that idea, u have my blessing
  #dcn #msgithub #getoffgithub
  In conversation about 12 days ago permalink
  Attachments
  1. Untitled attachment
  2. Untitled attachment
- Embed this notice
  feld (feld@friedcheese.us)'s status on Sunday, 27-Oct-2024 00:15:59 JST feld
  in reply to
  @silverpill @frogzone @jeffcliff @p Filecoin was supposed to have a new and improved storage backend but I haven't seen any details about it
  
  In conversation about 12 days ago permalink
- Embed this notice
  pistolero (p@fsebugoutzone.org)'s status on Sunday, 27-Oct-2024 01:59:16 JST pistolero
  in reply to
  
  @feld trkzw is by the same dude that did TokyoCabinet/KyotoCabinet. Very good shit (TokyoTyrant spoke HTTP and memcache's protocol over the same port) but he got fascinated with C++ so I haven't used the later ones. trkzw has Go bindings (that is, not a Go implementation, and CGO is useless if you are on Plan 9, but IPFS relies on a bunch of fluoridated Linux stuff anyway so CGO is necessary for them).
  
  In conversation about 12 days ago permalink
- Embed this notice
  feld (feld@friedcheese.us)'s status on Sunday, 27-Oct-2024 01:59:16 JST feld
  in reply to
  - pistolero
  @p Ahh, ok. Last I checked LMDB blew away TokyoCabinet in benchmarks but I don't think I've ever seen it compared to trkzw
  
  In conversation about 12 days ago permalink
- Embed this notice
  سورق (syzygy@clubcyberia.co)'s status on Sunday, 27-Oct-2024 02:01:45 JST سورق
  in reply to
  - pistolero
  @p @feld
  trzkw sounds like a twitter handle of some jap drawing MS Paint scribbles of anime girls that would then get reposted by 300 lbs balding trannies crying how sad their life is on /r9k/.
  
  In conversation about 12 days ago permalink
  
  likes this.
- Embed this notice
  silverpill (silverpill@mitra.social)'s status on Sunday, 27-Oct-2024 06:05:46 JST silverpill
  in reply to
  @p @frogzone @jeffcliff @feld What is your current stance on FEP-ef61? Does it conflict with what you're doing in Revolver, or complement that?
  https://codeberg.org/fediverse/fep/src/branch/main/fep/ef61/fep-ef61.md
  Significant progress has been made since we last discussed it. Today there two implementations, Mitra and Streams, and they are partially interoperating.
  
  In conversation about 12 days ago permalink
- Embed this notice
  pistolero (p@fsebugoutzone.org)'s status on Sunday, 27-Oct-2024 06:05:47 JST pistolero
  in reply to
  @frogzone @silverpill @jeffcliff @feld
  
  > sounds like i may need to look closer :p
  
  I explained the block splitting to nyanide's final boss a few days ago: https://fsebugoutzone.org/notice/AnGc6MOGfUsuVOsblQ . Really, the better thing to read, though, would be the Kademlia paper (attached) and the venti paper, which is at http://doc.cat-v.org/plan_9/4th_edition/papers/venti/ .
  
  > the reason i thought we could do it with ipfs was that it already was being used in i2p, but its only being used as a proof of concept at this stage.
  
  I hesitate to say too much about numbers but I will say that I think it would be hard to make something as slow as IPFS. You'd have to try really hard.
  kpos.pdf
  In conversation about 12 days ago permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: doc.cat-v.org
    
    Venti: a new approach to archival storage
  2. kpos.pdf
  3. No result found on File_thumbnail lookup.
    
    https://fsebugoutzone.org/notice/AnGc6MOGfUsuVOsblQ
- Embed this notice
  frogzone@wizard.casa's status on Sunday, 27-Oct-2024 06:05:48 JST frogzone
  in reply to
  @p @silverpill @jeffcliff @feld >re rust and (A)GPL to BSD
  I'll need to keep an eye on that
  >ocelot/revolver
  sounds like i may need to look closer :p
  the reason i thought we could do it with ipfs was that it already was being used in i2p, but its only being used as a proof of concept at this stage.
  
  In conversation about 12 days ago permalink
- Embed this notice
  pistolero (p@fsebugoutzone.org)'s status on Sunday, 27-Oct-2024 06:05:49 JST pistolero
  in reply to
  @frogzone @silverpill @jeffcliff @feld
  
  > foss on mshithub tends to not improve in ways that would allow it to compete with corposhit,
  
  The mass "Rewrite it in Rust" bit tends to have the side effect of converting (A)GPL software to MIT/BSD.
  
  > so yeah if anyone wants to run with that idea,
  
  :ocelot: I may be a bit ahead of you on this. :revolvertan:
  
  In conversation about 12 days ago permalink
- Embed this notice
  pistolero (p@fsebugoutzone.org)'s status on Monday, 28-Oct-2024 01:44:03 JST pistolero
  in reply to
  @silverpill @frogzone @jeffcliff @feld I haven't kept up with it. I have some notes but you can feel free to ignore them; I don't want to get in the way of some people doing a thing. Wherever it lands, I'll try to make sure that we can interoperate without breaking the FEP semantics if possible, but some places, it's not going to be possible. So, here are the notes (and quick read, so I may have missed things or I may have the wrong idea, because the stupid DDoS yesterday cost me a lot of time and I'm trying to catch up today):
  
  > ap://did:example:123456/path/to/object?name=value#key-id
  
  If it's a new protocol, I don't know why we're still doing a fragment for the key ID. I don't love JSON (anti-web) but we've got to live with it; on the other hand, I don't want to abuse JSON or URLs.
  
  > Ed25519
  
  I'm using RSA keys so that it's easy to port myself from Pleroma. It is entirely possible that there is something I am missing and that we shouldn't be using RSA keys; maybe key algo is overspecifying.
  
  > /.well-known/apgateway
  
  I like this, but I wonder if there's a way around it without hard-coding another URL.
  
  > The client MUST specify an Accept header with the application/ld+json; profile="https://www.w3.org/ns/activitystreams" media type.
  
  This is actually objectively wrong. I don't know why this should be done rather than using the HTTP standard interpretation of "Accept:" headers. If the client doesn't include "application/ld+json" or "*/*", then you just return a 406. The way Pleroma/Masto do these things kind of shits on RFC 2616 and I really love RFC 2616. (This is why I'm not doing the terrible sniffing algorithm to differentiate between a browser and another server on the /objects/$id URLs.) If there is a good reason, maybe, but if we're tossing out the well-defined HTTP semantics that HTTP clients rely on, we can't call it HTTP.
  
  > Portable actors
  
  This looks roughly compatible with what I am doing; there's an ID for an actor, the network protocol is used to find the actor, etc., and then to use an AP gateway, you sign a request to be canonically located on one, the owner of that domain approves or ignores, and that's the PoT for your account as far as any other AP server knows. So your /inbox and /outbox, the server with the responsibility for delivering your posts when it sees new ones, etc.
  
  So this section looks like it's roughly compatible with that. Instead of ordered gateways, Revolver can just send one gateway.
  
  > Portable objects
  
  This part, I don't know how I'm going to manage this. The "attributedTo" being tied to the gateway URLs, and then those potentially changing, that's going to be a mess with content-addressed storage. If you create a new activity by signing the head of a tree that contains pointers to objects in a stream, then those objects changing (e.g., because a gateway changed) is going to make hopping computationally infeasible once you have a long enough history (which is going to be almost all of the initial users, since they will mostly be people on FSE/bae.st that follow the move to Revolver).
  In conversation about 11 days ago permalink
  Attachments
  1. Untitled attachment
- Embed this notice
  silverpill (silverpill@mitra.social)'s status on Monday, 28-Oct-2024 01:44:03 JST silverpill
  in reply to
  @p
  >I'll try to make sure that we can interoperate without breaking the FEP semantics if possible, but some places, it's not going to be possible.
  If you want to see how it all works in practice, there is @nomad
  It is a Mastodon-compatible FEP-ef61 actor that is managed from a client.
  >If it's a new protocol, I don't know why we're still doing a fragment for the key ID.
  'ap' URLs resolve to ActivityStreams documents, and fragments are useful for pointing to specific sections within those documents. The URL you're citing is just an example, and implementers are not required to use fragments for key IDs (although that might be a good idea).
  I shall change fragment ID in this example to something neutral.
  >>Ed25519
  >I'm using RSA keys so that it's easy to port myself from Pleroma. It is entirely possible that there is something I am missing and that we shouldn't be using RSA keys; maybe key algo is overspecifying.
  Nowadays security specialists advise against using RSA (1, 2), and some modern standards do not support it (for example, there is no RSA cryptosuite for Data Integity, which we use for signing objects).
  Fediverse needs to migrate to something else and EdDSA seems to be the best option (according to the rough consensus among fedi devs). Also, Ed25519 key size is very small, that's pretty cool.
  >>/.well-known/apgateway
  >I like this, but I wonder if there's a way around it without hard-coding another URL.
  Yes, follow-your-nose approach is mentioned in "Discussion - Discovering locations". Existing implementations rely on well-known locations, but I think we can support follow-your-nose if needed.
  >> The client MUST specify an Accept header with the application/ld+json; profile="https://www.w3.org/ns/activitystreams" media type.
  >This is actually objectively wrong. I don't know why this should be done rather than using the HTTP standard interpretation of "Accept:" headers. If the client doesn't include "application/ld+json" or "*/*", then you just return a 406. The way Pleroma/Masto do these things kind of shits on RFC 2616 and I really love RFC 2616.
  But the FEP should say something about Accept header and media type, I don't see any way around that. The exact same requirement exists in ActivityPub spec.
  How do you prefer it to be written?
  >> Portable objects
  >This part, I don't know how I'm going to manage this. The "attributedTo" being tied to the gateway URLs, and then those potentially changing
  IDs are not supposed to change. You include this special query parameter ("location hint") when you create an object, and then it stays there even if you start using a different gateway.
  Maybe there's a better way to provide location hints, I don't know. Existing implementations use compatible IDs (regular 'http' URLs with 'ap' URL appended), so not much thought has been put into pure 'ap' IDs.
  >that's going to be a mess with content-addressed storage
  How do you process Update activities then?
  @frogzone @jeffcliff @feld
  In conversation about 11 days ago permalink
  Attachments
  1. Untitled attachment
- Embed this notice
  feld (feld@friedcheese.us)'s status on Monday, 28-Oct-2024 03:37:42 JST feld
  in reply to
  @p @silverpill @frogzone @jeffcliff We'll do ed25519 if it's gonna actually stick. I don't think it's been revisited since the original proposal was seen and I didn't even know Honk was supporting it.
  
  Anyone else? Mastodon?
  
  In conversation about 11 days ago permalink
- Embed this notice
  pistolero (p@fsebugoutzone.org)'s status on Monday, 28-Oct-2024 03:37:44 JST pistolero
  in reply to
  @silverpill @frogzone @jeffcliff @feld
  
  > Fediverse needs to migrate to something else and EdDSA seems to be the best option (according to the rough consensus among fedi devs). Also, Ed25519 key size is very small, that's pretty cool.
  
  Ah, okay. Reasonable. I'll put it on the list; I had seen support for it in Honk but because Pleroma didn't have it, I just put a note in a comment and ignored it.
  
  > But the FEP should say something about Accept header and media type, I don't see any way around that.
  
  The Accept header mechanics don't need redefining; you just say it's served with this MIME-type, right, and then servers that handle Accept headers correctly will handle it correctly. If you're worried an informal reading will make sloppy implementations create a de facto bad standard, then you could put a note in that the client and server are expected to implement HTTP-compliant semantics.
  
  > The exact same requirement exists in ActivityPub spec.
  
  It's broken in the ActivityPub spec. It is not just broken in the ActivityPub spec, but it is irreparably brain-damaged. Feel free to skip these next two paragraphs; I explain why what the ActivityPub spec says can/should be ignored, but I also cannot type this without ranting.
  
  Why call it the "Accept" header if the server isn't going to treat it that way? If the ActivityPub spec says that a byte is nine bits, it's exceeded its authority. cwebber wants to treat it like a secret handshake because cwebber is a greasy illiterate dingus and this is network software: fuckups take years to resolve.
  
  "Accept:" is intended for content negotiation, not like a secret handshake like cwebber seems to think: the idea was initially that a server can represent content in several possible forms, clients say which forms they prefer and indicates preference, the server tries to give the client one of its preferred forms. Support for parameters *besides* preference was explicitly dropped, so the ActivityPub spec if not just broken, it fails to comply with an obsolete version. You sure as hell don't put some stupid fucking XML-style wank in there where you cram a *namespace* into the Accept header. Dear god, I haven't seen a worse idea than ` The client MUST specify an Accept header with the application/ld+json; profile="https://www.w3.org/ns/activitystreams" media type in order to retrieve the activity.` in forever.
  
  Ahem.
  
  > How do you prefer it to be written?
  
  If I were writing it, I'd leave it out: you take it as read that if they are doing web server software, they should be following HTTP's semantics. Maybe a "See also RFC 9110 sections 8.3, 12, and 15.5.7" or something. You don't specify that the client MUST send some value for the Accept header, you take a well-formed HTTP message from the client, the semantics are in the spec for what you've got to do when the message is well-formed but semantically wrong, and you've not broken anyone's clients as long as you stick to the spec. So you say what the server should do in case the message is well-formed but the semantics specify something the server can't do, and you refer to the other specs for semantics and behavior of lower layers instead of re-specifying it.
  
  Again, you know, you're the one putting in the work so take anything I say about as seriously as is warranted: here are my thoughts.
  
  > Maybe there's a better way to provide location hints,
  
  Well, I think, like, you embed the location hints in the user's profile but you expect that if someone has the object, then they can figure out how to get the user. You don't get an object out of context, it's part of someone's stream.
  
  I think maybe "how the URI is constructed and how it should be interpreted semantically" should probably be a different document.
  
  > How do you process Update activities then?
  
  Updates update the Actor, and those are trivial, because the actor's metadata is a slot, not a stream. Other stuff, longer answer, and food is ready.
  In conversation about 11 days ago permalink
  Attachments
  1. Untitled attachment
- Embed this notice
  silverpill (silverpill@mitra.social)'s status on Monday, 28-Oct-2024 07:10:28 JST silverpill
  in reply to
  @p @frogzone @jeffcliff @feld Thank you, this is a useful feedback. I'll adjust the FEP
  
  In conversation about 11 days ago permalink
- Embed this notice
  silverpill (silverpill@mitra.social)'s status on Monday, 28-Oct-2024 07:18:22 JST silverpill
  in reply to
  @feld @frogzone @jeffcliff @p Projects that implement FEP-8b32 are already using Ed25519 keys (most notable of them is probably Hubzilla).
  Some other developers are also considering Ed25519, either for embedded (FEP-8b32) or HTTP signatures (RFC 9421).
  >Mastodon
  Mastodon too: https://github.com/mastodon/fediverse_auxiliary_service_provider_specifications/pull/27/files#diff-2caefb0fa42bedec7feb2490de59da44151d6e5e98fbab4640495535229b079cR65
  In conversation about 11 days ago permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: avatars.githubusercontent.com
    
    Replace OAuth 2.0 with HTTP Message Signatures by oneiros · Pull Request #27 · mastodon/fediverse_auxiliary_service_provider_specifications
    
    Fixes #24 See that issue for a discussion. Please note that in this draft, I also chose to sign the HTTP method and the target URI. This would mean that the receiver can verifiy that the action is ...
- Embed this notice
  feld (feld@friedcheese.us)'s status on Monday, 28-Oct-2024 07:26:09 JST feld
  in reply to
  @silverpill @p @jeffcliff @frogzone looking at that mastodon pull request: they're really trying to overcomplicate everything aren't they
  
  In conversation about 11 days ago permalink
- Embed this notice
  silverpill (silverpill@mitra.social)'s status on Monday, 28-Oct-2024 07:34:58 JST silverpill
  in reply to
  @feld @frogzone @jeffcliff @p The way of Mastodon
  
  In conversation about 11 days ago permalink
- Embed this notice
  pistolero (p@fsebugoutzone.org)'s status on Tuesday, 29-Oct-2024 00:58:38 JST pistolero
  in reply to
  @silverpill @feld @frogzone @jeffcliff
  
  > It is a Mastodon-compatible FEP-ef61 actor that is managed from a client. [ https://codeberg.org/silverpill/fep-ae97-client ]
  
  That is awesome.
  
  > Other stuff, longer answer, and food is ready.
  
  I don't have an answer for Edits; I don't plan to support them initially, maybe ever. I don't think it's a good idea. But the way a Delete works internally versus how it tries to convey this to the network, like, your activity stream is a list, as much as is needed can be backfilled, but deletion is O(n) for the actor doing it and O(1) for everyone else. I'm not sure what's a good way to convey that back and forth to fedi, you know, I don't wanna get stuck processing some Delete-flood from someone with 400k Creates on some masto instance.
  
  (I think I had other stuff to say here but I don't remember. Weekend before Halloween, couch, bourbon and pizza, woman, Dawn of the Dead, Day of the Dead, really great movies (the remakes sucked). Anyway, fell asleep on the couch, put on I forget what, and cannot remember what I was supposed to put in this box. Rather than feeling refreshed after a day off, I feel sluggish and off my game, so I figure weekends and vacations are for suckers.)
  In conversation about 10 days ago permalink
  Attachments
  1. Domain not in remote thumbnail source whitelist: codeberg.org
    
    fep-ae97-client
    
    from silverpill
    
    fep-ae97-client
  ✙ dcc :pedomustdie: :phear_slackware: likes this.
- Embed this notice
  pistolero (p@fsebugoutzone.org)'s status on Tuesday, 29-Oct-2024 02:03:40 JST pistolero
  in reply to
  @feld @frogzone @jeffcliff @silverpill
  
  > We'll do ed25519 if it's gonna actually stick.
  
  Well, it's not too much trouble from my end to support verifying signatures using another key type.
  
  I do not have strong opinions on the keys in use (because it doesn't make a huge difference and will change every few years), just that the old ones stay supported indefinitely.
  
  In conversation about 10 days ago permalink
  
  feld likes this.

Public

Conversation

Notices

Feeds