Conversation

Notices

Embed this notice
rakoo (rakoo@blah.rako.space)'s status on Tuesday, 17-Jun-2025 01:48:29 JST rakoo
@ska

The whole point of databases is that there is no need for additional systems for indexing, because storing is useless without querying. I see the value in storing in a *single* system that provides features rather than being yet again elitist and telling people to do their own stuff on their own, resulting in no one doing it. Let's not dismiss changes just because you know how to use a system
@lispi314 @dalias @lanodan @ptrc @navi

In conversation about 7 days ago from blah.rako.space permalink
- Embed this notice
  Haelwenn /элвэн/ :triskell: (lanodan@queer.hacktivis.me)'s status on Tuesday, 17-Jun-2025 01:48:26 JST Haelwenn /элвэн/ :triskell:
  in reply to
  - Rich Felker
  - anna
  - LisPi
  @rakoo @dalias @ptrc @ska @navi @lispi314 Except emails are large immutable data where only few headers (some which added by your mail server) need indexing.
  And then good luck having a db-structure which accounts for MIME so you don't end up with your queries returning whole emails (potentially several megabytes) instead of just the main text (usually few kilobytes). Something which filesystems don't really have a problem with as the MUA can use the Content-Length header + read(2) whenever present.
  
  In conversation about 7 days ago permalink
- Embed this notice
  Haelwenn /элвэн/ :triskell: (lanodan@queer.hacktivis.me)'s status on Tuesday, 17-Jun-2025 01:51:14 JST Haelwenn /элвэн/ :triskell:
  in reply to
  @rakoo @dalias @lispi314 @navi @ptrc @ska Also to me the elitism claim is entirely offtopic. mbox vs. maildir vs. database is pretty much just an implementation detail, but one which has consequences.
  
  In conversation about 7 days ago permalink
- Embed this notice
  rakoo (rakoo@blah.rako.space)'s status on Tuesday, 17-Jun-2025 02:01:24 JST rakoo
  in reply to
  
  @lanodan
  
  No, indexing will include text as well because obviously I want to search in it. That's the whole point.
  
  To be clear I'm not talking about getting read of MUAs but just rethinking the interface between it and content. So the mua *can* know the mime structure and will use the *same* tricks when querying. That's the whole point of a database
  @dalias @ptrc @ska @navi @lispi314
  
  In conversation about 7 days ago permalink
- Embed this notice
  Haelwenn /элвэн/ :triskell: (lanodan@queer.hacktivis.me)'s status on Tuesday, 17-Jun-2025 02:02:08 JST Haelwenn /элвэн/ :triskell:
  in reply to
  - Rich Felker
  - anna
  - LisPi
  @rakoo @dalias @ptrc @ska @navi @lispi314 So full text search over emails?
  
  Sure could be useful, but I hope you have a *lot* of RAM (roughly 10% of your textual data), like the reason full text search doesn't works on the fediverse is almost always the lack of RAM (few other times it's PostgreSQL/SQLite footguns).
  
  In conversation about 7 days ago permalink
- Embed this notice
  Haelwenn /элвэн/ :triskell: (lanodan@queer.hacktivis.me)'s status on Tuesday, 17-Jun-2025 02:04:13 JST Haelwenn /элвэн/ :triskell:
  in reply to
  - Rich Felker
  - anna
  - LisPi
  @rakoo @dalias @ptrc @ska @navi @lispi314 There's several maildir indexation systems out there, mutt includes one for example, there's notmuch, and I'd doubt courrier-imapd doesn't have an index.
  
  In conversation about 7 days ago permalink
- Embed this notice
  rakoo (rakoo@blah.rako.space)'s status on Tuesday, 17-Jun-2025 02:04:14 JST rakoo
  in reply to
  @lanodan
  
  It's the "You can always do [additional, undocumented process] to do the thing you wan to do" part that is elitist, not the choice of technical solution.
  @dalias @ptrc @ska @navi @lispi314
  
  In conversation about 7 days ago permalink
- Embed this notice
  rakoo (rakoo@blah.rako.space)'s status on Tuesday, 17-Jun-2025 02:21:37 JST rakoo
  in reply to
  @lanodan
  
  full text search doesn't need ram, it needs proper data structures. Proper data structures can be stored on disk and popped in and out of memory as needed, but don't have to
  @dalias @ptrc @ska @navi @lispi314
  
  In conversation about 7 days ago permalink
- Embed this notice
  Haelwenn /элвэн/ :triskell: (lanodan@queer.hacktivis.me)'s status on Tuesday, 17-Jun-2025 02:22:08 JST Haelwenn /элвэн/ :triskell:
  in reply to
  - Rich Felker
  - anna
  - LisPi
  @rakoo @dalias @ptrc @ska @navi @lispi314 Do you have an example of that in production?
  
  Because to me it reads like something where you'd need to frequently re-order the data on disk so they're nicely packed, and so trashes the I/Os it would itself need to fetch the indexes as it needs them.
  
  In conversation about 7 days ago permalink
- Embed this notice
  Haelwenn /элвэн/ :triskell: (lanodan@queer.hacktivis.me)'s status on Tuesday, 17-Jun-2025 02:25:30 JST Haelwenn /элвэн/ :triskell:
  in reply to
  - Rich Felker
  - anna
  - LisPi
  @rakoo @dalias @ptrc @navi @lispi314 Uuh… notmuch is a component for MUAs and courrier-imapd is as the name implies an IMAP daemon.
  
  In conversation about 7 days ago permalink
- Embed this notice
  rakoo (rakoo@blah.rako.space)'s status on Tuesday, 17-Jun-2025 02:25:31 JST rakoo
  in reply to
  @lanodan
  
  yes, each application has its own index. The point is that if they all need the same feature anyway, why not mutualize it ?
  
  Silly comparison: we could say that everything should be stored as blocks laid out randomly on a disk, and each app defines an index from an identifier to a list of blocks. If everyone does it, might as well do it at the layer below
  @dalias @ptrc @ska @navi @lispi314
  
  In conversation about 7 days ago permalink
- Embed this notice
  Haelwenn /элвэн/ :triskell: (lanodan@queer.hacktivis.me)'s status on Tuesday, 17-Jun-2025 02:32:28 JST Haelwenn /элвэн/ :triskell:
  in reply to
  @rakoo @dalias @lispi314 @navi @ptrc Also it's kind of funny, I've pretty much never seen software use a database format that's standard enough it can be used by other applications without something like a migration, it otherwise always ends up with a backend in front of the database.
  While Maildir was designed to be concurrently used by many different software.
  
  In conversation about 7 days ago permalink

Public

Conversation

Notices

Feeds