Conversation

Notices

1. Embed this notice
   feld (feld@friedcheese.us)'s status on Sunday, 27-Apr-2025 09:00:29 JST feld
   Does anyone know of an existing script/tool that can successfully split mbox files and maybe output them one by one to stdout or something? It's not quite as easy as splitting on the cutom "From " marker that mbox uses (From+ space, not From: ) because that could be in the email itself so I guess some additional heuristics are required for accuracy anyway
   
   I'm sure I could whip up something with Perl but there's gotta be a tool out there already that can let me iterate over the messages in mbox format
   • Embed this notice
     feld (feld@friedcheese.us)'s status on Sunday, 27-Apr-2025 09:10:27 JST feld

     in reply to
     heh there's a mb2md script written in Perl that people have used for ages to convert from mbox to maildir which is not exactly what I want but it would suffice.
     
     But it doesn't work right on these mbox files; it thinks there's only one message inside.
     
     OOPS.
     
     There's also a python version but I'm not so hopeful about it
   • Embed this notice
     feld (feld@friedcheese.us)'s status on Sunday, 27-Apr-2025 09:15:12 JST feld

     in reply to
     Nope, doesn't work either. Amazing.
   • Embed this notice
     feld (feld@friedcheese.us)'s status on Sunday, 27-Apr-2025 09:19:38 JST feld

     in reply to

     • Hal Pomeranz
     @hal_pomeranz I'd like to believe that's true but that doesn't seem to be reality. I have a valid mbox file that mutt can read no problem but tools that should be able to convert it to maildir does not work. I also tried a quick hack with awk and perl and encountered problems. Very strange.
   • Embed this notice
     Hal Pomeranz (hal_pomeranz@infosec.exchange)'s status on Sunday, 27-Apr-2025 09:19:39 JST Hal Pomeranz

     in reply to

     @feld “\nFrom “ (newline, From, space) is the correct mbox message separator. The RFC states that this pattern appearing in the message MUST be rewritten as “\n>From “ to avoid confusion.

     feld likes this.
   • Embed this notice
     feld (feld@friedcheese.us)'s status on Sunday, 27-Apr-2025 09:43:05 JST feld

     in reply to
     @hal_pomeranz
     
     ./mgrep Subject /tmp/junk.mbox
     
     works as far as spitting out every message to stdout
     
     Anyway, I found a dirty hack on how to do this with mutt
     
     .muttrc:
     
     set folder = "/tmp/maildir"
     macro index <F3> "<tag-pattern>~A<enter><tag-prefix><copy-message>=out<enter>y<quit>y"
     folder-hook . push <F3>
     
     
     /usr/local/bin/mutt -F /root/.muttrc -n -m Maildir -R -f foo.mbox
     
     
     this outputs a correct maildir to /tmp/maildir
   • Embed this notice
     Hal Pomeranz (hal_pomeranz@infosec.exchange)'s status on Sunday, 27-Apr-2025 09:43:06 JST Hal Pomeranz

     in reply to

     @feld Here's a tool that I wrote to do pattern matching in mbox files and return matching messages. I know this tool works. Try it on your mbox, and if it fails then you don't have a typical mbox file.

     https://github.com/halpomeranz/dfis/blob/master/mgrep

   • Embed this notice
     feld (feld@friedcheese.us)'s status on Sunday, 27-Apr-2025 09:53:06 JST feld

     in reply to
     @hal_pomeranz here's a sample mbox file if you're curious though. So weird that mb2md which as existed for 20+ years now can't handle it????
     
     https://obj.feld.me/files/junk.mbox