@feld<\/a> Here's a tool that I wrote to do pattern matching in mbox files and return matching messages. I know this tool works. Try it on your mbox, and if it fails then you don't have a typical mbox file.<\/p> https:\/\/github.com\/halpomeranz\/dfis\/blob\/master\/mgrep<\/a><\/p>","generator":{"id":"tag:gnusocial.jp,2026-08-31:notice-source:ActivityPub","objectType":"application","status_net":{"source_code":"ActivityPub"}},"id":"https:\/\/infosec.exchange\/users\/hal_pomeranz\/statuses\/114407086919902007","object":{"id":"https:\/\/infosec.exchange\/users\/hal_pomeranz\/statuses\/114407086919902007","objectType":"note","content":" @feld<\/a> Here's a tool that I wrote to do pattern matching in mbox files and return matching messages. I know this tool works. Try it on your mbox, and if it fails then you don't have a typical mbox file.<\/p> https:\/\/github.com\/halpomeranz\/dfis\/blob\/master\/mgrep<\/a><\/p>","url":"https:\/\/infosec.exchange\/@hal_pomeranz\/114407086919902007","status_net":{"notice_id":null},"inReplyTo":{"objectType":"note","id":"https:\/\/friedcheese.us\/objects\/7deb4425-0097-4e21-b78b-f00c0e0fc87f","url":"https:\/\/friedcheese.us\/objects\/7deb4425-0097-4e21-b78b-f00c0e0fc87f"}},"to":[{"objectType":"http:\/\/activitystrea.ms\/schema\/1.0\/person","id":"https:\/\/friedcheese.us\/users\/feld"},{"objectType":"http:\/\/activitystrea.ms\/schema\/1.0\/collection","id":"http:\/\/activityschema.org\/collection\/public"}],"status_net":{"conversation":"tag:gnusocial.jp,2025-04-27:objectType=thread:nonce=dd1265e94d74ca87","notice_info":{"local_id":"9720331","source":"ActivityPub"}},"published":"2025-04-27T00:43:06+00:00","provider":{"objectType":"service","displayName":"GNU social JP","url":"https:\/\/gnusocial.jp\/"},"verb":"post","url":"https:\/\/infosec.exchange\/@hal_pomeranz\/114407086919902007"},{"actor":{"id":"https:\/\/friedcheese.us\/users\/feld","displayName":"feld","status_net":{"avatarLinks":[{"url":"https:\/\/gnusocial.jp\/avatar\/274913-original-tmp20241012191747.webp","rel":"avatar","type":"image\/webp","width":214,"height":214},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-96-20241012191855.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-48-20241012191855.webp","rel":"avatar","type":"image\/webp","width":48,"height":48},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-24-20241012191855.webp","rel":"avatar","type":"image\/webp","width":24,"height":24}],"profile_info":{"local_id":"274913"}},"image":{"url":"https:\/\/gnusocial.jp\/avatar\/274913-96-20241012191855.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},"objectType":"person","summary":"Up and not crying","url":"https:\/\/friedcheese.us\/users\/feld","portablecontacts_net":{"preferredUsername":"feld","displayName":"feld","note":"Up and not crying"}},"content":"@hal_pomeranz<\/a> @feld<\/a> \u201c\\nFrom \u201c (newline, From, space) is the correct mbox message separator. The RFC states that this pattern appearing in the message MUST be rewritten as \u201c\\n>From \u201c to avoid confusion.<\/p>","generator":{"id":"tag:gnusocial.jp,2026-08-31:notice-source:ActivityPub","objectType":"application","status_net":{"source_code":"ActivityPub"}},"id":"https:\/\/infosec.exchange\/users\/hal_pomeranz\/statuses\/114407041752040186","object":{"id":"https:\/\/infosec.exchange\/users\/hal_pomeranz\/statuses\/114407041752040186","objectType":"note","content":" @feld<\/a> \u201c\\nFrom \u201c (newline, From, space) is the correct mbox message separator. The RFC states that this pattern appearing in the message MUST be rewritten as \u201c\\n>From \u201c to avoid confusion.<\/p>","url":"https:\/\/infosec.exchange\/@hal_pomeranz\/114407041752040186","status_net":{"notice_id":null},"inReplyTo":{"objectType":"note","id":"https:\/\/friedcheese.us\/objects\/86d2c6b6-5056-4ddc-8271-585d770c9695","url":"https:\/\/friedcheese.us\/objects\/86d2c6b6-5056-4ddc-8271-585d770c9695"}},"to":[{"objectType":"http:\/\/activitystrea.ms\/schema\/1.0\/person","id":"https:\/\/friedcheese.us\/users\/feld"},{"objectType":"http:\/\/activitystrea.ms\/schema\/1.0\/collection","id":"http:\/\/activityschema.org\/collection\/public"}],"status_net":{"conversation":"tag:gnusocial.jp,2025-04-27:objectType=thread:nonce=dd1265e94d74ca87","notice_info":{"local_id":"9720150","source":"ActivityPub"}},"published":"2025-04-27T00:19:39+00:00","provider":{"objectType":"service","displayName":"GNU social JP","url":"https:\/\/gnusocial.jp\/"},"verb":"post","url":"https:\/\/infosec.exchange\/@hal_pomeranz\/114407041752040186"},{"actor":{"id":"https:\/\/friedcheese.us\/users\/feld","displayName":"feld","status_net":{"avatarLinks":[{"url":"https:\/\/gnusocial.jp\/avatar\/274913-original-tmp20241012191747.webp","rel":"avatar","type":"image\/webp","width":214,"height":214},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-96-20241012191855.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-48-20241012191855.webp","rel":"avatar","type":"image\/webp","width":48,"height":48},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-24-20241012191855.webp","rel":"avatar","type":"image\/webp","width":24,"height":24}],"profile_info":{"local_id":"274913"}},"image":{"url":"https:\/\/gnusocial.jp\/avatar\/274913-96-20241012191855.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},"objectType":"person","summary":"Up and not crying","url":"https:\/\/friedcheese.us\/users\/feld","portablecontacts_net":{"preferredUsername":"feld","displayName":"feld","note":"Up and not crying"}},"content":"@hal_pomeranz<\/a> I'd like to believe that's true but that doesn't seem to be reality. I have a valid mbox file that mutt can read no problem but tools that should be able to convert it to maildir does not work. I also tried a quick hack with awk and perl and encountered problems. Very strange.","generator":{"id":"tag:gnusocial.jp,2026-08-31:notice-source:ActivityPub","objectType":"application","status_net":{"source_code":"ActivityPub"}},"id":"https:\/\/friedcheese.us\/objects\/7deb4425-0097-4e21-b78b-f00c0e0fc87f","object":{"id":"https:\/\/friedcheese.us\/objects\/7deb4425-0097-4e21-b78b-f00c0e0fc87f","objectType":"note","content":"@hal_pomeranz<\/a> I'd like to believe that's true but that doesn't seem to be reality. I have a valid mbox file that mutt can read no problem but tools that should be able to convert it to maildir does not work. I also tried a quick hack with awk and perl and encountered problems. Very strange.","url":"https:\/\/friedcheese.us\/objects\/7deb4425-0097-4e21-b78b-f00c0e0fc87f","status_net":{"notice_id":null},"inReplyTo":{"objectType":"note","id":"https:\/\/infosec.exchange\/users\/hal_pomeranz\/statuses\/114407041752040186","url":"https:\/\/infosec.exchange\/@hal_pomeranz\/114407041752040186"}},"to":[{"objectType":"http:\/\/activitystrea.ms\/schema\/1.0\/person","id":"https:\/\/infosec.exchange\/users\/hal_pomeranz"},{"objectType":"http:\/\/activitystrea.ms\/schema\/1.0\/collection","id":"http:\/\/activityschema.org\/collection\/public"}],"status_net":{"conversation":"tag:gnusocial.jp,2025-04-27:objectType=thread:nonce=dd1265e94d74ca87","notice_info":{"local_id":"9720151","source":"ActivityPub"}},"published":"2025-04-27T00:19:38+00:00","provider":{"objectType":"service","displayName":"GNU social JP","url":"https:\/\/gnusocial.jp\/"},"verb":"post","url":"https:\/\/friedcheese.us\/objects\/7deb4425-0097-4e21-b78b-f00c0e0fc87f"},{"actor":{"id":"https:\/\/friedcheese.us\/users\/feld","displayName":"feld","status_net":{"avatarLinks":[{"url":"https:\/\/gnusocial.jp\/avatar\/274913-original-tmp20241012191747.webp","rel":"avatar","type":"image\/webp","width":214,"height":214},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-96-20241012191855.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-48-20241012191855.webp","rel":"avatar","type":"image\/webp","width":48,"height":48},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-24-20241012191855.webp","rel":"avatar","type":"image\/webp","width":24,"height":24}],"profile_info":{"local_id":"274913"}},"image":{"url":"https:\/\/gnusocial.jp\/avatar\/274913-96-20241012191855.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},"objectType":"person","summary":"Up and not crying","url":"https:\/\/friedcheese.us\/users\/feld","portablecontacts_net":{"preferredUsername":"feld","displayName":"feld","note":"Up and not crying"}},"content":"Nope, doesn't work either. Amazing.","generator":{"id":"tag:gnusocial.jp,2026-08-31:notice-source:ActivityPub","objectType":"application","status_net":{"source_code":"ActivityPub"}},"id":"https:\/\/friedcheese.us\/objects\/533b1e97-9b6f-44bf-8c6c-35414495c124","object":{"id":"https:\/\/friedcheese.us\/objects\/533b1e97-9b6f-44bf-8c6c-35414495c124","objectType":"note","content":"Nope, doesn't work either. Amazing.","url":"https:\/\/friedcheese.us\/objects\/533b1e97-9b6f-44bf-8c6c-35414495c124","status_net":{"notice_id":null},"inReplyTo":{"objectType":"note","id":"https:\/\/friedcheese.us\/objects\/be92964e-a073-4fd2-87dc-2b2b48d951a8","url":"https:\/\/friedcheese.us\/objects\/be92964e-a073-4fd2-87dc-2b2b48d951a8"}},"to":[{"objectType":"http:\/\/activitystrea.ms\/schema\/1.0\/collection","id":"http:\/\/activityschema.org\/collection\/public"}],"status_net":{"conversation":"tag:gnusocial.jp,2025-04-27:objectType=thread:nonce=dd1265e94d74ca87","notice_info":{"local_id":"9720127","source":"ActivityPub"}},"published":"2025-04-27T00:15:12+00:00","provider":{"objectType":"service","displayName":"GNU social JP","url":"https:\/\/gnusocial.jp\/"},"verb":"post","url":"https:\/\/friedcheese.us\/objects\/533b1e97-9b6f-44bf-8c6c-35414495c124"},{"actor":{"id":"https:\/\/friedcheese.us\/users\/feld","displayName":"feld","status_net":{"avatarLinks":[{"url":"https:\/\/gnusocial.jp\/avatar\/274913-original-tmp20241012191747.webp","rel":"avatar","type":"image\/webp","width":214,"height":214},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-96-20241012191855.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-48-20241012191855.webp","rel":"avatar","type":"image\/webp","width":48,"height":48},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-24-20241012191855.webp","rel":"avatar","type":"image\/webp","width":24,"height":24}],"profile_info":{"local_id":"274913"}},"image":{"url":"https:\/\/gnusocial.jp\/avatar\/274913-96-20241012191855.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},"objectType":"person","summary":"Up and not crying","url":"https:\/\/friedcheese.us\/users\/feld","portablecontacts_net":{"preferredUsername":"feld","displayName":"feld","note":"Up and not crying"}},"content":"heh there's a mb2md script written in Perl that people have used for ages to convert from mbox to maildir which is not exactly what I want but it would suffice.
.\/mgrep Subject \/tmp\/junk.mbox
works as far as spitting out every message to stdout
Anyway, I found a dirty hack on how to do this with mutt
.muttrc:
set folder = \"\/tmp\/maildir\"
macro index <F3> \"<tag-pattern>~A<enter><tag-prefix><copy-message>=out<enter>y<quit>y\"
folder-hook . push <F3>
\/usr\/local\/bin\/mutt -F \/root\/.muttrc -n -m Maildir -R -f foo.mbox
this outputs a correct maildir to \/tmp\/maildir","generator":{"id":"tag:gnusocial.jp,2026-08-31:notice-source:ActivityPub","objectType":"application","status_net":{"source_code":"ActivityPub"}},"id":"https:\/\/friedcheese.us\/objects\/2d5fc1d4-0689-489d-a142-f6daba72dbe5","object":{"id":"https:\/\/friedcheese.us\/objects\/2d5fc1d4-0689-489d-a142-f6daba72dbe5","objectType":"note","content":"@hal_pomeranz<\/a>
.\/mgrep Subject \/tmp\/junk.mbox
works as far as spitting out every message to stdout
Anyway, I found a dirty hack on how to do this with mutt
.muttrc:
set folder = \"\/tmp\/maildir\"
macro index <F3> \"<tag-pattern>~A<enter><tag-prefix><copy-message>=out<enter>y<quit>y\"
folder-hook . push <F3>
\/usr\/local\/bin\/mutt -F \/root\/.muttrc -n -m Maildir -R -f foo.mbox
this outputs a correct maildir to \/tmp\/maildir","url":"https:\/\/friedcheese.us\/objects\/2d5fc1d4-0689-489d-a142-f6daba72dbe5","status_net":{"notice_id":null},"inReplyTo":{"objectType":"note","id":"https:\/\/infosec.exchange\/users\/hal_pomeranz\/statuses\/114407086919902007","url":"https:\/\/infosec.exchange\/@hal_pomeranz\/114407086919902007"}},"to":[{"objectType":"http:\/\/activitystrea.ms\/schema\/1.0\/collection","id":"http:\/\/activityschema.org\/collection\/public"}],"status_net":{"conversation":"tag:gnusocial.jp,2025-04-27:objectType=thread:nonce=dd1265e94d74ca87","notice_info":{"local_id":"9720332","source":"ActivityPub"}},"published":"2025-04-27T00:43:05+00:00","provider":{"objectType":"service","displayName":"GNU social JP","url":"https:\/\/gnusocial.jp\/"},"verb":"post","url":"https:\/\/friedcheese.us\/objects\/2d5fc1d4-0689-489d-a142-f6daba72dbe5"},{"actor":{"id":"https:\/\/infosec.exchange\/users\/hal_pomeranz","displayName":"Hal Pomeranz","status_net":{"avatarLinks":[{"url":"https:\/\/gnusocial.jp\/avatar\/71425-original-tmp20240920010113.webp","rel":"avatar","type":"image\/webp","width":400,"height":400},{"url":"https:\/\/gnusocial.jp\/avatar\/71425-96-20240920194012.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},{"url":"https:\/\/gnusocial.jp\/avatar\/71425-48-20240920101916.webp","rel":"avatar","type":"image\/webp","width":48,"height":48},{"url":"https:\/\/gnusocial.jp\/avatar\/71425-24-20240920194013.webp","rel":"avatar","type":"image\/webp","width":24,"height":24}],"profile_info":{"local_id":"71425"}},"image":{"url":"https:\/\/gnusocial.jp\/avatar\/71425-96-20240920194012.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},"objectType":"person","summary":"Computer Forensics and Information Security, often with a spicy Linux flavor. Expert Witness. Trainer.","url":"https:\/\/infosec.exchange\/@hal_pomeranz","portablecontacts_net":{"preferredUsername":"hal_pomeranz","displayName":"Hal Pomeranz","note":"Computer Forensics and Information Security, often with a spicy Linux flavor. Expert Witness. Trainer."}},"content":"
But it doesn't work right on these mbox files; it thinks there's only one message inside.
OOPS.
There's also a python version but I'm not so hopeful about it","generator":{"id":"tag:gnusocial.jp,2026-08-31:notice-source:ActivityPub","objectType":"application","status_net":{"source_code":"ActivityPub"}},"id":"https:\/\/friedcheese.us\/objects\/be92964e-a073-4fd2-87dc-2b2b48d951a8","object":{"id":"https:\/\/friedcheese.us\/objects\/be92964e-a073-4fd2-87dc-2b2b48d951a8","objectType":"note","content":"heh there's a mb2md script written in Perl that people have used for ages to convert from mbox to maildir which is not exactly what I want but it would suffice.
But it doesn't work right on these mbox files; it thinks there's only one message inside.
OOPS.
There's also a python version but I'm not so hopeful about it","url":"https:\/\/friedcheese.us\/objects\/be92964e-a073-4fd2-87dc-2b2b48d951a8","status_net":{"notice_id":null},"inReplyTo":{"objectType":"note","id":"https:\/\/friedcheese.us\/objects\/86d2c6b6-5056-4ddc-8271-585d770c9695","url":"https:\/\/friedcheese.us\/objects\/86d2c6b6-5056-4ddc-8271-585d770c9695"}},"to":[{"objectType":"http:\/\/activitystrea.ms\/schema\/1.0\/person","id":"https:\/\/friedcheese.us\/users\/feld"},{"objectType":"http:\/\/activitystrea.ms\/schema\/1.0\/collection","id":"http:\/\/activityschema.org\/collection\/public"}],"status_net":{"conversation":"tag:gnusocial.jp,2025-04-27:objectType=thread:nonce=dd1265e94d74ca87","notice_info":{"local_id":"9720111","source":"ActivityPub"}},"published":"2025-04-27T00:10:27+00:00","provider":{"objectType":"service","displayName":"GNU social JP","url":"https:\/\/gnusocial.jp\/"},"verb":"post","url":"https:\/\/friedcheese.us\/objects\/be92964e-a073-4fd2-87dc-2b2b48d951a8"},{"actor":{"id":"https:\/\/friedcheese.us\/users\/feld","displayName":"feld","status_net":{"avatarLinks":[{"url":"https:\/\/gnusocial.jp\/avatar\/274913-original-tmp20241012191747.webp","rel":"avatar","type":"image\/webp","width":214,"height":214},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-96-20241012191855.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-48-20241012191855.webp","rel":"avatar","type":"image\/webp","width":48,"height":48},{"url":"https:\/\/gnusocial.jp\/avatar\/274913-24-20241012191855.webp","rel":"avatar","type":"image\/webp","width":24,"height":24}],"profile_info":{"local_id":"274913"}},"image":{"url":"https:\/\/gnusocial.jp\/avatar\/274913-96-20241012191855.webp","rel":"avatar","type":"image\/webp","width":96,"height":96},"objectType":"person","summary":"Up and not crying","url":"https:\/\/friedcheese.us\/users\/feld","portablecontacts_net":{"preferredUsername":"feld","displayName":"feld","note":"Up and not crying"}},"content":"Does anyone know of an existing script\/tool that can successfully split mbox files and maybe output them one by one to stdout or something? It's not quite as easy as splitting on the cutom \"From \" marker that mbox uses (From+ space, not From: ) because that could be in the email itself so I guess some additional heuristics are required for accuracy anyway
I'm sure I could whip up something with Perl but there's gotta be a tool out there already that can let me iterate over the messages in mbox format","generator":{"id":"tag:gnusocial.jp,2026-08-31:notice-source:ActivityPub","objectType":"application","status_net":{"source_code":"ActivityPub"}},"id":"https:\/\/friedcheese.us\/objects\/86d2c6b6-5056-4ddc-8271-585d770c9695","object":{"id":"https:\/\/friedcheese.us\/objects\/86d2c6b6-5056-4ddc-8271-585d770c9695","objectType":"note","content":"Does anyone know of an existing script\/tool that can successfully split mbox files and maybe output them one by one to stdout or something? It's not quite as easy as splitting on the cutom \"From \" marker that mbox uses (From+ space, not From: ) because that could be in the email itself so I guess some additional heuristics are required for accuracy anyway
I'm sure I could whip up something with Perl but there's gotta be a tool out there already that can let me iterate over the messages in mbox format","url":"https:\/\/friedcheese.us\/objects\/86d2c6b6-5056-4ddc-8271-585d770c9695","status_net":{"notice_id":null}},"to":[{"objectType":"http:\/\/activitystrea.ms\/schema\/1.0\/collection","id":"http:\/\/activityschema.org\/collection\/public"}],"status_net":{"conversation":"tag:gnusocial.jp,2025-04-27:objectType=thread:nonce=dd1265e94d74ca87","notice_info":{"local_id":"9720045","source":"ActivityPub"}},"published":"2025-04-27T00:00:29+00:00","provider":{"objectType":"service","displayName":"GNU social JP","url":"https:\/\/gnusocial.jp\/"},"verb":"post","url":"https:\/\/friedcheese.us\/objects\/86d2c6b6-5056-4ddc-8271-585d770c9695"}],"links":[{"url":"https:\/\/gnusocial.jp\/conversation\/4960370","rel":"alternate","type":"text\/html"}]}