Conversation

Notices

1. Embed this notice
   jonny (jonny@social.coop)'s status on Monday, 10-Jun-2024 17:27:15 JST jonny

   More fun publisher surveillance:
   Elsevier embeds a hash in the PDF metadata that is *unique for each time a PDF is downloaded*, this is a diff between metadata from two of the same paper. Combined with access timestamps, they can uniquely identify the source of any shared PDFs.

   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:30 JST jonny

     in reply to

     you go to school to study "the brain" and then the next thing you know you're learning how to debug surveillance in PDF rendering to understand how publishers have so contorted the practice of science for profit. how can there be "normal science" when this is normal?

   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:31 JST jonny

     in reply to

     updated the above gist with correctly extracted tags, and included python code to extract your own, feel free to add them in the comments. since we don't know what they contain yet not adding other metadata. definitely patterned, not a hash, but idk yet.
     https://twitter.com/json_dirs/status/1486289288115359747?t=QwmBvbOgh2fCkjSOZSh3Fw&s=19

     Abhiseck Paira :gnu: :gnuhurd: repeated this.
   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:32 JST jonny

     in reply to

     of course there's smarter watermarking, the metadata is notable because you could scan billions of pdfs fast. this comment on HN got me thinking about this PDF /OpenAction I couldn't make sense of earlier, on open, access metadata, so something with sizes and layout...

   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:33 JST jonny

     in reply to

     this is the way to get the correct tags:
     (on mac i needed to install gnu grep with homebrew `brew install grep` and then use `ggrep` )
     will follow up with dataset tomorrow.
     https://twitter.com/horsemankukka/status/1486268962119761924?s=20

   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:34 JST jonny

     in reply to

     https://twitter.com/SchmiegSophie/status/1486206774159970305?t=GT8fV5QG-4SGTkLadYpCNQ&s=19

   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:36 JST jonny

     in reply to

     https://twitter.com/kmagnacca/status/1486209676979032064?t=GT8fV5QG-4SGTkLadYpCNQ&s=19

   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:37 JST jonny

     in reply to

     for any security researchers out there, here are a few more "hashes" that a few have noted do not appear to be random and might be decodable. exiftool apparently squashed the whitespace so there is a bit more structure to them than in the OP:
     https://gist.github.com/sneakers-the-rat/6d158eb4c8836880cf03191cb5419c8f

   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:38 JST jonny

     in reply to

     The metadata appears to be preserved on papers from sci-hub. since it works by using harvested academic credentials to download papers, this would allow publishers to identify which accounts need to be closed/secured
     https://twitter.com/json_dirs/status/1486135162505072641?t=Wg5XAzujycz79Cop_ap8vQ&s=19

   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:44 JST jonny

     in reply to

     here's a shell script that recursively removes metadata from pdfs in a provided (or current) directory as described above. For mac/*nix-like computers, and you need to have qpdf and exiftool installed:
     https://gist.github.com/sneakers-the-rat/172e8679b824a3871decd262ed3f59c6

   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:45 JST jonny

     in reply to

     Links:
     exiftool: https://www.exiftool.org/
     qpdf: https://qpdf.sourceforge.io/
     dangerzone (GUI, render PDF as images, then re-OCR everything): https://dangerzone.rocks/
     mat2 (render PDF as images, don't OCR): https://0xacab.org/jvoisin/mat2

   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:46 JST jonny

     in reply to

     Also present in the metadata are NISO tags for document status indicating the "final published version" (VoR), and limits on what domains it should be present on. Elsevier scans for PDFs with this metadata, so good idea to strip it any time you're sharing a copy.

   • Embed this notice
     jonny (jonny@social.coop)'s status on Tuesday, 11-Jun-2024 04:18:47 JST jonny

     in reply to

     You can see for yourself using exiftool.
     To remove all of the top-level metadata, you can use exiftool and qpdf:

     exiftool -all:all= <path.pdf> -o <output1.pdf>
     qpdf --linearize <output1.pdf> <output2.pdf>

     To remove *all* metadata, you can use dangerzone or mat2