Conversation

Notices

1. Embed this notice
   Paul Cantrell (inthehands@hachyderm.io)'s status on Thursday, 10-Apr-2025 01:11:52 JST Paul Cantrell

   • Steve Bellovin

   All this from @SteveBellovin! Furthermore, the existence of this “Database of Ruin” — a panopticon that merges every available source of data about every individual — would lead to far more false positives, and far more confidence in those false positives.

   Data quality is always a problem. Overconfidence is data is always a problem. A data panopticon multiplies both those problems. The Database of Ruin doesn’t just connect individuals to closely guarded secrets; it will inevitably •create• completely false secrets out of thin air, at scale. https://infosec.exchange/@SteveBellovin/114308854802931253

   • Embed this notice
     Steve Bellovin (stevebellovin@infosec.exchange)'s status on Thursday, 10-Apr-2025 01:25:58 JST Steve Bellovin

     in reply to

     @inthehands Yes. See slides 21-25 of https://www.cs.columbia.edu/~smb/classes/f23/l_intro.pdf

   • Embed this notice
     Paul Cantrell (inthehands@hachyderm.io)'s status on Thursday, 10-Apr-2025 01:25:58 JST Paul Cantrell

     in reply to

     • Steve Bellovin

     @SteveBellovin
     Good slides! But I don’t see the part that talk about how inferences can be wrong, can thus fabricate conclusions? (Maybe just missing it?)

   • Embed this notice
     Steve Bellovin (stevebellovin@infosec.exchange)'s status on Thursday, 10-Apr-2025 01:29:26 JST Steve Bellovin

     in reply to

     @inthehands Sorry—that's slide 27 of https://www.cs.columbia.edu/~smb/classes/f23/l_ml.pdf

   • Embed this notice
     Paul Cantrell (inthehands@hachyderm.io)'s status on Thursday, 10-Apr-2025 01:30:32 JST Paul Cantrell

     in reply to

     • Steve Bellovin

     @SteveBellovin
     Yes, that’s the thought! ML, and also more mundane inference / data linking / disaggregation.

   • Embed this notice
     Paul Cantrell (inthehands@hachyderm.io)'s status on Thursday, 10-Apr-2025 01:38:03 JST Paul Cantrell

     in reply to

     We have a simple example of false inference in the news right now: the false inference that “Venezuelan” + “has a tattoo” must mean “gang member.” It’s a foolish inference, •highly• likely to be incorrect — but that didn’t stop the fascists at ICE from plucking people off the street and disappearing them to a foreign hell-prison with little hope of return.

     In the wrong hands, a dubious inference leads to confident, irreversible harm.

   • Embed this notice
     Paul Cantrell (inthehands@hachyderm.io)'s status on Thursday, 10-Apr-2025 01:40:30 JST Paul Cantrell

     in reply to

     That’s individuals making bad inferences about other individuals.

     Now imagine these same kind of people turned loose with a population-sized database, making those same kinds of bad inferences — but at population scale, about millions of people at once.

   • Embed this notice
     Paul Cantrell (inthehands@hachyderm.io)'s status on Thursday, 10-Apr-2025 01:46:33 JST Paul Cantrell

     in reply to

     Note that for an authoritarian, false positives are •feature•, not a bug. •Everyone• should be terrified all the time (cf the Soviet joke about camels and foxes).

     This is authoritarian promise that Palantir and Peter Thiel have been pursuing.

     This is one of the underlying reasons for Doge’s illegal seizure of control of computer systems.

     Read this, and imagine these people with their hands on •all• the data that’s in possession of the IRS, social security, Medicare, the whole federal gov:

     https://www.404media.co/inside-a-powerful-database-ice-uses-to-identify-and-deport-people/