Notices by Matthew Green (matthew_d_green@ioc.exchange)

Technical question: does anyone know of (open source) LLMs that have been trained or fine-tuned to speak various binary network protocols?

Looks like Hungary will bring the CSAM reg (“chat control”) — a plan to mass-scan all private messages — back for another vote in the EU Council this Wednesday and Thursday. They’ve picked up support from Italy this time.

In a very short timespan it’s going to be expected that your phone can answer questions about what you did or talked about recently, what restaurants you went to. More capability is going to drive more data access, and people will grant it.

I absolutely do believe that (at least initially), vendors will try to do this privately. The models will live on your device or, like Apple Intelligence, they’ll use some kind of secure outsourcing. It’ll be required for adoption.

One of the things we need to discuss is that LLMs listening to your conversations and phone calls, reading your texts and emails — this is all going to be normalized and inevitable within seven years.

The question is: what do governments and law enforcements do when they realize every single person has a little agent on their phone that can answer literally *any* question about their activities, in simple human language? The “crypto wars” are going to look quaint.

The temptation to legislate government access to this agent will be enormous. It’ll start with heinous crimes like child sexual abuse or terrorism, and it will appear tightly targeted. But it will be applied at massive scale, to millions of people.

Do people who live in Europe ever have to grapple with this insanity? I’ll move tomorrow if the answer is “no”.

@jawnsy @filippo And Java!

@jawnsy @filippo Fortunately there will always be C.

It’s been a while since we had a good 512-bit RSA key controlling anything important, and I’m here for it. https://arstechnica.com/security/2024/08/home-energy-system-gives-researcher-control-of-virtual-power-plant

This is getting out of hand.

Apple, unlike most other mobile providers, has traditionally done a lot of processing on-device. For example, all of the machine learning and OCR text recognition on Photos is done right on your device. 2/

So Apple has introduced a new system called “Private Cloud Compute” that allows your phone to offload complex (typically AI) tasks to specialized secure devices in the cloud. I’m still trying to work out what I think about this. So here’s a thread. 1/

But if you send your tasks out to servers in “the cloud” (god using quotes makes me feel 80), this means sending incredibly private data off your phone and out over the Internet. That exposes you to spying, hacking, and the data hungry business model of Silicon Valley. 4/

The problem is that while modern phone “neural” hardware is improving, it’s not improving fast enough to take advantage of all the crazy features Silicon Valley wants from modern AI, including generative AI and its ilk. This fundamentally requires servers. 3/

TL;DR: it is not easy. Building trustworthy computers is literally the hardest problem in computer security. Honestly it’s almost the only problem in computer security. But while it remains a challenging problem, we’ve made a lot of advances. Apple is using almost all of them. 6/

The solution Apple has come up with is to try to build secure and trustworthy hardware in their own data centers. Your phone can then “outsource” heavy tasks to this hardware. Seems easy, right? Well: here’s the blog post. https://security.apple.com/blog/private-cloud-compute/ 5/

Then they’re throwing all kinds of processes at the server hardware to make sure the hardware isn’t tampered with. I can’t tell if this prevents hardware attacks, but it seems like a start. 8/

The first thing Apple is doing is using all the advances they’ve made in building secure phones and PCs in their new servers. This involves using Secure Boot and a Secure Enclave Processor (SEP) to hold keys. They’ve presumably turned on all the processor security features. 7/

A second protection is that the operating system can “attest” to the software image it’s running. Specifically, it signs a hash of the software and shares this with every phone/client. If you trust this infrastructure, you’ll know it’s running a specific piece of software. 10/

They also use a bunch of protections to ensure that software is legitimate. One is that the software is “stateless” and allegedly doesn’t keep information between user requests. To help ensure this, each server/node reboot re-keys and wipes all storage. 9/