Conversation

Notices

1. Embed this notice
   Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:14:00 JST Matthew Green

   So Apple has introduced a new system called “Private Cloud Compute” that allows your phone to offload complex (typically AI) tasks to specialized secure devices in the cloud. I’m still trying to work out what I think about this. So here’s a thread. 1/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:50 JST Matthew Green

     in reply to

     As best I can tell, Apple does not have explicit plans to announce when your data is going off-device for to Private Compute. You won’t opt into this, you won’t necessarily even be told it’s happening. It will just happen. Magically.

     I don’t love that part. 17/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:51 JST Matthew Green

     in reply to

     Quick intermission: I also posted this thread on Twitter, and then this happened right about this part. It was… uncomfortable.

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:51 JST Matthew Green

     in reply to

     Back to the substance.

     I admit that as I learned about this feature, it made me kind of sad. The thought that was going through my head was: this is going to be too much of a temptation. Once you can “safely” outsource tasks to the cloud, why bother doing them locally. Outsource everything! 16/

     this.ven repeated this.
   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:52 JST Matthew Green

     in reply to

     Ok there are probably half a dozen more technical details in the blog post. It’s a very thoughtful design. Indeed, if you gave an excellent team a huge pile of money and told them to build the best “private” cloud in the world, it would probably look like this. 14/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:52 JST Matthew Green

     in reply to

     But now the tough questions. Is it a good idea? And is it as secure as what Apple does today? And most importantly: can users opt out of this thing? 14/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:54 JST Matthew Green

     in reply to

     Security researchers will get *some code* and a VM they can use to run the software. They’ll then have to reverse-engineer the binaries to see if they’re doing unexpected things. It’s a little suboptimal. 12/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:54 JST Matthew Green

     in reply to

     When your phone wants to outsource a task, it will contact Apple and obtain a list of servers/nodes and their keys. It will then encrypt its request to all servers, and one will process it. They use a load balancer to make sure no single server processes all your requests. They’re even using fancy anonymous credentials and a third part relay to hide your IP. 13/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:55 JST Matthew Green

     in reply to

     Of course, knowing that the phone is running a specific piece of software doesn’t help you if you don’t trust the software. So Apple plans to put each binary image into a “transparency log” and publish the software.

     But here’s a sticky point: not with the full source code. 11/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:56 JST Matthew Green

     in reply to

     They also use a bunch of protections to ensure that software is legitimate. One is that the software is “stateless” and allegedly doesn’t keep information between user requests. To help ensure this, each server/node reboot re-keys and wipes all storage. 9/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:56 JST Matthew Green

     in reply to

     A second protection is that the operating system can “attest” to the software image it’s running. Specifically, it signs a hash of the software and shares this with every phone/client. If you trust this infrastructure, you’ll know it’s running a specific piece of software. 10/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:57 JST Matthew Green

     in reply to

     The first thing Apple is doing is using all the advances they’ve made in building secure phones and PCs in their new servers. This involves using Secure Boot and a Secure Enclave Processor (SEP) to hold keys. They’ve presumably turned on all the processor security features. 7/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:57 JST Matthew Green

     in reply to

     Then they’re throwing all kinds of processes at the server hardware to make sure the hardware isn’t tampered with. I can’t tell if this prevents hardware attacks, but it seems like a start. 8/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:58 JST Matthew Green

     in reply to

     The solution Apple has come up with is to try to build secure and trustworthy hardware in their own data centers. Your phone can then “outsource” heavy tasks to this hardware. Seems easy, right? Well: here’s the blog post. https://security.apple.com/blog/private-cloud-compute/ 5/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:58 JST Matthew Green

     in reply to

     TL;DR: it is not easy. Building trustworthy computers is literally the hardest problem in computer security. Honestly it’s almost the only problem in computer security. But while it remains a challenging problem, we’ve made a lot of advances. Apple is using almost all of them. 6/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:59 JST Matthew Green

     in reply to

     The problem is that while modern phone “neural” hardware is improving, it’s not improving fast enough to take advantage of all the crazy features Silicon Valley wants from modern AI, including generative AI and its ilk. This fundamentally requires servers. 3/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:13:59 JST Matthew Green

     in reply to

     But if you send your tasks out to servers in “the cloud” (god using quotes makes me feel 80), this means sending incredibly private data off your phone and out over the Internet. That exposes you to spying, hacking, and the data hungry business model of Silicon Valley. 4/

   • Embed this notice
     Matthew Green (matthew_d_green@ioc.exchange)'s status on Wednesday, 12-Jun-2024 03:14:00 JST Matthew Green

     in reply to

     Apple, unlike most other mobile providers, has traditionally done a lot of processing on-device. For example, all of the machine learning and OCR text recognition on Photos is done right on your device. 2/