Kevin Beaumont
For those who aren’t aware, Microsoft have decided to bake essentially an infostealer into base Windows OS and enable by default.
From the Microsoft FAQ: “Note that Recall does not perform content moderation. It will not hide information such as passwords or financial account numbers."
Info is stored locally - but rather than something like Redline stealing your local browser password vault, now they can just steal the last 3 months of everything you’ve typed and viewed in one database.
JJ
@GossiTheDog At no point in this video do they mention anyone asking for this. We didn't.
I love that they're leaning hard on the fact that the data is protected from other users. That... doesn't matter. You're running automated enumeration for anyone that gets access under my user context. Don't do that. Jesus.
Kevin Beaumont
I've written up my thoughts on the Copilot Recall feature in Microsoft Copilot+ PCs
I think it will enable fraud and endanger users, and is not the sign of a company who are committed to security first.
Kevin Beaumont
The UK’s ICO have opened an investigation into Copilot+ Recall. https://www.bbc.co.uk/news/articles/cpwwqp6nx14o
Ditol
@GossiTheDog Except they didn't bake it into into base Windows OS. Yet. It's only in their "Ai PCs". Unclear for now long.
Kevin Beaumont
Copilot+ Recall has been enabled by default globally in Microsoft Intune managed users, for businesses.
You need to enable DisableAIDataAnalysis to switch it off. https://learn.microsoft.com/en-us/windows/client-management/manage-recall
Michał "rysiek" Woźniak · 🇺🇦
@tokyo_0 It's not available in Home edition.
Yet.
Kevin Beaumont
Here’s Copilot+ Recall search in action, showing instant text based search finding a WhatsApp chat and a PDF from 6 months ago being viewed on screen.
Anthony, of course
@GossiTheDog wait, what? How is it showing something from 6 months ago? Tell me this is internal/mocked up
Kevin Beaumont
Two quick updates -
A) if you disallow recording of a website in Control Panel or GPO, in Chrome it is still recorded - disallow recording only works in Edge browser
B) Firefox and Tor Browser is recorded always, including in private mode - the exception is Hollywood DRM’d videos
Kevin Beaumont
I got ahold of the Copilot+ software.
Recall uses a bunch of services themed CAP - Core AI Platform. Enabled by default.
It spits constant screenshots (the product brands then “snapshots”, but they’re hooked screenshots) into the current user’s AppData as part of image storage.
The NPU processes them and extracts text, into a database file.
The database is SQLite, and you can access it as the user including programmatically. It 100% does not need physical access and can be stolen.
plat
@GossiTheDog so could you tell windows your application contains DRMed content to prevent it from taking screenshots?
Xyleth
@GossiTheDog they’re not even doing anything interesting like vectorising the scraped text for use in an on-board RAG type system? It’s just a DL OCR and plain old search?
Still can’t see why any business that is vaguely sane would allow this on their fleet, or why any personal user would either. It screams privacy accident waiting to happen.
Tom Bellin :picardfacepalm:
@vees @GossiTheDog This is like an intern's summer project. A proof of concept for a grant.
Rob Carlson :ally: :BLM:
@GossiTheDog We're all shocked to find that the Core AI Platform is built on SQLite, curl, and text files right?
uzayran
@GossiTheDog programmatically as in any program running can access everything I've been doing for the last 3 months? Fun.
Michael Fisher
@GossiTheDog Can you set a scheduled task to delete the SQLite DB or Delete * From * every time you reboot?
mark
@GossiTheDog if the user can access this, then why dont we all just trash it constantly.
mark
@GossiTheDog it's harder to remember to change the setting back constantly than to write a script that constantly zeroes this out
Kevin Beaumont
And if you didn’t believe me.. found this on TikTok.
There’s an MSFT employee in the background saying “I don’t know if the team is going to be very happy…”
System Adminihater
@GossiTheDog All they care about is money and they are too big to take to court. Capitalism.
gwire
@ianbetteridge @GossiTheDog Given this requires a new sort of PC, it's not impossible that some people can gain the impression that this stuff lives in a special, inaccessible, place. e.g. how Apple talks about the Secure Enclave.
Ian Betteridge
@gwire @GossiTheDog It *sort of* does, I think… not inaccessible but *less* accessible. From the video Kevin reposted, it shows that to access the stored data file, you need to explicitly put in your Admin password – even to get to in in Windows Explorer. That means, I believe any app which wants to reach it also has to request access.
Ian Betteridge
@GossiTheDog Hmmm. But that’s true for *anything* on your file system. And AFAIK no one has yet invented a way to store info locally that isn't on your file system ;)
So yes, a compromised machine where someone has set up remote access to it without you knowing would allow them to spy on your activity. But that is true today, too, on any machine, on any platform.
Plus if someone has remote access, they would be WAY better off installing akeylogger than relying on a feature I can turn off.
Ian Betteridge
@GossiTheDog @gwire Explain (or don’t, which is also good)
unlucio 🌍 :mastodon:
@GossiTheDog I never doubted for a second that it was actually being done on the NPU.
What I'll _never_ believe is that they'll not be using that stuff to monetize their users. More banners spammed to users, more value for shareholders.
It worths remembering Microsoft's main strategy: Embrace, Extend, Extinguish
This time the target is humans, because fewer humans to pay, more value for the shareholder.
Thomas
@GossiTheDog Can you translate from Microsoft? Does this mean that corporate users will be able to turn it on/off individually, or will it be up to the IT department
Kevin Beaumont
I ponder if Microsoft's engineers are following the SQLite code of ethics, since they're using it in Windows OS with Copilot+ Recall? :D
Thibault D.
@Powareverb @gerowen @GossiTheDog @shanselman that's fine he's the one doing transparency here and I would love to read his thoughts about the privacy concerns of Recall.
Scott Hanselman 👸🏽🐝🌮
@ThibaultDu @Powareverb @gerowen @GossiTheDog I don’t work on the project but I find the NPU tech and the open SDKs behind it (and onyx runtime) interesting. My opinion is it should be not just opt-in but something you download explicitly and install if you want it. Similar to RescueTime and TimeSnapper and AugmenD and other apps that have done this stuff for years (using OCR). This should be as secure as your browser history, encrypted at rest, non roaming, etc.
Gavin Jones :tinoflag:
@gerowen @GossiTheDog he's Scott Hanselman, I won't like to his profile but yes he's on Mastodon I belive.
Marcus Adams
@GossiTheDog I forget his name but the guy talking is actually here on Mastodon. He at least was, and I think still is an employee at Microsoft.
Kevin Beaumont
So the code underpinning Copilot+ Recall includes a whole bunch of Azure AI backend code, which has ended up in the Windows OS. It also has a ton of API hooks for user activity monitoring.
Apps themselves can also search and make themselves more searchable.
It opens a lot of attack surface.
The semantic search element is fun.
They really went all in with this and it will have profound negative implications for the safety of people who use Microsoft Windows.
Crows Labyrinth
@GossiTheDog I have only one simple question: is there a kill switch to turn off this Copilot Recall feature and turn it off completely, or are we stuck with this permanently running spyware?
SeattleSwiftie
@GossiTheDog what?! You're telling me that Microsoft is being the truth on what "On your local PC" means? That they'll just hand your data off to other partners and apps. Shocked. Absolutely shocked.
Interiorce
@GossiTheDog luckily for now it works only on a specific laptop that I would never buy
Kevin Beaumont
If you want to know where tech companies are with AI safety, know Microsoft Recall won’t record screenshots of DRM’d movies..
..but will record screenshots of your financial records and WhatsApp messages, as corporate interests were prioritised over user safety.
And it’s enabled by default.
Natasha Nice
@GossiTheDog really?
Natasha Nice
@GossiTheDog I keep my privacy with Onion browser
📷🦿 | kwer
@GossiTheDog @Tatianalaurent Could it be that they just reuse the capabilities of their screenshot tools and that’s the reason why it excludes DRM content? I’d wager a guess that they wanted to ship this fast and not re-develop things..
Chris
@GossiTheDog It'll be interesting to see what this OOBE actually is. They seem to imply you could turn it off, but given how OneDrive (as example) seems to force folder sync how clear will it be turning this off is possible? https://support.microsoft.com/en-us/windows/privacy-and-control-over-your-recall-experience-d404f672-7647-41e5-886c-a3c59680af15
Urzl
@GossiTheDog Most users of Windows aren't equipped to take these warnings seriously.
They hear very regularly both how terrible Windows security is and how little they have to really worry about it as a private citizen with no big visible public persona.
This will work basically like Y2k - everybody freaks out briefly, the furor dies down after some very smart people do tons of mitigation, and then the layperson won't even believe anything happened.
Urzl
@GossiTheDog I'm aware, I'm just feel like it's a perspective we need to keep in mind.
There's been such a harsh consequence for users straying off the beaten path that it is easy to get a cadre of power users motivated but then to not being able to reach anybody else.
New Year, Old Jen
@GossiTheDog so there's no actual way to turn this off at all then??
Fellows
@GossiTheDog sounds like a massive shit show waiting to happen. I better stock up on🍿 !
Ashley Walker
@GossiTheDog does it screenshot RDP sessions? Because if it does, this is going to cause headaches on some of my clients who allow BYOD devices to remote into the Terminal Server.
Kevin Beaumont
I’ve managed to get Recall working in full on a non-Copilot+ system, without an NPU. Will accelerate testing.
Albert ARIBAUD ✎
@GossiTheDog Have they addressed already how his... feature... articulates with the GDPR?
docht
Interested layperson here:
From what you've seen so far, can you draw a conclusion that a computer with recall running sends more data than usual to Microsoft, maybe "disguised" as part of diagnostics data for example.
I wonder if recall is a fishing expedition in task mining and if so, how Microsoft, despite having promised that recall does not run in the cloud, could get the data nevertheless.
Interiorce
@GossiTheDog but all the same stuff with the same Qualcomm CPU. Just took the reference design and placed in a shell
Steven Bodzin
@GossiTheDog thanks for this work, Kevin.
As a reporter, I'm especially concerned about what happens when we cross borders with our computers. At least here in the US, border agents have the right to inspect and copy anything they want from computers. This system appears to make it impossible to delete drafts or notes, use disappearing messages, or otherwise clean a computer to protect sources and clients.
Kevin Beaumont
Copilot+ Recall feature pop quiz:
You deal with a sensitive matter on my Windows PC. E.g. an email you delete. Does Copilot Recall still store the deleted email?
Answer: yes. There's no feature to delete screenshots of things you delete while using your PC. You would have to remember to go and purge screenshots that Recall makes every few seconds.
If you or a friend use disappearing messages in WhatsApp, Signal etc, it is recorded regardless.
Steven Bodzin
@GossiTheDog how does a Windows user know if this "functionality" is turned on?
Kevin Beaumont
It comes up a lot as people are rightly confused, but if you wonder what problem Microsoft are trying to solve with Recall:
It isn't them being evil, it's business leaders who are middle aged and can't remember what they're doing driving decision making about which problems to solve.
A huge amount of business leaders are dudes who have no idea what the fuck is happening. This leads to the Recall feature.
Microsoft exists and is driven by that bubble.
Dave
@GossiTheDog how exactly is a fire hose of screenshots going to help with this?
Kevin Beaumont
I asked Microsoft Copilot to write a song about Copilot+ Recall.
Noah Cook
@GossiTheDog Most government agencies have policies and procedures for destroying/disposing of sensitive documents. I'm not even talking about classified natsec stuff, just basic things like PHI at health agencies, or the PII stored by tax, driving, and municipal utility agencies.
Improper disposal of those documents can be every bit as serious as improper disclosure of classified documents. Either Microsoft has sat down with attorneys and spent months going over liability issues, or...
Misuse Case
@Laird_Dave If @GossiTheDog is correct about who drove the decision to create Recall and why (and IMO it’s very plausible), then it means business leaders don’t talk to their legal departments, CISOs, records managers, or any of the other roles in their organizations that could have told them why this was bad.
Dave
@MisuseCase @GossiTheDog yeah, that would probably help a lot more while being a lot less creepy (and liability risk)
In turn, the liability risk will be the nicest "feature" of this shit. I'll drive a wooden stake through Recalls heart for my org before it has a chance to lay eggs.
Misuse Case
@GossiTheDog @Laird_Dave So this is definitely geared towards not just middle-aged dudes but managers.
Like, can these people not just organize their emails into folders by topic? Microsoft could have re-worked the Outlook rules function to make this easier instead of whatever this is.
Misuse Case
@GossiTheDog @Laird_Dave Sure, I can see that. But Microsoft has a lot of enterprise customers with CISOs, legal departments, regulatory requirements, etc. for whom Recall is worse than useless. That actually describes most of their largest enterprise customers!
Do they even pay attention to their own customers at all?
Sure enterprises can use GPO to turn it off but why make something that most of your biggest customers are going to have to turn off?
Athanasius
@GossiTheDog Not just the screenshots, also any references that might contain data from the SQLite DB, right ?
I assume that DB *must* at least have keywords in it to know which screenshot(s) to pull up for a given query.
Athanasius
@GossiTheDog Yup, I figured *you* knew this, just it bears stating for anyone else.
If people start thinking "oh, just nuke the screenshots folder manually" they might get a surprise when a bunch of data can still be pulled out of the SQLite database.
Allpoints
@GossiTheDog respectfully, I can't tell if you're joking or not. The idea that Microsoft is making business decisions to be helpful to anyone indicates a fundamental misunderstanding of how large companies work. They do things to make money. If they happen to help someone in the process that's fine but that's a byproduct, not the goal
Allpoints
@GossiTheDog just because the raw data doesn't leave the device doesn't mean they can't use it for monetization. In fact, the ability to offload user profiling to the device is a tremendous benefit to the advertising space. It's fantastic for scaling and reducing your infrastructure costs.
I get the impression we're not going to agree on this one but that's okay
Mia
@GossiTheDog imagine if they actually did it to help menopausal, chemotherapy and long covid etc folk living with brain fog! (I don't believe it for a second but still, imagine...)
Susan Vanderplas
@GossiTheDog @UncivilServant so now everyone in healthcare has to switch to Mac or Linux to avoid this liability minefield. Brilliant!
Mans R
@GossiTheDog Could a messaging (or whatever) app flag its window as containing DRM material and thus avoid being captured?
Kevin Beaumont
Managed to find out how BBC News printed in a headline story that it was not possible to steal Recall data without being physically at the device (which is false) - this is from the journalist:
Kevin Beaumont
Some screenshots of Recall's SQLite database here: https://mastodon.social/@detective/112513529733646088
Just to clarify, I can access it without SYSTEM too. Microsoft are about to set cybersecurity back a decade by empowering cyber criminals via poor AI safety. Feature ships in a few weeks.
Adam ♿
@GossiTheDog Hanselman burning decades of good will for that one, wow.
arialdo
@GossiTheDog Is it really enabled by default? How to check if it is enabled and running?
Janneke
@GossiTheDog nobody cares.
As if using Microsoft was anywhere near a sane choice a year ago wrt privacy, security, user freedom,...
Kevin Beaumont
The latest Risky Business episode on Recall is good, but one small correction - it doesn’t need SYSTEM rights.
Here’s a video of two MSFT employees gaining access to the Recall database folder - with SQLite database right there. Watch their hacking skills. (You don’t need to go this length as an attacker, either). Cc @riskybusiness
Novyx
@GossiTheDog @riskybusiness @hacks4pancakes I continue to be amazed by how apt the product name is, and eagerly await the release of the Microsoft Recall Recall.
Kevin Beaumont
Stealing everything you’ve ever typed or viewed on your own Windows PC is now possible with two lines of code — inside the Copilot+ Recall disaster.
My look at the feature, FAQs from the community etc
Andrew Bartlett
@GossiTheDog I wonder if https://www.microsoft.com/en-gb/windows/copilot-plus-pcs#shop explains the inability to walk this back.
It is one thing to walk it back after a pre launch preview and an insider build, but to do so after a panel of hardware vendors have built production laptops is another thing entirely.
I also wonder if the at edge dragnet is part of trying to do safer, personalised AI, like we would want, and a competitive advantage compared to Google, but which shows that would never be safe either.
samiamsam
@GossiTheDog so if you happen to be looking at your medical records.....
ummmmmmm.........
Kevin Beaumont
this is the out of box experience for Windows 11's new Recall feature on Copilot+ PCs. It's enabled by default during setup and you can't disable it directly here. There is an option to tick "open Settings after setup completes so I can manage my Recall preferences" instead.
HT @tomwarren
FRANK.MCCONNEL
@GossiTheDog Microsoft think most users run as admin? Srsly, having set up Windows systems so the users don't get to run admin and then tried to set up some third-party software for them, I think there's certainly third-party developers who think all users run as admin.
Justinmwhitaker
@hairyvisionary @GossiTheDog I think they’re right.
Most users don’t understand the difference between a normal user and admin, they just want to install their apps. So they run as admins.
The only time they are saved from themselves is if they’re running a work rig that’s managed by IT.
CaveDave
@GossiTheDog @tomwarren I really wanna know how this gets past the DMA in the EU
Kevin Beaumont
You allow BYOD so people can pick up webmail and such. It’s okay, because when they leave you revoke their access, and your MDM removes all business data from the machine ✅
What the employee does: opens Recall, searches their email, files etc and pastes the data elsewhere.
Nothing is removed from Recall, as it is a photographic memory of everything the former employee did.
Fellows
@GossiTheDog these are things folks really need to consider. A big thanks Kevin for bringing all of these considerations to light!
8tpercent
@GossiTheDog I'd posted in a discord, your fab link. Someone replied to me that i shouldn't be alarmed because... :blobthinkingeyes:
Kevin Beaumont
Just in time for Copilot+ Recall!
Daniel Feldman
@GossiTheDog Thanks for the awesome thread! I don't have an ARM machine other than my Mac, but I was able to get Recall running in an ARM Azure VM very easily.
It's interesting that this is an ARM-only feature actually! It runs fine in a fairly small Azure VM with no GPU or NPU, so it would work on x86 just as well. It must be a marketing/strategy reason, not a technical reason.
Kevin Beaumont
Security and privacy researchers - You can now install Copilot+ Recall on any ARM hardware (doesn’t need an NPU) or in Azure VMs.
Guide from @detective
The devices launch THIS MONTH to customers so I suggest people look at this.
Johannes
@dfeldman @GossiTheDog there is a github project with explanations on how to install it in a VM on ARM Macs and a bunch of other platforms: https://github.com/thebookisclosed/AmperageKit
Kevin Beaumont
Nvidia just announced that Copilot+ and Recall are coming to AMD systems. https://www.theverge.com/2024/6/2/24169568/microsoft-copilot-plus-gaming-pc-nvidia-amd
Edgar Whelp
@GossiTheDog no mention as to why the hell gaming PCs need Copilot+ taking screenshots of their CoD game every few seconds.
Kevin Beaumont
Somebody made a tool called Total Recall to dump Recall database and screenshots.
tuban_muzuru
Knowing what we do about Copilot+Recall, I cannot imagine any competent CIO allowing it on the corporate premises.
Fish of Rage
Jon Greig
@GossiTheDog it’s like they got a focus group of cybercriminals together when making this
Marius Kießling
@GossiTheDog my only light at the end of the tunnel at this point is that big orgs will still take some time before they have any devices running the software and hopefully Microsoft gets their shit together for a more secure implementation until then.
?? Humpleupagus ??
Zergling_man
munir
Zergling_man
Kevin Beaumont
Recent DHS published report handed to the US President which said it had "identified a series of Microsoft operational and strategic decisions that collectively pointed to a corporate culture that deprioritized enterprise security investments and rigorous risk management"
Microsoft: let’s use AI to screenshot everything users do every 5 seconds, OCR the screenshots, make it searchable and store it in AppData!
Kevin Beaumont
Searching Recall database for passwords.
ISO8601
@GossiTheDog With Recall I'm unsure how to handle suppliers remoting in to provide support.
We can't control whether Recall is enabled on *their* PC.
We'd need a remote control app which specifically won't proceed if Recall is enabled on the viewer's side.
Kevin Beaumont
🫡
Kevin Beaumont
If anybody is wondering if you can enable Recall on a machine remotely without Copilot+ hardware support - yep.
I’ve also found a way to disable the tray icon.
Kevin Beaumont
I went and looked at YouTube for Recall to get out of the echo chamber and I can only find one positive video. Even the people at the event are slating it, including people with media provided Copilot+ PCs.
There’s some content creators who’ve realised it records their credit cards, so they’re making videos of their cards going walkies.
acquirer
Kevin Beaumont
It’s going to be interesting to see how Microsoft get out of this one. They may have contractual commitments to ship Recall with external parties.
I thought they were risking crashing the Copilot brand with this one, but I was wrong looking at the videos and comments on them - I think they’re crashing the Windows consumer brand.
The reaction to photographic memory of what people do at home has - you’ll be surprised to know - not been seen as a reason to buy a device, but a reason why not to.
Forgi :neofox_woozy:
@GossiTheDog It sucks because the prospect of good ARM laptops that aren’t made by Apple is finally here, but this puts them in jeopardy if sales tank due to Recall.
Longhorn
@GossiTheDog it's too late at this point. MS _will_ ship Recall, but what happens afterwards will be fun to see.
Nicole Parsons
Investments by oil despots yields products that despots want.
https://www.businessinsider.com/microsoft-google-hand-dissident-data-to-saudi-arabia-activists-say-2023-7
https://www.arabnews.com/node/2507356/business-economy
https://www.consultancy-me.com/news/8148/pwc-and-microsoft-launch-ai-centre-of-excellence-in-riyadh
Nicole Parsons
Saudi Arabia had been flooding American tech companies with cash since 2018.
Twitter was just one example of anti-democracy oil oligarchs hijacking tech.
Kushner's $2 billion in Saudi sovereign funds isn't buying beach-front condos in Gaza, it's being spent on torpedoing tech brands like Microsoft.
Google, Apple, Oracle, Amazon, Microsoft, all had MBS visit in 2018. The investments continued after the Khassoghi murder & its accelerated in recent months.
https://www.vox.com/technology/2023/5/1/23702451/silicon-valley-saudi-money-khashoggi
Kevin Beaumont
Windows Central, about the only outlet giving Recall positive coverage and having articles tweeted by Microsoft staff - have updated their take after being hands on with a device. https://www.windowscentral.com/software-apps/windows-11/microsoft-should-recall-windows-recall-security-researcher-finds-microsofts-new-ai-tool-woefully-insecure
João Tiago Rebelo (NAFO J-121)
@GossiTheDog they still say finding Recall data scrappers is unlikely... it might be right now, but as you say it won't be in the future if MS doesn't correct this App Data plain text SQLite database fiasco.
Kevin Beaumont
Microsoft has been declining to comment on criticism of Recall for a week - but they have apparently told a journalist at Future that changes will be made before Copilot+ devices drop in the coming days.
This may include an attempt to invalidate researcher criticism, we’ll see.
Kevin Beaumont
WIRED has a piece about Total Recall, a now released tool which dumps keypresses, text and screenshots (they’re JPEGs) from Microsoft Recall
https://www.wired.com/story/total-recall-windows-recall-ai/
Total Recall software by @xaitax https://github.com/xaitax/TotalRecall
Example search for ‘password’:
🪟 Captured Windows: 133
📸 Images Taken: 36
🔍 Search results for 'password': 22
📄 Summary of the extraction is available in the file:
C:\Users\alex\Downloads\TotalRecall\2024-06-04-13-49_Recall_Extraction\TotalRecall.txt
Tom Dewar
@GossiTheDog if it is not off by default for all enterprise users they are grossly negligent
Kevin Beaumont
I hadn’t been aware until today of the external reaction to Recall. Holy shit. Tim Apple must be pleased.
Everything from media coverage to YouTube to TikTok is largely negative. All the comments are negative.
These videos have tens of millions of views and hundreds of thousands of comments.
I knew it would be bad but.. it’s worse. I’ve spent hours looking at the sentiment and.. well, they probably would have got better coverage from launching an NFT of pregnant Clippy.
Fellows
@GossiTheDog Macrumors.com picked up your article along with many other places I’m sure.
Grant Gulovsen
@GossiTheDog Maybe there's more going on behind the scenes, but other than the OCR, what part of this has anything to do with "AI?" Or is that Microsoft's way of trying to put an already questionable color of lipstick that its shareholders like on a really ugly and dangerous pig?
Kevin Beaumont
A key element of Recall is Microsoft say only you can access your Recall, it is per user.
ArsTechnica enabled Recall on Windows 11 box and tested the claim. By logging in as another user they could access the database and screenshots.
Kevin Beaumont
If you want to know how Microsoft have got themselves into this giant mess with Recall, here’s what the documentation says between the lines:
you, the customer, are a simpleton who doesn’t want to be an AI genius yet. Have a caveman mode.
https://gigatexal.blog -he/him
@GossiTheDog I’m so scatter brained and stressed I no joke could benefit from this.
Kevin Beaumont
Recall and Copilot+ is also coming to ASUS systems, including AMD, in a deal with Microsoft.
ASUS Announces Complete Portfolio of AI-Powered Copilot+ PCs https://www.asus.com/us/news/pnm9tg6qccql6ern/
Nvidia announced they are bringing Copilot+ and Recall to PCs, in a deal with Microsoft: https://www.theverge.com/2024/6/2/24169568/microsoft-copilot-plus-gaming-pc-nvidia-amd
goedelchen
@GossiTheDog How does Recall treat Citrix Workspace sessions? (I mean, I'm using my private computer to log into my works desktop)
I'm sure my employer would be very happy to learn that it's recorded on my private machine when I access client data.
翠星石
Antifa Franklin 🇵🇸
@GossiTheDog year of the linux desktop baby
翠星石
Janneke
@GossiTheDog if you didn't move to GNU/Linux when Windows95 crashed 5 times a day, and didn't after the '99 finding of facts (fof) by judge Jackson, what makes you think people will choose privacy, security, freedom now?
I'm sure people will continue to like sensationalism...
Kevin Beaumont
Three Copilot+ Recall questions that keep coming up.
Q. Can you alter the Recall history?
A. Yes. You can change the OCR database and change the screenshots as the logged in user or as software running as the local user. There is no audit log of changes.
Q. Are they snapshots, as Microsoft says, or screenshots?
A. They are just screenshots, jpegs.
Q. What is to stop apps on your machine accessing your Recall covertly?
A. Nothing. There is no audit log of access.
Kevin Beaumont
.@awakecoding becomes the latest person reverse engineering Microsoft Recall https://x.com/awakecoding/status/1798168395583746216
Kevin Beaumont
If anybody is wondering what Microsoft's reaction to any of the Copilot+ Recall concerns are, they're continuing to decline comment to every media outlet.
I've seen comments MS staff have been given for enterprise customers, which are nonsense handwaving.
Product ships live on devices from Dell, Lenovo etc this month.
Asta [AMP]
@GossiTheDog@cyberplace.social "Yes. You can change the OCR database and change the screenshots as the logged in user or as software running as the local user. There is no audit log of changes."
One of the very first tasks I had to do at Microsoft was implement an audit log for a service that was used by maybe 200 different people (or groups). It was considered important!
Good thing Windows isn't used by more than 200 people, or
pettter
@GossiTheDog @maswan snälla säg att hela det här debaclet går att använda för att puckla på ITS om UmUs Microsoftberoende. Det är väl ändå uppenbart att de inte går att lita på för fem öre åtminstone när det gäller dataskydd?
Naiara :neocat_flag_trans:
@GossiTheDog@cyberplace.social is doublepulsar.com your blog? can you use the user preferred color scheme instead of just hardcoding white on black?
some ppl have SERIOUS struggles reading white on black, and that includes me, sadly
rmcv42
@GossiTheDog did you catch Steve Gobson’s take on recall, after your wonderful breakdown, on this week’s episode of Security Now episode 977?
Apple link https://podcasts.apple.com/us/podcast/security-now-audio/id79016499?i=1000657874927
Grc’s website 16 kb downloadable page (not there yet though 🤷)
Kevin Beaumont
As @tiraniddo rightly points out, anybody can programmatically reach the Recall database without admin rights. https://infosec.exchange/@tiraniddo/112566044174482506
Kevin Beaumont
TotalRecall has been updated to exfiltrate Recall database and screenshots without needing admin rights: https://github.com/xaitax/TotalRecall
Kevin Beaumont
You can now remotely dump Recall data and screenshots over the internet from Linux etc. Changes in flight for parsing data too.
Kevin Beaumont
YouTubers are continuing to have fun with Recall
Kevin Beaumont
Turns out speaking out works.
Microsoft are making significant changes to Recall, including making it specifically opt in, requiring Windows Hello face scanning to activate and use it, and actually encrypting the database.
There is obviously going to be devils in the details - potentially big ones.
Microsoft needs to commit to not trying to sneak users to enable it in the future, and it needs turning off by default in Group Policy and Intune for enterprise orgs.
https://www.theverge.com/2024/6/7/24173499/microsoft-windows-recall-response-security-concerns
Kevin Beaumont
Obviously, I recommend you do not enable Recall, and you tell your family not to enable it too.
It’s still labelled Preview, and I’ll believe it is encrypted when I see it.
There are obviously serious governance and security failures at Microsoft around how this played out that need to be investigated, and suggests they are not serious about AI safety.
maswan
@pettter Jadu, jag tror det är svårt att rubba på det, med tanke på hur hårt de har gått in för att det bara är Microsoft som gäller.
Bara den mentala gymnastiken för att vara tokstrikta med GDPR för alla tjänsteleverantörer som inte heter Microsoft är ju imponerande i sig, om än tragisk.
pettter
@maswan kan man gå över deras huvud? Kanske kan Frank Drewes trycka på nåt nu när han är prefekt? Facket..?
Kevin Beaumont
Microsoft President Brad Smith is going to be grilled by US gov next week. https://therecord.media/microsoft-reverses-course-recall-opt-in
Suzanne Aldrich (she/her)
@GossiTheDog I read that as “Bard Smith” and imagined the Google AI attending Congressional hearings and what a poor job it would do 😆
evacide
@GossiTheDog This absolutely would not have happened without you.
Kevin Beaumont
I should be transparent btw that I took Satya and Charlie’s commitment to security at face value too - I even published a blog on it backing that up - and I have concerns (it isn’t just me).
They’re now going to have to win trust back about winning trust back.
Kensan
@GossiTheDog @evacide I want to believe there was internal dissent?
Kevin Beaumont
I know somebody at a retailer in Europe that is selling Copilot+ PCs. They’ve had fewer than a thousand preorders through to customers.
In relative terms, for them it’s about as successful as Suicide Squad Kill The Justice League.
Graham Sutherland / Polynomial
@GossiTheDog I can't keep track of which of the batman-extended-universe games flopped at this point so I'm going to assume the sales numbers are bad.
System Adminihater
@GossiTheDog Doesnt matter they have contracts with US feds for cloud. Nobody can make them change. They are too big to hold accountable.
I guess we could all start using Apple PCs but naw?
gigantos
@GossiTheDog which is super annoying to me.
They had the chance to finally sell a powerful #Windows laptop to compete with Apple on battery life and performance.
And what did they do? They made it all about the NPU, so now the only reason to buy it is a feature nobody wants or understands.
Petrus Hilarius
@GossiTheDog And because they are useless morons they will enable it again next week without telling anyone. Trusting Microsoft to do anything but screw users or companies over is ... similarly moronic.
Petrus Hilarius
@GossiTheDog Apparently it already is true one day later? 🤷 https://soapbox.hackdefendr.com/@jeff/112578463669346139 But in the end, it matters not.
Rasmus Lindegaard
@GossiTheDog Encrypting the database, at rest? If anything can get ahold of unencrypted data from the database, its all fubar, no matter what they do.
But it is good that they are changing some things
Wuzzy
@GossiTheDog AI will be remembered by historians as "the thing that generated memes". 🤠
namlaz
@GossiTheDog Makes sense as Copilot is still not available in the European Economic Area, which is fine by me.
namlaz
@GossiTheDog why do multiple sources and OEM themselves they say it isn’t available ?
https://www.asus.com/support/faq/1053182/
https://www.dell.com/support/kbdoc/en-uk/000218761/copilot-in-windows
Kevin Beaumont
A reminder that a few weeks ago at RSA, Microsoft signed CISA's Secure By Design pledge.
Kevin Beaumont
I think MS are a way off extracting themselves from Recall situation they've got themselves into.
This is just one YouTube comments section on a video since the not-enabled-by-default change - 500k views - but there's loads more, similar on TikTok.
I imagine it's going to continue through week and into next week when the laptops ship.
I have heard rumblings MS are discussing trying to take action against me over the whole thing, which a) good luck and b) would be pouring petrol on the flames.
Kevin Beaumont
Some backstory - it's being reported Microsoft developed Recall in secret to try to avoid scrutiny. https://www.windowscentral.com/software-apps/windows-11/microsoft-has-lost-trust-with-its-users-windows-recall-is-the-last-straw
I'm hearing that various MSFT people are furious about how this played out over the past few weeks, which IMHO represents a serious lack of introspection.
Kierkegaanks regretfully
@GossiTheDog let us pray that satya nadella is fired
Kierkegaanks regretfully
@GossiTheDog a man can dream. Nearly all enshittification of windows, office, and microsoft accounts are his doing. (I don’t work with Azure so I have no idea what’s going down there). He is the wizard of oz of pulling profit from users while abusing them.
But if this becomes a threshold moment, an infected wound where everything that’s shit about microsoft now turns people away, and keeps people away, valuation will freefall, Azure-AI or not.
Won’t happen, but I can dream
Walt Wooton
@GossiTheDog Heads need to roll.
Noah Cook
@GossiTheDog That article isn't even good propaganda, I give it a C-.
But yeah, I suspect that this was kept secret from their policy and legal teams as well, because I'm going to assume that the people working on those teams are competent.
I also really, really, really want to know the gender makeup of the supersekrit skunkworks team that tested this. I mean, I feel confident in my hypothesis, but best to await confirmatory evidence.
Kevin Beaumont
Microsoft have paused the rollout of Windows 11 24H2 in preview channel, it was the version containing Recall. Microsoft have not explained why.
https://x.com/brandonleblanc/status/1799478915582542199
I don't know if it was publicly known but it was possible to use Recall on more hardware via Mach2, before this was pulled.
Gloopsies :fedora:
"Speaking out works"
I hate to be that guy but this is a known Microsoft playbook...
1. Advertise something awful that people hate
2. People spread the hate online
3. Microsoft acts as a good guy and tones it down a little bit so people can say that they changed and they are the good guys even if the new way is still awful
4. In a couple of updates return to the original proposal silently
Works every time...
Kevin Beaumont
To put this one into perspective, there's one broadcast TV network looking at Recall still, and an investigative journalist.
Plus I imagine @evacide, @wdormann etc would have something to say if MS tried holding anybody but themselves accountable for their own actions.
System Adminihater
@GossiTheDog I dont understand why they cannot do things correctly.
Kevin Beaumont
Cyber Threat Intelligence 2024 is going well
Kevin Beaumont
I have an image where when viewed on a Copilot+ Recall PC, a Windows process crashes as it tries to process the screenshot.
New email signature?
eddy
@GossiTheDog Teams avatar
Kevin Beaumont
If anybody is wondering, with a Copilot+ PC, you can still programmatically access the Recall database as of today with a few commands. Launch is a few days away.
Andres Jalinton
@GossiTheDog
What if I'm using a PC for watching DRM content?
Should content producers demand Microsoft?
I'm thinking yes, absolutely.
Collin Kreklow
@gigantos @morten_skaaning @GossiTheDog
What about the data you view of other people (thinking in the context of BYOD or just businesses that don’t disable it)
gigantos
@morten_skaaning @GossiTheDog probably not, as the data is stored in your machine and never sent to the cloud. There are many issues, but I suspect GDPR is not one of them.
Morten Hilker-Skaaning
@GossiTheDog isn't Recall a super GDPR violation?
Dr Lee A. Christie
@GossiTheDog Didn’t they change it to opt-in now? So now the hacker has to preemptively enable it before they can start collecting data?
Rich Felker
@GossiTheDog @cjkreklow @gigantos @morten_skaaning If Microsoft installed the malware on your computer without your consent, it seems they are now the data processor.
Kevin Beaumont
Microsoft’s President Brad Smith appears before US House Committee on Homeland Security tomorrow.
His testimony: https://homeland.house.gov/wp-content/uploads/2024/06/2024-06-13-HRG-Testimony-Smith.pdf
In this bit he talks about Recall (not named), where he pats himself and Microsoft on the back for “a feature change” and job well done.
Given it has been a complete cybersecurity and privacy car crash - and as of tomorrow the changes (plural) they’re referring to haven’t even been implemented - it seems like Microsoft fails to grasp customer needs: safety.
Kevin Beaumont
One other thing - Microsoft's written testimony to the US House says, quoting, bolded by MS:
"Before I say anything else, I think it’s especially important for me to say that Microsoft accepts responsibility for each and every one of the issues cited in the CSRB’s report. Without equivocation or
hesitation. And without any sense of defensiveness."
Counterpoint: they publicly disputed the report in the media. https://www.theverge.com/2024/4/25/24139914/microsoft-cyber-security-incidents-trust-report
Kevin Beaumont
I should say that if Brad is asked about Recall tomorrow, the answers may raise some.. uh... eyebrows here.
I don't know what MS SLT have been told, but expect fun when the feature drops on consumer laptops in a few days.
As I mentioned in my blog, there is some more security hardening there on Copilot+ PCs (this was before MS put out their blog)... but it's still easily bypassable.
Kevin Beaumont
Nessus now detects Recall as a vulnerability.
Kevin Beaumont
Microsoft’s Recall puts the Biden administration’s cyber credibility on the line
https://cyberscoop.com/microsoft-recall-secure-by-design/
Interesting article. All through this, CISA and the DHS have declined to comment.
Kevin Beaumont
The Verge reports today that "Windows engineers are scrambling to get additional changes tested and ready for the release of Copilot+ PCs next week."
It also says "Recall was developed in secret at Microsoft, and it wasn’t even tested publicly with Windows Insiders."
I've also been told Microsoft security and privacy staff weren't provided Recall, as the feature wasn't made available broadly internally either.
https://www.theverge.com/2024/6/13/24177703/microsoft-xbox-game-showcase-windows-recall
Kevin Beaumont
Microsoft President Brad Smith just testified to the US House that Recall is a good example of Secure By Design, and that they have the time to get it right (it’s supposed to launch in 3 working days).
Kaveinthran
@GossiTheDog Do you have your blog link handy?
GNU social JP is a social network, courtesy of GNU social JP管理人. It runs on GNU social, version 2.0.2-dev, available under the GNU Affero General Public License.
All GNU social JP content and data are available under the Creative Commons Attribution 3.0 license.