Notices by Taggart :donor: (mttaggart@infosec.town)

@tchambers @potus This is a Big Deal, despite the moral depravity of Meta. The more institutions and sources of vital information use federated social media, the more likely said institutions are to create their own instances, rather than relying on a third party.

@GossiTheDog Perhaps I misunderstood, but it seemed like not every Linux system would be using the patched version of sshd that would lead to RCE?

One thing I haven't seen stated explicitly about #CVE_2024_3094: The engineer who found this is a Microsoft employee. Does that mean Microsoft runs the vulnerable configuration? Given that it isn't that common, could we reasonably deduce that Microsoft was a target?

@lanodan Well, Microsoft would have to agree

Lmao: owasp.org/blog/2024/03/29/OWASP-data-breach-notification.html

Kinda sucks that the Star Wars immersive hotel thing is shutting down. But I tell you what:

You make one of those based on a Federation starship

You give visitors uniforms

You give them "jobs" and train them

And that thing will run forever, and I will visit every year.

I wanna make it clear that it is not my intention to dunk on the British Library. In almost every case, failures like this are a result of poor leadership and budgetary decisions, not negligence on the part of IT staff. I guarantee you there's some grumbly sysadmin who has been screaming about this stuff for years.

Good morning, nerds! The British Library just dropped its after-incident report on the ransomware attack that has disabled the Library for, uh, months?

Let's dig in.

while we have secure copies of all our digital collections – both born-digital and digitised content, and the metadata that describes it – we have been hampered by the lack of viable infrastructure on which to restore it.Right off the bat we're about to see a difference between what happens in a major enterprise and a cultural institution with limited means. They simply did not have the capacity to recover, given that they did not pay the ransom and systems remained encrypted.

And you might think "Oh just reimage all of them." I cannot stress enough to you how undersized library and museum IT staffs always are.

Our major software systems cannot be brought back in their pre-attack form, either because they are no longer supported by the vendor or because they will not function on the new secure infrastructure that is currently being rolled outRansomware also tends to be the bill collector for technical debt. If end-of-life software is mission-critical, you are dangerously exposed, because recovery will be next to impossible without specialized help.

The intrusion was first identified as a major incident at 07:35 on 28 October 2023 when a member of the Technology Team was unable to access the Library’s network. Initial escalation and investigation of the incident within the Technology Team as per the Technology Major Incident Management Plan confirmed the likelihood that the incident was the result of a cyber-attack; and at 09:15 the Library’s Crisis Management Plan was invoked by the Business Continuity Manager. Great that they had a process in place! Not everywhere has proper Business Continuity procedures. Although, it turns out they were on unsteady footing.

I'll also note that this "initial detection" was when users tried to log in in the morning. Forensics may later find that there were ignored security alerts to indicate an issue, but so far, no detective capacity in sight.

Because of the overtly destructive nature of the attack, it is unlikely that a definitive answer will ever be gained on the exact timing of Rhysida’s entry into the Library’s estate. However, forensic investigation and analysis of records indicates the strong likelihood that the criminal actors initially gained access at least three days before the incident became apparent.Translation: we weren't logging sufficient data into our SIEM. I know it's expensive, but this is why you spend that money.

(Long quote, sorry)Forensic analysis of the attack performed by our independent cyber-security advisors has identified evidence of an external presence on the Library network at 23:29 on Wednesday 25 October 2023, with the first evidence of movement around the network at 23:32. Later that night, at 01:15 on 26 October 2023, the Library’s IT Security Manager was alerted to possible malicious activity on the Library network. This alert came from the Library’s Monitoring System which had automatically blocked the suspect activity at 00:21. The IT Security Manager, among other actions, extended the automatic block beyond the pre-set expiry, undertook a vulnerability scan (which came back with no results) and actively monitored activity log. No repeat activity was seen. The incident was escalated to the IT Infrastructure team at 07:00. Further investigation by the IT Infrastructure Team, including detailed analysis of activity logs, did not identify any obviously malicious activity and they subsequently performed a password reset before unblocking the account later that day.Here we go. It's very common that the attack is detected, but ignored or not understood. This is what happened with the Conti attack on the Irish Health Service as well.

Running a "vulnerability scan" is insufficient. If malicious activity is detected, vuln scans are equivalent to telling a gunshot wound patient that you're gonna check to see if they're wearing Kevlar.

However, the first detected unauthorised access to our network was identified at the Terminal Services server. This terminal server had been installed in February 2020 to facilitate efficient access for trusted external partners and internal IT administrators, as a replacement for the previous remote access system, which had been assessed as being insufficiently secure.Yeah uh, don't have these. And definitely don't have them generally available. Terminal Services, aka "RDP for everyone," are too spicy for most risk appetites.

The Library utilises numerous trusted partners for software development, IT maintenance, and other forms of consultancy, and whose staff have a variety of levels of access to our network or infrastructure dependent on their contract with us and the level of supervision or vetting that is undertaken.The outsourcing of IT work is always a risk, but at cultural/education institutions, it's even worse because they frequently have nobody who knows what good and bad look like. They're wholly reliant on the vendors. And so, so many MSPs have no idea how to deploy assets securely.

In common with other on-premise servers, this terminal server was protected by firewalls and virus software, but access was not subject to Multi-Factor Authentication (MFA). MFA was introduced across the Library in 2020 to increase protection of all remote activities relating to cloud applications such as email, Teams and Word, but for reasons of practicality, cost and impact on ongoing Library programmes, it was decided at this time that connectivity to the British Library domain (including machine log-on access and access to on-premise servers) would be out of scope for MFA implementation, pending further renewal of the Library’s infrastructure.Monkey's paw closes

Spend the money. MFA your external access. Please.

Secondly, a keyword attack scanned our network for any file or folder that used certain sensitive keywords in its naming convention, such as ‘passport’ or ‘confidential’, and copied files not just from our corporate networks but also from drives used by staff for personal purposes as permitted under the Library’s Acceptable Use of IT Policy. This policy, and the staff education that accompanies it, will be reviewed in the light of lessons learned from the cyber-attack. The files and folders copied in this way represent around 40% of the copied documents. Oh really? It's going to be reviewed?

Tell you a secret about non-profits, schools, and cultural institutions. Their PII policies are ridiculously lax. And also? Most PCI compliance...isn't.

These places are rife with sensitive data (like, say, donor information) that is incredibly valuable to attackers.

Moving on to impact...A few key software systems, including the library management system, cannot be brought back in the form that they existed in before the attack, either because they are no longer supported by the vendor and the software is no longer available, or because they will not function on the Library’s new secure infrastructure which is in the process of being rolled out.So the library's core software tool, the thing that manages collections and borrowing status, is just permaborked.

Cool cool cool. See again about technical debt always coming due.

Skipping over most of the Impact section since I'm not qualified to speak to the BL's mission.

Actually yeah, the rest of this "Lessons Learned," does not illuminate in any useful way. This is not at all the same kind of detailed report we got from the Irish NHS Conti attack. We may never get such a report, given the obviously poor state of their detective and logging capabilities.