Conversation

Notices

1. Embed this notice
   Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:53:11 JST Taggart :donor:

   Good morning, nerds! The British Library just dropped its after-incident report on the ransomware attack that has disabled the Library for, uh, months?
   
   Let's dig in.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:52:53 JST Taggart :donor:

     in reply to

     To Recap
     
     The British Library:
     
     - used unsupported, unpatched software as critical infra
     - used multiple IT vendors with varying levels of access
     - lacked sufficient in-house staff to coordinate a proper security policy
     - lacked resources (or leadership, probably) to appropriately fund an infrastructure refresh program
     - launched remote access during COVID WITHOUT MFA
     
     And although we cannot say for sure that the Terminal Server was the point of access, it's a good dang bet. Rhysida works smarter, not harder.
     
     As usual, the reality of defense is not sexy malware research. It's not breathlessly shouting about patching 0-days. It's the quotidian work of getting the basics right, not taking shortcuts, and making security—across the CIA triad—a budget priority.
     
     In cultural and educational institutions, it is very common to think of IT systems as an afterthought, or ancillary to the primary mission. These institutions maintain this mentality at their own considerable risk. This extends to the governments and organizations who fund these institutions.

     Tokyo Outsider (337ppm) repeated this.
   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:52:54 JST Taggart :donor:

     in reply to

     Actually yeah, the rest of this "Lessons Learned," does not illuminate in any useful way. This is not at all the same kind of detailed report we got from the Irish NHS Conti attack. We may never get such a report, given the obviously poor state of their detective and logging capabilities.

     Tokyo Outsider (337ppm) repeated this.
   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:52:56 JST Taggart :donor:

     in reply to

     Skipping over most of the Impact section since I'm not qualified to speak to the BL's mission.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:52:58 JST Taggart :donor:

     in reply to

     Moving on to impact...A few key software systems, including the library management system, cannot be brought back in the form that they existed in before the attack, either because they are no longer supported by the vendor and the software is no longer available, or because they will not function on the Library’s new secure infrastructure which is in the process of being rolled out.So the library's core software tool, the thing that manages collections and borrowing status, is just permaborked.
     
     Cool cool cool. See again about technical debt always coming due.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:52:59 JST Taggart :donor:

     in reply to

     Secondly, a keyword attack scanned our network for any file or folder that used certain sensitive keywords in its naming convention, such as ‘passport’ or ‘confidential’, and copied files not just from our corporate networks but also from drives used by staff for personal purposes as permitted under the Library’s Acceptable Use of IT Policy. This policy, and the staff education that accompanies it, will be reviewed in the light of lessons learned from the cyber-attack. The files and folders copied in this way represent around 40% of the copied documents. Oh really? It's going to be reviewed?
     
     Tell you a secret about non-profits, schools, and cultural institutions. Their PII policies are ridiculously lax. And also? Most PCI compliance...isn't.
     
     These places are rife with sensitive data (like, say, donor information) that is incredibly valuable to attackers.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:53:00 JST Taggart :donor:

     in reply to

     In common with other on-premise servers, this terminal server was protected by firewalls and virus software, but access was not subject to Multi-Factor Authentication (MFA). MFA was introduced across the Library in 2020 to increase protection of all remote activities relating to cloud applications such as email, Teams and Word, but for reasons of practicality, cost and impact on ongoing Library programmes, it was decided at this time that connectivity to the British Library domain (including machine log-on access and access to on-premise servers) would be out of scope for MFA implementation, pending further renewal of the Library’s infrastructure.Monkey's paw closes
     
     Spend the money. MFA your external access. Please.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:53:01 JST Taggart :donor:

     in reply to

     The Library utilises numerous trusted partners for software development, IT maintenance, and other forms of consultancy, and whose staff have a variety of levels of access to our network or infrastructure dependent on their contract with us and the level of supervision or vetting that is undertaken.The outsourcing of IT work is always a risk, but at cultural/education institutions, it's even worse because they frequently have nobody who knows what good and bad look like. They're wholly reliant on the vendors. And so, so many MSPs have no idea how to deploy assets securely.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:53:03 JST Taggart :donor:

     in reply to

     However, the first detected unauthorised access to our network was identified at the Terminal Services server. This terminal server had been installed in February 2020 to facilitate efficient access for trusted external partners and internal IT administrators, as a replacement for the previous remote access system, which had been assessed as being insufficiently secure.Yeah uh, don't have these. And definitely don't have them generally available. Terminal Services, aka "RDP for everyone," are too spicy for most risk appetites.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:53:04 JST Taggart :donor:

     in reply to

     (Long quote, sorry)Forensic analysis of the attack performed by our independent cyber-security advisors has identified evidence of an external presence on the Library network at 23:29 on Wednesday 25 October 2023, with the first evidence of movement around the network at 23:32. Later that night, at 01:15 on 26 October 2023, the Library’s IT Security Manager was alerted to possible malicious activity on the Library network. This alert came from the Library’s Monitoring System which had automatically blocked the suspect activity at 00:21. The IT Security Manager, among other actions, extended the automatic block beyond the pre-set expiry, undertook a vulnerability scan (which came back with no results) and actively monitored activity log. No repeat activity was seen. The incident was escalated to the IT Infrastructure team at 07:00. Further investigation by the IT Infrastructure Team, including detailed analysis of activity logs, did not identify any obviously malicious activity and they subsequently performed a password reset before unblocking the account later that day.Here we go. It's very common that the attack is detected, but ignored or not understood. This is what happened with the Conti attack on the Irish Health Service as well.
     
     Running a "vulnerability scan" is insufficient. If malicious activity is detected, vuln scans are equivalent to telling a gunshot wound patient that you're gonna check to see if they're wearing Kevlar.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:53:05 JST Taggart :donor:

     in reply to

     Because of the overtly destructive nature of the attack, it is unlikely that a definitive answer will ever be gained on the exact timing of Rhysida’s entry into the Library’s estate. However, forensic investigation and analysis of records indicates the strong likelihood that the criminal actors initially gained access at least three days before the incident became apparent.Translation: we weren't logging sufficient data into our SIEM. I know it's expensive, but this is why you spend that money.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:53:06 JST Taggart :donor:

     in reply to

     The intrusion was first identified as a major incident at 07:35 on 28 October 2023 when a member of the Technology Team was unable to access the Library’s network. Initial escalation and investigation of the incident within the Technology Team as per the Technology Major Incident Management Plan confirmed the likelihood that the incident was the result of a cyber-attack; and at 09:15 the Library’s Crisis Management Plan was invoked by the Business Continuity Manager. Great that they had a process in place! Not everywhere has proper Business Continuity procedures. Although, it turns out they were on unsteady footing.
     
     I'll also note that this "initial detection" was when users tried to log in in the morning. Forensics may later find that there were ignored security alerts to indicate an issue, but so far, no detective capacity in sight.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:53:08 JST Taggart :donor:

     in reply to

     Our major software systems cannot be brought back in their pre-attack form, either because they are no longer supported by the vendor or because they will not function on the new secure infrastructure that is currently being rolled outRansomware also tends to be the bill collector for technical debt. If end-of-life software is mission-critical, you are dangerously exposed, because recovery will be next to impossible without specialized help.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:53:10 JST Taggart :donor:

     in reply to

     while we have secure copies of all our digital collections – both born-digital and digitised content, and the metadata that describes it – we have been hampered by the lack of viable infrastructure on which to restore it.Right off the bat we're about to see a difference between what happens in a major enterprise and a cultural institution with limited means. They simply did not have the capacity to recover, given that they did not pay the ransom and systems remained encrypted.
     
     And you might think "Oh just reimage all of them." I cannot stress enough to you how undersized library and museum IT staffs always are.

   • Embed this notice
     Taggart :donor: (mttaggart@infosec.town)'s status on Saturday, 09-Mar-2024 18:53:17 JST Taggart :donor:

     in reply to

     I wanna make it clear that it is not my intention to dunk on the British Library. In almost every case, failures like this are a result of poor leadership and budgetary decisions, not negligence on the part of IT staff. I guarantee you there's some grumbly sysadmin who has been screaming about this stuff for years.