ICYMI: Committee Examines CrowdStrike Processes in First Congressional Hearing on the Disastrous July Global IT Outage
September 26, 2024
WASHINGTON, D.C. — This week, the House Homeland Security Subcommittee on Cybersecurity and Infrastructure Protection, led by Committee Chairman Mark E. Green, MD (R-TN) and Subcommittee Chairman Andrew Garbarino (R-NY), held a hearing to examine CrowdStrike’s defective software update that caused a major information technology (IT) outage on July 19, 2024. In the hearing, Members received witness testimony from CrowdStrike’s Senior Vice President of Counter Adversary Operations Adam Meyers. The Committee initially requested testimony from CEO George Kurtz on July 22, but was told by the company that Mr. Meyers was the appropriate witness.
Members examined how the defective update caused an outage across industry sectors, as well as how CrowdStrike has since adjusted its processes for pre-deployment testing and the rollout of updates. Members also expressed concerns about the company’s security culture, the impact of the outage on government networks, such as the Cybersecurity and Infrastructure Security Agency (CISA), and how the cross-sector impacts of the outage could serve as a dangerous inspiration for America’s cyber adversaries.
In Chairman Green’s opening statement, he highlighted the potential homeland security implications of the IT outage:
“As the July 19th outage has demonstrated yet again, our networks are increasingly interconnected. While we know that nation-state actors and criminals try to exploit our networks, we would not expect companies to defend themselves from these targeted attacks. However, as I emphasized with the President of Microsoft in June, we do expect companies to implement the strongest cybersecurity practices possible. Our nation’s security depends on a strong public-private partnership for protecting our networks. … In August, CISA Director Jen Easterly described this incident as, ‘a useful exercise — a dress rehearsal for what China may want to do to us.’ We look forward to working with you to make sure we never make it to opening night.”
In his line of questioning, Chairman Green asked Meyers how the decision was made to launch the update:
“Who made the decision to launch the update? Did AI do that or did an individual do that––and can you tell me how that decision was made?”
Meyers answered:
“AI was not responsible for making any decision in that process. It is part of a standard process. We release 10 to 12 of these updates, content updates, every single day. So, that was part of our standard operating procedure.”
Chairman Green continued:
“These updates are automatic globally?”
Meyers answered:
“The updates were distributed to all customers in one session. We’ve since revised that. In the full testimony, I’ve included a graphic that depicts what that now looks like and that is no longer the case.”
Subcommittee Chairman Garbarino questioned Meyers on why government agencies were also affected:
“There was reporting about CrowdStrike’s faulty software update—it’s largely focused on commercial operations, like emergency services, flights, but there was also a big impact on federal agencies such as the FCC, Social Security, CBP, and even CISA. Although networks are becoming increasingly interconnected, government networks should be isolated from commercial ones. Why were federal agencies impacted by this outage? Are there different updates to test for commercial versus government business when you deal with your clients, or is it all the same?”
Meyers replied:
“The updates went to Microsoft Windows operating system sensors that CrowdStrike had deployed. So that would have impacted any system that was running Microsoft operating system with that particular version of CrowdStrike Falcon that was online during the time period that the channel file was distributed.”
Chairman Garbarino continued:
“So, as long as Microsoft was on that computer, using that system––whether it was government or commercial––it didn’t matter. It was affected.”
Meyers replied:
“As long as the CrowdStrike sensor is running on the Microsoft operating systems––on those systems at that time––yes.”
Representative Mike Ezell (R-NY) asked Meyers about concerning reports on how CrowdStrike handles staffing decisions and support:
“A recent article stated that, ‘engineers and threat hunters were given just two months for work that would have normally taken a year.’ Additionally, the article noted that CrowdStrike confirmed its use of ‘existing engineers instead of hiring a new team of cloud threat hunters.’ Pearl River Community College and many others in my district offer an excellent cybersecurity technology program for our next generation of students to help fill this unsettling skills gap. Do you make these staffing decisions because of a lack of adequate job force in the industry?”
Meyers answered:
“We have a robust internship program. We bring some of the most talented from these internal and external internship programs, and recruit from all over the country and all over the world in order to fill positions.”
Rep. Ezell continued:
“What steps do you take to better support your staff and ensure that they have the right tools and skills to succeed?”
Meyers answered:
“We have extensive internal training programs. We also send our team to various trainings across the globe, different industry trainings at conferences, and other programs where they can learn new skills and continue to develop their existing skills. We also have some of our own researchers and analysts conduct trainings at those same events to help train individuals that are not yet in the workforce, or working at other companies, in order to learn some of the critical skills that are needed to identify and to track advanced threat actors.”
Representative Laurel Lee (R-FL) questioned Meyers on the risks and benefits of CrowdStrike’s cybersecurity software running at the core of the operating system––the kernel––rather than the user space:
“CrowdStrike has this really extraordinary access into the kernel of the operating system, and you all were talking a bit about the risk versus efficiency of having this kind of access and making updates within the kernel. Share with me your thoughts on whether this incident could have been averted, or future incidents could be averted, by using the user space for this kind of update.”
Meyers replied:
“Thank you for the question. The kernel, as I said, provides the visibility, the enforcement mechanism, the telemetry and visibility, as well as the anti-tamper. So, I would suggest that while things can be conducted in user mode from a security perspective, kernel visibility is certainly critical to ensuring that a threat actor does not insert themselves into the kernel themselves and disable or remove the security products and features.”
Rep. Lee continued:
“So, is it your assessment then that it’s not possible, really in realistic terms, to do it outside of the kernel?”
Meyers replied:
“With the current kernel architecture, this is the most effective way to get the visibility and to prevent an adversary from tampering with security tools.”
Rep. Lee pressed:
“So, it’s ‘the most effective way,’ but it’s not the only way possible?”
Meyers replied:
“It is certainly the industry standard to use the kernel for visibility, enforcement, and anti-tamper––to ensure that you can stop a threat.”
Rep. Lee continued:
“You’ve testified thus far that you’ve made modifications to the phased rollout approach and also the pre-deployment testing. What other modifications has CrowdStrike made or changes to your internal practices to avert future-similar incidents?”
Meyers answered:
“That is the primary change that we’ve made.”
###