CrowdStrike,
Review of the Blue Screen Of Death on July 19, 2024

G09a Desktop 231010 02 1

A few words of context


On July 19, 2024, the digital world came to a standstill as a massive outage rippled through Microsoft Windows systems running CrowdStrike Falcon’s Endpoint Detection and Response (EDR) software. At precisely 04:09 UTC, an update to this agent triggered a cascade of “Blue Screens of Death” (BSOD) on a staggering 8.5 million machines. In a matter of moments, planes were grounded, hospitals experienced disruptions, media outlets faced communication breakdowns, operators were hampered, and even the stock market felt the impact. Not to mention, half of the Fortune 500 companies that rely on these security products were thrown into chaos. From Canberra to Seattle, the digital landscape was abruptly transformed. Hundreds of organizations, many critical to everyday life, were either slowed down or brought to a complete halt. Some have labeled it “the largest IT outage in history,” and it undeniably forced the biggest return to pen and paper in recent memory.

The power of a tiny file


At the heart of this disruption was a seemingly innocuous 40 KB update from CrowdStrike. This tiny file, which would typically go unnoticed, contained a crucial logical error. Falcon (EDR) attempted to access a memory address that had become invalid, causing an immediate crash. Since this software operates at the kernel level, its malfunction brought entire operating systems to their knees. To complicate matters, Falcon was configured to automatically restart, leading to a relentless crash loop and making repairs even more challenging.

Ironically, while the systems were down, they were at least securely crashed!

A technical glitch with cyberattack-like consequences


Though this outage stemmed from a technical error rather than a malicious attack, the aftermath bore striking similarities to high-profile supply chain attacks, like SolarWinds, MoveIT, or 3CX.

The global paralysis underscored the vulnerability of critical infrastructures to human or technical errors. The shutdown was so abrupt and comprehensive that many initially suspected a large-scale cyberattack, given how closely the effects mirrored those of a coordinated malicious effort. This event also highlighted the broader impacts on production chains and business operations. It serves as a powerful reminder that the choices we make today regarding digital security solutions have far-reaching consequences beyond our own networks.

So, what exactly is an EDR?


CrowdStrike Falcon exemplifies what’s known as an Endpoint Detection and Response (EDR) solution. These security tools continuously monitor endpoints, equipped with agents, to detect suspicious activity. These agents, installed on every compatible endpoint, gather extensive data, from running processes and applications to driver loading, memory and disk usage, registry activity, and even network connections. The systems then automatically respond to block threats and secure the critical data on the endpoint.

Falcon, like other EDR solutions, operates with high-level privileges, allowing for deep integration with the operating system. However, in this case, the same deep integration that’s usually an advantage turned into a liability, as the malfunction paralyzed millions of systems. The update’s consequences were amplified, making repairs a laborious process. Although CrowdStrike and Microsoft quickly issued remediation guidelines and a script to automate the repair, the process was particularly time-consuming for affected companies, requiring manual intervention on every affected machine.

NDR technology: a complementary, agentless alternative


Network Detection and Response (NDR) technology offers a different approach, monitoring overall network traffic to detect threats and provide SOC teams with extensive visibility. Like EDR, NDR identifies suspicious behavior early in the kill chain, whether in physical or virtual environments.

 

Seamless Integration, Maximum Efficiency

Beyond the typical benefits of NDR—such as immediate and comprehensive visibility across all network assets—Gatewatcher NDR provides a passive, agentless security alternative that can significantly bolster system resilience. NDR seamlessly and flexibly integrates with an organization’s existing security ecosystem, including standardized solutions like EDR, XDR, SIEM, and SOAR. This smooth integration maximizes the efficiency of SOC teams without disrupting day-to-day operations. As a passive solution, NDR doesn’t require agents on endpoints and doesn’t interfere with network activity. If needed, data interception is carried out through TAPs (Test Access Points) on an exact copy of the network, ensuring that operations continue uninterrupted.

 

The Bigger Picture: Lessons Learned

The CrowdStrike Falcon outage exposed the fragility of our critical infrastructures. It demonstrated how a simple bug could abruptly halt an organization’s activities, leading to severe, even life-threatening, consequences—especially in healthcare settings. This incident highlights not only our growing dependence on technology but also the complex interconnectivity that binds various systems and services together. A failure in one key component can set off a chain reaction, paralyzing entire service infrastructures.

 

Moreover, this incident emphasizes the importance of a multi-layered security approach. Solutions must not only protect the organization but also be tailored to its specific sector, needs, and strategies. They should be designed to ensure business continuity, whether in the event of an outage or a cyberattack. EDR, with its agents, offers targeted intervention at each endpoint. Meanwhile, NDR provides a broader view, detecting and analyzing threats across the network. Together, they form a formidable defense: EDR acts as a security guard at each critical point, while NDR serves as the surveillance system, ensuring no threat goes unnoticed. It’s the perfect combination to safeguard your IT infrastructure. Check out one of our latest article on the combination of EDR and NDR.