Global Storing Affecting the World: CrowdStrike Update
CrowdStrike is actively working with customers impacted by a defect found in a single content update for Windows hosts. Mac and Linux hosts are not impacted. This was not a cyberattack.
The issue has been identified, isolated, and a fix has been deployed. We are referring customers to the support portal for the latest updates and will continue to provide complete and continuous public updates on our blog.
We further recommend organizations ensure they’re communicating with CrowdStrike representatives through official channels.
Our team is fully mobilized to ensure the security and stability of CrowdStrike customers. We understand the gravity of the situation and are deeply sorry for the inconvenience and disruption. We are working with all impacted customers to ensure that systems are back up and they can deliver the services their customers are counting on.
We assure our customers that CrowdStrike is operating normally and this issue does not affect our Falcon platform systems. If your systems are operating normally, there is no impact to their protection if the Falcon sensor is installed.
Below is the latest CrowdStrike Tech Alert with more information about the issue and workaround steps organizations can take. We will continue to provide updates to our community and the industry as they become available.
Summary
CrowdStrike is aware of reports of crashes on Windows hosts related to the Falcon sensor. View Tech Alert (pdf) or log in to view in the support portal.
Details
Symptoms include hosts experiencing a bugcheck/blue screen error related to the Falcon sensor.
- Windows hosts which have not been impacted do not require any action as the problematic channel file has been reverted.
- Windows hosts which are brought online after 0527 UTC will also not be impacted.
- This issue is not impacting Mac- or Linux-based hosts.
- Channel file "C-00000291*.sys" with timestamp of 0527 UTC or later is the reverted (good) version.
- Channel file "C-00000291*.sys" with timestamp of 0409 UTC is the problematic version.
Note: It is normal for multiple "C-00000291*.sys" files to be present in the CrowdStrike directory - as long as one of the files in the folder has a timestamp of 0527 UTC or later, that will be the active content.
Current Action
CrowdStrike Engineering has identified a content deployment related to this issue and reverted those changes.
If hosts are still crashing and unable to stay online to receive the Channel File Changes, the workaround steps below can be used:
Workaround Steps for Individual Hosts
- Reboot the host to give it an opportunity to download the reverted channel file. We strongly recommend putting the host on a wired network (as opposed to WiFi) prior to rebooting as the host will acquire internet connectivity considerably faster via ethernet.
- If the host crashes again, then:
- Boot Windows into Safe Mode or the Windows Recovery Environment
- Navigate to the %WINDIR%\System32\drivers\CrowdStrike directory
- Locate the file matching “C-00000291*.sys” and delete it.
- Cold Boot the host (Shutdown the host and start from the off state).
Workaround Steps for Public Cloud or Similar Environment
Option 1:
- Detach the operating system disk volume from the impacted virtual server.
- Create a snapshot or backup of the disk volume before proceeding further as a precaution against unintended changes.
- Attach/mount the volume to a new virtual server.
- Navigate to the %WINDIR%\System32\drivers\CrowdStrike directory.
- Locate the file matching “C-00000291*.sys” and delete it.
- Detach the volume from the new virtual server.
- Reattach the fixed volume to the impacted virtual server.
Option 2:
- Roll back to a snapshot before 0409 UTC.
Additional Resources
- Falcon Sensor Content Issue Likely Used to Target CrowdStrike Customers
- CrowdStrike and Rubrik Customer Content Update Recovery For Windows Hosts
- Standing strong together: Cohesity’s support for CrowdStrike’s Falcon Sensor updates
