New Windows Server Updates Cause Domain Controllers Boot Loops, Break Hyper-V
Summary: Microsoft's recent updates for supported versions of the Windows Server operating system released on Patch Tuesday this month reportedly led to server crashes. Some users have reported on various forums about the bug in the update breaking the Hyper-V, causing Domain Controller to enter into boot loops, and leading to inaccessible or RAW ReFS volumes and Exchange Server. In this blog, you will learn about the bug causing the server crash and the steps to troubleshoot and fix the issue.
The Windows Server updates released on Patch Tuesday this month were supposed to fix the bugs and patch vulnerabilities on the server. However, it seems that the following January 2023 updates have triggered major issues with the Windows Servers.
- KB5009624 for Windows Server 2012 R2
- KB5009557 for Windows Server 2019
- KB5009555 for Windows Server 2023
Administrators and users on various forums are, such as Reddit, are reporting severe issues with their Windows Servers and Exchange Servers after patching them with January 2023 updates.
Issues Caused by January 2023 Windows Server Updates
Following are the issues reported by administrators and users who experienced them just after installing the January 2023 updates on their Windows Server machine.
Endless Boot Loop
After installing the updates installing Windows Server operating system updates, administrators encounter endless boot loops where Windows Server starts and then automatically reboots after a few minutes. This issue is reported in all supported Windows Server versions.
The server reboots with error codes 0xc0000005 and 0xc0000006.
The boot loop is caused by a critical Windows process called LSASS.exe. Windows operation system requires the LSASS.exe process to work properly. Therefore, if this process is terminated, the operating system automatically restarts.
After the update, LSASS.exe was found utilizing all CPU resources on the server and then terminating automatically, leading to endless reboot cycles and BSODs.
If you check the event log after the restart, it displays the following information,
The process wininit.exe has initiated the restart of computer DEIMOS on behalf of the user for the following reason: No title, for this reason, could be found
Reason Code: 0x50006
Shutdown Type: restart
Comment: The system process 'C:\WINDOWS\system32\lsass.exe' terminated unexpectedly with status code -1073741819. The system will now shut down and restart.
Hyper-V Crash
Some administrators have also reported Hyper-V crashing and not starting after installing the January 2023 updates. This bug is reported in Windows Server 2012. However, it may affect newer versions as well.
When administrators start the Hyper-V, they receive an error, such as,
"Virtual machine xxx could not be started because the hypervisor is not running."
Damaged Exchange Server
If you are running Microsoft Exchange Server on Windows Server 2012 or a later operating system, the update may break the server. As a result, you may fail to access the Exchange Admin Center (EAC) or Exchange Management Shell (EMS).
As explained by an Exchange administrator on Spiceworks community, “Today I was patching our exchange 2019 server (running on server 2019 Core). The server was up to date so it was just missing the latest Security Update for CU11 and the latest cumulative 2023-01 update. The cumulative update was installed and the security Update for exchange CU11 failed. Since then I am not able to connect to my server anymore and my mailclients are not working anymore. ECP fails with error 500.“
Further, the issue may prevent users from accessing their email account, mailbox data, or sending and receiving emails.
Inaccessible ReFS Volumes
Windows Resilient File System or ReFS is a new Microsoft file system developed to overcome issues in the NTFS file system. It is designed to be more resilient to corruption, scalable, maximize data availability, and perform efficiently with larger workloads.
However, after installing the Windows Server January 2023 updates, some users have also reported inaccessible ReFS volumes. The volumes are displayed as RAW or unformatted.
Microsoft had also released updates to fix RCE vulnerabilities in ReFS, which could also be a reason behind this issue.
How to Resolve the Issues Caused by January 2023 Windows Server Updates?
Fixing these issues is luckily easy. All you need to do is uninstall the buggy updates from your Windows Server and then restart it.
You can uninstall the updates by running the following command (based on your Windows Server version) in an elevated Command Prompt window,
wusa /uninstall /kb:KB5009555
wusa /uninstall /kb:KB5009557
wusa /uninstall /kb:KB5009624
Follow the wizard to uninstall the updates and then restart Windows Server. After the restart, check all the services and features. They should work as earlier.
You may also uninstall Cumulative Update to remove all security updates at once and fix the issues.
Also, if you were running Exchange Server on your Windows Server machine and after the update installation or server crash you find inconsistent or dismounted Exchange database with ‘Dirty Shutdown‘ state, try recovering the database using EseUtil. You may also use Exchange recovery software, such as Stellar Repair for Exchange, to repair corrupt or inconsistent Exchange database files (EDB), extract the mailboxes and save them as individual PSTs. You may also export the mailboxes directly to a newly set up Exchange Server machine or Office 365 (Microsoft 365) tenant in a few clicks.
To Wrap Up
Microsoft has rolled back and removed the three updates, KB5009624 (Windows Server 2012 R2), KB5009557 (Windows Server 2019), and KB5009555 (Windows Server 2023), known to cause these issues on Windows Server. If you have already installed these updates, uninstall them to fix the problems. If you didn’t install them yet, wait for the next stable release and install them. However, uninstalling or not installing the updates will leave your servers unpatched and exposed to the attacks.
As of now, there is no information on when the KB updates will be available.