How to Fix – “Content Index State Failed and Suspended” Error
Summary: The content index state is failed and suspended” is an error that appears in the Exchange Server DAG setup that can prevent the DAG from switchover or failover to another server if a disaster strikes or the primary server stops working. This can impact the users as they won't be able to connect or access their mailboxes. In this blog, we have shared some solutions that you may follow to try to fix the error and restore client-server connectivity.
You might encounter an unusual error on the Exchange Server database index saying, “The content index state is failed and suspended.” The error does not impact the accessibility to the databases as the users will still be able to access, read, send, and receive the emails. However, this will impact any user who is trying to connect via Outlook Web Access (OWA) and trying to search in the mailbox.
The functionality impact of the error on the Exchange Server infrastructure is that if you have a Database Availability Group (DAG) setup, it can cause issues when doing a switchover or failover to another server. Although it is not critical to the users, you will not want any issues with your server. If something occurs on the server and it fails to do a failover, it will leave your users without access to their mailboxes.
Reasons for Content Index “failed” and “suspended”
- This issue might occur during a migration from an older version of Exchange Server or due to a problem of storage space.
- Issues might occur post an installation of the latest Cumulative Update (CU) either to a bad update or wrongfully installed an incompatible version.
- The server ended with no disk space where the database is stored and ended up with a failure on the index.
- Antivirus software which is not compatible with the Exchange Server installed or wrongly configured, can lock the Exchange Server files, and stop the indexing from working successfully. It’s important to not have an aggressive configuration as this might disrupt the system.
- Another thing is to ensure that the replication is working fine in a Database Availability Group (DAG) as this would also affect the indexing and of course the data in case of a failover.
If the issue has been reported by the users that they cannot search on their mailboxes, the only way to identify the issue is by checking the mailbox database copy status.
For this, you can use the PowerShell cmdlet Get-MailboxDatabaseCopyStatus in the Exchange Management Shell (EMS).
Get-MailboxDatabaseCopyStatus * | sort name | Select name,status,contentindexstate
This will show the status of the databases and the content index state. If the state shows FailedAndSuspended, then you need to intervene to resolve the problem.
Resolve ‘Content Index State Failed’ on Single Exchange Server
When your Exchange Server is a standalone server and is not part of a Database Availability Group (DAG), you can rebuild the content index by following the below procedure.
Note: Although this is a simple procedure, you might end up with performance degradation on the server as it is very resource-hungry on the CPU utilization. It is strongly suggested to run this process during a maintenance window or at night or on a weekend.
- The first step is to stop the Microsoft Exchange Search and the Microsoft Exchange Search Host Controller services.
- This can be done from the services.msc. For this, right-click on the service and then click on stop. Alternatively, this can be done from PowerShell by using the Stop-Service cmdlet (as given below).
Stop-Service MSExchangeFastSearch
Stop-Service HostControllerService
- The next step is to delete the content index from the path of the database. This can be done by using File Explorer. For this, browse to the folder where the EDB file is stored which corresponds to the database with the problem, and delete the folder with the GUID.
- Make sure that the services mentioned above are stopped, otherwise you will not be allowed to delete the folder. If you are unsure of the file path, you can use the PowerShell cmdlet Get-MailboxDatabase to get the file path of the database.
Get-MailboxDatabase <database name> | select EdbFilePath
- After you have deleted the folder of the content index, start the services as given below.
- You can also use the below PowerShell cmdlet to start the services.
Start-Service MSExchangeFastSearch
Start-Service HostControllerService
- Now, the services will start crawling and indexing all the content of the database. The indexing may take some time to complete, depending on the size and number of items of the database. Once this is done, re-run the Get-MailboxDatabaseCopyStatus PowerShell cmdlet. You will see the database state as healthy.
Fix ‘Content Index State Failed’ in DAG Exchange Setup
- When your database is in a Database Availability Group (DAG), you need to run the re-seeding of the database from a healthy database copy. This can be done by using the Update-MailboxDatabaseCopy PowerShell cmdlet (as given below).
Update-MailboxDatabaseCopy <database name>\<source server name> -CatalogOnly -BeginSeed
- This operation may take time, depending on the connection between the servers and the size of the database. Once the operation is complete, re-run the Get-MailboxDatabaseCopyStatus PowerShell cmdlet. You will see the ContentIndexState as healthy.
If still issue is not fixed then you can follow these steps-
- Make sure you have installed latest update of Exchange server after checking the Microsoft site.
- Restart the Exchange server to ensure that any pending configuration changes, or updates are committed.
- Also make sure that you have enough HDD space on the drive that the database/index is on for a copy/expansion.
- Make sure you’ve configured antivirus exclusions for the database, logs and indexes.
- Also try simplifying your networking by removing the replication network if not needed.
- Create new Exchange Database and move the databases to that new DB, as the index will recreate automatically on the new DB.
You can also check this blog if content index state status is unknown.
To Conclude
While the above-given procedure resolves the issue, this does not mean that it will work every time. If the issue occurs on a regular basis, it indicates corruption in the database. If this is the case, you can use the Exchange server native tools, such as ESEUtil, to run a database recovery process. However, to do so, the database must be offline.
Alternatively, you can use Stellar Repair for Exchange to repair and export all mailboxes to PST and other formats, while being able to browse the whole database, mailboxes, contacts, calendar, tasks, and journal of each mailbox. By using the Exchange recovery software, you can also export all mailboxes directly to a live Exchange database or an Office 365 tenant, matching the mailboxes and other features.
FAQ
When this happens, one can try to reseed the database, but depending on the size, the performance of the server and the connectivity between servers, it will take its time. For more information one can see the link on how to reseed Exchange database
When installing the latest Exchange Cumulative Update (CU),, it’s important to first check the CU level of the current installation and then check the compatibility matrix on the Microsoft site and not just jump update versions as this will cause issues.
There isn’t a right size recommendation on size as it depends on the number of mailboxes and volume of data, but it is recommended to leave ample storage so that if something happens or if a backup is missed, the transaction logs don’t fill up the server and ending up with no disk space. Monitoring the storage is a must.
There are a lot of things that could go wrong during a CU updated. It’s important that before installing a CU, a backup is taken, and the system is thoroughly tested afterwards to ensure that all is working. If something happens, you can revert back, but if no testing is done, you will loose data if you do a restore. Usually, one would go for a new database and move all the data from one database to another if it is still mounted. If the database would give any issues, Stellar Repair for Exchange can assist in the matter of restoring and migration of data.
I tried these instructions for a non-DAG system and it went from “FailedAndSuspended” to “Failed”
Try following commands,
This command displays all database copies with content index in a failed state.
Get-MailboxDatabaseCopyStatus * | where {$_.ContentIndexState -eq “Failed”}
And then, Get-MailboxDatabaseCopyStatus * | where {$_.ContentIndexState -eq “Failed”} | Update-MailboxDatabaseCopy -CatalogOnly
Run the first command again to check there are no failed content indexes.