Skip to content

Solving an Exchange Online mailbox restore mistake Office 365 user was restored – removing the ImmutableID value | Part 23#23

In the current article, we provide the step by step instructions, for resolving a typical Exchange Online mailbox restore mistake, in Office 365 Directory synchronization environment.

  • The Mailbox restore mistake is realized by – restoring Soft Deleted Office 365 user account, instead of restoring the original Soft Deleted On-Premise Active Directory user account.
  • The proposed solution – is based on a concept of – “Encourage” the Directory Synchronization SOFT Match process by deleting the Office 365 user ImmutableID value.

In our scenario Exchange, Online mailbox deleted because of an event in which an On-Premise Active Directory user account deleted.

  • The On-Premise Active Directory user “bound” to his Office 365 user account “replica”
  • The Office 365 user account had Exchange Online license and Exchange Online mailbox assigned to him.

The common restores mistake is that – instead of restoring the original On-Premise Active Directory user account that deleted the Administrator performs the following actions:

  • The Soft Deleted Office 365 user account restored, and the Exchange Online mailbox that “attached” to the Office 365 user account.
  • A NEW On-Premise Active Directory user who is seemingly identical details as the deleted user account created and the information is synchronized to the Office 365 Directory.

The offered solution for this restore mistake scenario is a “little trick,” which I describe as “encourage Directory Synchronization Hard Match.”

The trick implemented by removing the ImmutableID value from the Soft Deleted Office 365 user account.

I know that at the current time the description of the problem and the description of the solution, seems a little vague but, have patience, it turns out later.

Office 365 user account restored instead of the On-Premise Active Directory user | The characters of Exchange Online restore mistake

To be able to understand better:

  1. What are the characters of the “Wrong Exchange Online Mailbox Recovery Operation” in which a NEW Active Directory user account is created?
  2. What are the results of this “Wrong Exchange Online Mailbox Recovery Operation?”
  3. What is the offered solution that we will implement in dealing with the “Wrong Exchange Online Mailbox Recovery Operation?”

Let’s use the following scenario:

Organization mail infrastructure

  • An organization uses Office 365 services, and Exchange Online as his mail infrastructure.

Directory infrastructure

  • Directory management is implemented via the On-Premise Active Directory, and Directory synchronization server (Azure AD Connect).
  • The Directory synchronization server is responsible for synchronizing information from the local On-Premise Active Directory to the Office 365 Directory (Azure Active Directory).

The deletion event

  1. On-Premise Active Directory user account named Nicki (which his login name is Nicki@o365info.com) deleted (number 1).
  2. The information about the “On-Premise Active Directory user deletion,” synchronized by the Directory synchronization server (Azure AD Connect) to the Office 365 Directory (Azure Active Directory) (number 2).
  3. The result is, that the Nicki Office 365 user account that “bound” to the Nicki deleted On-Premise Active Directory user account”, also deleted (number 3).
  4. When Nicki Office 365 user account deleted, the Exchange Online license that assigned to Nicki Office 365 user account, removed (deleted) (number 3).
  5. Azure Active Directory synchronizes the information to the Exchange Online infrastructure.
  6. When Exchange Online gets the information about the fact that the Nicki Exchange Online license removed, Exchange Online deletes the Nicki Exchange Online mailbox (number 4).
user deletion - flow of the events- Directory synchronization based environment -01

The restore request

The Administrator, got a request to – recover Nicki Exchange Online deleted mailbox, and enable Nicki to access his restored Exchange Online mailbox.

The “right” process of recovering Exchange Online in Directory synchronization environment

The “right” restore process supposed to start with – recovering Nicki Soft Deleted On-Premise Active Directory user account, and the rest of the Exchange Online mailbox recovery steps were supposed to “roll along” automatically.

You can read more information about the “right procedure” of recovering Exchange Online in Directory synchronization environment in the article – The special characters of Directory synchronization in an Office 365 environment | Article 2#2 | Part 12#23

The main characters of the Exchange Online restore mailbox mistake

The Exchange Online mailbox recovery mistake

Office 365 admin center offers a user-friendly management interface that enables us to perform various tasks.

For example, restoring Soft Deleted Office 365 user account is an easy and simple process, when using the Office 365 admin center vs. the On-Premise Active Directory interface, that can consider as more complicated or less easy to use.

Because it’s easier to perform the “user restores operation” via the Office 365 admin interface, the “plan” of the Administrator, composed of the following parts:

Phase 1#2

  • Restore Nicki Soft Deleted Office 365 user account. The restore process should also restore the Exchange Online license that assigned to Nicki Soft Deleted user account. When Nicki Exchange Online license is restored, the Exchange Online mailbox will also be restored.
Problematic Exchange mailbox restore - The Office 365 user account was restored -02

Phase 2#2

  • Creating a NEW On-Premise Active Directory user account, with seemingly identical details as the “deleted Nicki’s user account” (the same login name and the same E-mail address). In our example, the login name that configured for the NEW Active Directory user account is – Nicki@o365info.com.
  • Activating the Directory synchronization process, and synchronize the information about the “recovered” Nicki On-Premise Active Directory user account to – Office 365 Directory (Azure Active Directory).
Problematic Exchange mailbox restore - The Office 365 user account was restored -03

The Administrator underlying assumption was, that when the Directory synchronization process will run, the mechanism of Soft Match will be automatically executed.

Th expected result was that the Directory Synchronization Soft Match mechanism, will “bind together”, the NEW Nicki On-Premise Active Directory user account, with the Azure Active Directory – Soft Deleted Nicki’s user account, because they have the same user login name and the same E-mail address.

You can read more information about the Directory Synchronization Soft Match mechanism in the article – The special characters of Directory synchronization in an Office 365 environment | Article 1#2 | Part 11#23

The result

Phase 1#2

  1. Nicki Office 365 user account successfully restored + Nicki Exchange Online license.
  2. Nicki Exchange Online mailbox successfully restored.
  3. Nicki Office 365 user account status changed to “in cloud” instead of the previous status – synced with Active Directory. In other words, Nicki user account is not considered anymore as “synced user account”.

Phase 2#2

When the Administrator activates the Directory synchronization process, a strange phenomenon occurs:

  1. Nicki Office 365 user account is deleted along with Nicki Exchange Online license.
  2. Nicki Exchange Online mailbox is deleted.

The reason for this strange phenomenon is a “collision” between Nicki On-Premise Active Directory NEW user account and Nicki Office 365 user account that restored because booth of the user accounts has the same login name and the same E-mail address (Nicki@o365info.com).

The “responsible” element for this “strange behavior” is the Directory synchronization server (Azure AD Connect).

For me, it looks like a “problem” in the way that the Directory synchronization server (Azure AD Connect) interpreted this “event.”

Instead of reporting about a failed synchronization process, the Directory synchronization server (Azure AD Connect) “instruct” Azure Active Directory to delete the Office 365 user account.

This “problem” will probably be fixed in the future so, the Directory synchronization server (Azure AD connects) will just report about a collision between two users accounts instead of deleting the Office 365 user account.

The outcome

Nicki cannot access her Exchange Online mailbox. Even if we restore again Nicki Soft Deleted Office 365 user account, every three hours, Nicki Office 365 user account and her Exchange Online mailbox will be deleted.

Description of the offered solution – Encourage a Soft Match process by removing the ImmutableID value of the Office 365 user account

The proposed solution

Delete the ImmutableID attribute of the restored Office 365 account.

In the current section, we are going to implement a solution which I describe as – “Encourage the Directory Synchronization Soft Match process.”

I use the term “Encourage” because, in our scenario, the Soft Match between the NEW Active Directory user account, and the restored Office 365 user account will not occur by default!

To be able to “trigger” the Directory Synchronization Soft Match process, we will use the following trick:

Step 1#2 – Removing the ImmutableID (sourceAnchor attribute) value of the Office 365 user account.

We will access the properties of the restored Office 365 user account (by using PowerShell), and delete the value of the ImmutableID.

Encourage the Directory Synchronization SOFT Match process - Step 1-2

Step 2#2 – Activate the Directory Synchronization

In this step, all we need to do is – just activate the Directory synchronization process.

After a short period, we will see that the status of Nicki Office 365 user account is updated from “In cloud” into a status of – synced with Active Directory.

In case that you are wondering how this “magic” happened, the answer is as follows:

When the Directory synchronization server synchronizes the information about the NEW Active Directory user whom his login name is – Nicki@o365info.com, the Directory Synchronization “notice,” that there is Office 365 user who also uses an identical login name.

The Directory synchronization process, “understand” that the two user accounts are related to each other, and probably need to be connected.

The Directory synchronization will activate the Soft Match process.

The process of “Soft Match,” will lead to the result of “Hard Match.”

The term “Hard Match” describes a process in which the GUID value of the NEW Active Directory user account, will be converted and saved as the ImmutableID value of the Nicki Office 365 account.

In case that you wonder why we need to implement all this complex step, the answer is that before we remove the ImmutableID value of Nicki Soft Deleted Office 365 user, the Directory synchronization can not activate the process of Soft Match because the Directory synchronization cannot “decide by himself” to remove an existing Office 365 user ImmutableID value.

Encourage the Directory Synchronization SOFT Match process - Step 2-2

You can read more information about the Directory Synchronization Soft Match mechanism in the article – The special characters of Directory synchronization in an Office 365 environment | Article 1#2 | Part 11#23

The Challenge

The Nicki Office 365 user account that we want to address, and remove his ImmutableID value considers as Soft Deleted.

At the current time, PowerShell does not provide the option of removing the ImmutableID value of a Soft Deleted Office 365 user account (the PowerShell command will display an error).

For this reason, we will need to restore again the Nicki Soft Deleted Office 365 user account, and only then, use the required PowerShell command that will remove the ImmutableID value.

Use a PowerShell command for removing the ImmutableID value

Implementing the solution of – Encourage a Soft Match process by removing the ImmutableID value of the Office 365 user account.

In the following sections, we will demonstrate a scenario that includes the following parts:

  • Step 1#3 – Simulating the event in which On-Premise Active Directory user account is deleted.
  • Step 2#3 – Simulating the Exchange Online mailbox restore mistake.
  • Step 3#3 – Fixing the Exchange Online mailbox restore mistake – remove the ImmutableID value of the Office 365 user account and activate the Directory synchronization process.

Step 1#3 – Simulating the event in which On-Premise Active Directory user account is deleted.

In the following screenshot, we can see that Nicki Office 365 user account that considers as “synchronized Office 365 user account.” We can see that the Sync Type status of Nicki is – synced with Active Directory.

View the properties of Office 365 synced user before the deletion -01

Nicki On-Premise Active Directory user account deleted, and the information synchronized by the Directory synchronization server (Azure AD Connect) to the Office 365 Directory (Azure Active Directory).

Simulating event of deleting Active Directory user account -02

The result is, that the Nicki Office 365 user account that “bound” to the Nicki deleted On-Premise Active Directory user account,” also deleted.

When Nicki Office 365 user account deleted, the Exchange Online license that assigned to Nicki Office 365 user account, removed (deleted).

Azure Active Directory synchronizes the information to the Exchange Online infrastructure.

Nicki Office 365 synchronized user account was deleted -03

When Exchange Online gets the information about the fact that the Nicki Exchange Online license removed, Exchange Online deletes the Nicki Exchange Online mailbox.

Nicki Office 365 synchronized Exchange Online mailbox was deleted -04

Step 2#3 – Simulating the Exchange Online mailbox restore mistake.

In this section, we will simulate the “Exchange Online mailbox restores mistake” that implemented by the Administrator.

Exchange Online mailbox restores mistake phase 1#2

Instead of using the “right restore process,” in which Nicki Soft Deleted On-Premise Active Directory was supposed to be restored, the Administrator access Office 365 admin center, and restore Nicki Office 365 Soft Deleted user account.

Restoring Nicki Office 365 synchronized user account -01

The restoring process is successfully completed, but we should note, a very important piece of information – the Sync Type status of Nicki changed to “In cloud”.

In simple words, the “binding” between Nicki Office 365 user account and Nicki Active Directory user account (that deleted) is “cut off”.

Another important thing that we cannot see in the Office 365 admin center graphic interface is, that the ImmutableID value of Nicki was not deleted. Later, we will see how this issue affects us.

Nicki Office 365 synchronized user account was successfully restored -02

Exchange Online mailbox restores mistake phase 2#2

In the next phase, the administrator performs the following actions:

  • Creating a NEW On-Premise Active Directory user account, with seemingly identical details as the “deleted Nicki’s user account” meaning the same login name and the same E-mail address. In our specific scenario, the NEW Active Directory login name is Nicki@o365info.com.
Creating NEW On-Premise Active Directory user account for Nicki -01
  • The Administrator activates the Directory synchronization process and synchronize the information about the “recovered” Nicki On-Premise Active Directory user account to – Office 365 Directory (Azure Active Directory).

When looking at the Directory synchronization log, we can see information about a “problem” in the synchronization process.

The issue appears as “completed export errors.”

Synchronizing information NEW On-Premise Active Directory user account -02

The Directory synchronization reports about an error that defined as “AttributeValueMustBeUnique”.

The “action” that the Directory synchronization process performs is a deletion of the Office 365 user account.

On the left pane, we can see that the actions defined as – “Deletes.”

I’m not sure that I agree with this logic.
I assume that the Directory synchronization “logic” is that if he locates an “orphan Office 365 user account,” that have ImmutableID value, but he is not connected to existing On-Premise Active Directory user account, the “Office 365 object” needs to be deleted.

Synchronizing information NEW On-Premise Active Directory user account -03

When we try to get more information about the synchronization error, we can see the following error:

Unable to update this object because the following attributes associated with this object have values that may already associate with another object in your local directory services: [UserPrincipalName Nicki@o365info.com;]. Correct or remove the duplicate values in your local directory. Please refer to http://support.microsoft.com/kb/2647098 for more information on identifying objects with duplicate attribute values.
Tracking Id: bde73132-d2e5-4237-bf09-a0a2a89c0d34

When we use the Office 365 admin center to verify the status of Nicki Office 365 user account, we can see that Nicki’s user account deleted!

Nicki Office 365 synchronized user account was deleted again -04

Step 3#3 – Fixing the Exchange Online mailbox restore mistake – remove the ImmutableID value of the Office 365 user account and activate the Directory synchronization process.

In the section, we will “fix” the Exchange Online mailbox recovery mistake by implementing the following procedure:

To enable the required “binding” between Nicki Active Directory user account and Nicki Office 365 user account, we will “encourage” the Directory Synchronization Soft Match process by removing the ImmutableID value of Nicki Office 365 user account.

In the first step, we would like to verify what is the status of Nicki ImmutableID value.

We will use the following PowerShell command:

Get-MsolUser -ReturnDeletedUsers -UserPrincipalName Nicki@o365info.com | Select ImmutableID

Notice that we use the parameter “ReturnDeletedUsers” because Nicki Office 365 user account is Soft Deleted. In other words – located in the Azure Active Directory recycle bin.

To be able to address Soft Deleted object, we will need to use the parameter – “ReturnDeletedUsers”.

Get-MsolUser -ReturnDeletedUsers -UserPrincipalName -01

In the next step, we will try to remove Nicki ImmutableID value.

We will use the following PowerShell command:

Set-MSOLUser -ReturnDeletedUsers -UserPrincipalName Nicki@o365info.com -ImmutableID ""
Set-MSOLUser -ReturnDeletedUsers -UserPrincipalName -02

When we execute the PowerShell command, the following error appears:

Set-MsolUser : A parameter cannot be found that matches parameter name ‘ReturnDeletedUsers’.
At line:1 char:14
+ Set-MSOLUser -ReturnDeletedUsers -UserPrincipalName Nicki@o365info.co …
+ ~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Set-MsolUser], ParameterBindingException
+ FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.Online.Administration.Automation.SetUserS

The meaning of this error is that is, that we cannot update the ImmutableID value of a Soft Deleted Office 365 user account.

To be able to continue with the process, we will need to restore again Nicki Soft Deleted Office 365 user account.

Set-MSOLUser -ReturnDeletedUsers -UserPrincipalName -03

After we restore Nicki Soft Deleted Office 365 user account, we will run the following PowerShell command:

Set-MSOLUser -UserPrincipalName Nicki@o365info.com -ImmutableID ""

To verify the status of Nicki ImmutableID value, we will use the following PowerShell command:

In the following screenshot, we can see that Nicki ImmutableID value is empty.

This is the required result!

Set-MSOLUser -ReturnDeletedUsers -UserPrincipalName -04

Directory synchronization infrastructure

In this step, we will initialize the Directory synchronization process.

Theoretically, we can initialize the standard “delta” Directory synchronization process, which will synchronize only the updates.

From my experience, initializing, the standard “delta” Directory synchronization process doesn’t complete the process successfully all the time.

For this reason, I recommend using the “Full synchronization” option.

We can start “Full synchronization” option by using the following PowerShell command-

Start-ADSyncSyncCycle Initial

In the following screenshot, we can see, that Directory synchronization processes complete successfully.

Directory synchronization defines the process as “Adds”.

Synchronizing information NEW Active Directory user removed ImmutableID -01

When looking at the event properties, we can see that the information about Nicki On-Premise Active Directory user account “added” to Office 365 Directory (Azure Active Directory).

Synchronizing information NEW Active Directory user removed ImmutableID -02

In the following screenshot, we can see that Nicki Office 365 user account that considers as “synchronized Office 365 user account.” We can see that the Sync Type status of Nicki is – synced with Active Directory.

Nicki Office 365 user account status is synced with Active Directory -03
o365info Team

o365info Team

This article was written by our team of experienced IT architects, consultants, and engineers.

This Post Has 0 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *