Skip to content

Analyzing the results of the Exchange spoof E-mail rule |Part 9#12

In the current article, we will review the options that are available for analyzing the result of the Exchange Online Spoof E-mail rule that we have created.
The term “analyze” relate to our ability to answer fundamental questions such as:

  1. How many times the Exchange Online rule that we have created was “triggered”?
  2. What is the scenario in which the Exchange Online rule that we have created was “triggered”?

This information is relevant for us for many different purposes, but for me, there are main three goals:

  1. The be able to identify false-positive scenarios – the meaning is a scene in which a legitimate mail is mistaken identified as Spoof E-mail and deleted or blocks instead of reaching into the organization user mailbox.
  2. Verifying that the Exchange Online Spoof E-mail rule fulfills its purpose, meaning – manage to identify a Spoof E-mail event and manage to execute that required “actions” that configured.
  3. Identify a particular “trend” of Spoof E-mail attack such as a Spoof E-mail attack that is sent from a specific recipient or aimed to a specific organization recipient etc.

The big question is – does Exchange Online include built reporting tools that can provide us this type of information?

The answer is “Yes” and “No”.
“Yes,” Exchange Online includes a built-in reporting tool that enables us to display “usage report” of a particular Exchange Online rule such as – the number of times that the Exchange on-Premises Exchange Online rule was “triggered.”

“No,” at the current time, the information that is provided by the Exchange Online reports doesn’t include information about each of the specific events in which the Exchange Online rule was “triggered.”

For example, by using the Exchange Online built report, we can understand that a particular Exchange Online rule was “trigger” 20 times in the last month, but the report doesn’t tell us for which “source recipient” or “destination recipient” the rule was triggered.

The good news is that there is a workaround solution for these Inabilities of the Exchange Online to provide a detailed report about each separate event in which the Exchange Online rule was triggered.

The solution that we will review in the current article is implemented by using the Exchange Online message trace tool.

Exchange Online message trace tool, can provide us a very detailed information about each incoming mail and for each outgoing mail that was sent to our Office 365 recipients and from our Office 365 recipients.

Exchange Online will keep this information for 90 days.

Also, Exchange Online includes a very useful feature, but at the same time Unknown and an unfamiliar feature that described as historical search.

The Exchange Online message trace feature – historical search, enable us to export all the information to a CSV file (CSV stand for comma separated values).

When we have the information in a CSV file, we can use tools such as Microsoft Excel that have capacities that will help us to analyze the data stored in the CSV file.

The information that Exchange Online export using the CSV can consider as very detailed information and the good news is that each time that a particular Exchange Online rule is triggered, the information will be saved and displayed in the CSV file.

The less good news is that the information about the Exchange Online rule that was triggered doesn’t relate to “name” of the Exchange Online rule, but instead, to the GUID (globally unique identifier) of the Exchange Online rule.

The GUID implemented as a numerical value.

In other words, when we need to analyze the data that stored in the CSV file, we will need to look for the GUID number the particular Exchange Online rule.

The question now is – how can we know what is the GUID number of a specific Exchange Online rule?

And the answer is – by using PowerShell command that will “reveal” this information

To summarize – the option of analyzing information about a particular Exchange Online rule exists, but we will need to a little effort to reap the fruits of success

Using the Exchange Online message trace tool and the historical search option

  • Log in to the Exchange admin portal
  • On the left menu bar, choose – mail flow
  • On the top menu bar, choose – message trace

The default Date range is 48 hours.
To be able to “activates” the option of Exchange Online message trace historical search we will need to use a custom time range (more than 7 days).

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -01

In the *Date range box, choose the option of Custom

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -02

As mentioned, to be able to activate the option of historical search, we will need to define date range that is equal or bigger than 7 days.

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -03

In the following steps, we will need to define the “search filter” such as – sender, recipient, etc.

The requirement is to specify at least one filter such as sender; the recipient is a mandatory requirement.

Pay attention that in our particular scenario, this “mandatory requirement” is a problem because our wish is to get information about all the mail transactions that occurs in a specific date range, so we will be able to locate a specific event in which our Exchange Online Spoofed E-mail rule triggered.

In our scenario, we don’t know if the spoofed E-mail attack performed by a specific “source recipient” (the sender), and we don’t know if the spoofed E-mail sent to a specific destination recipient (our organization recipient).

To overcome this obstacle, we will use a little trick in which we define a “search term” that relates to a particular domain name instead of a specific destination recipient name.

In our scenario, the organization domain name is – o365pilot.com

To be able to look for all the E-mail messages that were sent to our organization recipients (recipients whom their E-mail address includes the domain name o365pilot.com), we will look for a “wild card recipient” using the search term – *@<Domain name>

In our example, the search term is *@o365pilot.com

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -04

In the following screenshot, we can see additional options that are available for us.

The most prominent and important option described as – Include message events and details with the report (number 2).

When checking this option, we ask from Exchange Online to provide us a detailed report that will include all of the “additional information” that not included by default in the standard message trace results.

Other options are:

Report title (number 3). Exchange Online will generate a default report name who will include the current date and time; we can change this default name to another name who will suit our needs.

Notification E-mail address (number 4) – The process of generating the report and save the information to a CSV file can take 2 -6~ hours.

To be able to know when to process end, and to get a notification that informs us that the report (the CSV file) is ready for us, we can add an E-mail address of a designated recipient such as the E-mail address of the Exchange Online administrator.

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -05

After we “hit” the search button, the Exchange Online message trace – historical search option is “triggered”.

A notification message appears that inform us that the request for generating the required report was successfully submitted and that we need to check when the process completed.

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -06

In the following screenshot, we can see the result – the CSV file that included a detailed report about all the required mail transactions.

Note: Notice that the task of generating this report can take up to 2-6~ hours

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -07

In our specific example, we can see that the report includes 126 “rows of information” and we can download the report by choosing the link named- Download this report

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -08

Getting the GUID of the Exchange Online rule

In the next sections, we will review how to use the CSV file that we will get from performing the Exchange Online message trace historical search to analyze the data about a mail flow that is realities to a specific Exchange Online rule.

The pre-requirement for this operation is the information about the GUID value if the particular Exchange Online rule.

As mentioned, the information about the Exchange Online rule doesn’t appear using the “name” of the Exchange Online rule, but instead, by using the Exchange Online GUID value.

To get the required Exchange Online rule GUID, we need to use PowerShell command

The PowerShell command that we use for displaying the GUID value of the existing Exchange Online rules is

Get-TransportRule |FL name, guid
Get infrastructure about the specific Exchange Online rule GUID - 01

In our specific scenario, we want to get information about a specific Exchange Online rule that is named – Detect Spoofed E-mail + Send an incident report

The GUID for this rule is – 41b16365-5e3d-4476-b25b-468772545c01

Get infrastructure about the specific Exchange Online rule GUID

For more information about how to use remote PowerShell for connecting to Exchange Online, you can read the article – Connect to Exchange Online using PowerShell

Using Microsoft excel for analyzing the data in the CSV file

Before we begin with the specific instruction about how to “extract” the required information about a mail flow that is related to a specific Exchange Online rule, two “tips
that will enable us to manage “big chunk of data” when using excel to view the content of a file.

In the following screenshot, we can see the content of the CSV that was created by the Exchange Online message trace historical search

It’s clear to see that the “amount of data” can very easily cause headaches.

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -A01

By default, when we scroll down, the data table headers will “disappear”.

Our task is to “fixate” the table headers, so when we scroll down, it will be clear to what row the particular cell is “belong”.

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -A02

To “fixate” the table headers choose the menu View and click on the icon – Freeze Panes

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -A03

Another option that we should consider using is to copy the content of the Excel sheet that includes the report to an additional Excel sheet so that we will be can execute different Excel option such as filtering and more without “damaging” or “corrupting” the original data.

To be able to copy the content of a specific Excel sheet, right click on the TAB name and choose the menu Move or Copy…

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -B01

In the option box, choose the option – Create a copy

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -B02

In the following screenshot, we can see the copy of the “original Excel sheet.” We can rename the copy to any suitable name.

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -B03

Using Excel find option

In the previous section, we learned how to get the required GUID of the specific Exchange Online rule that we are looking for.

The simplest procedure that we can use to look for information about “mail transactions” that priced by the particular Exchange Online rule is to use a “simple find” option.

Use the keyword key combination CTRL + F

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -C01

In the following screenshot, we can see that the result appears as “ruleid=
in other words, Exchange Online provides information about a specific rule as Rule Id (identification) and relates to the rule by using a GUID.

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -C02

Using the Find all search option

More suitable option for our needs is the opting of “Find all” because the information about the specific rows that include the value that we look for will appear many times.

When using the option of the “Find”, click on the button – Find all

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -D01

In the following screenshot, we can see that the “Find BOX” include all the “instances” in which the particular keyword found.

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -D02

Using the Excel Filter option

An additional option that we can use is – the Excel Filter option.

To be able to “filter out” the specific rows that include the value that we look for, use the following steps:

  • Position the mouse cursor somewhere in the data table.
  • Choose the Data menu and click on the Filter icon
Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -E01

In the following screenshot, we can see that a “small arrow ” appears on the right side of each column header.

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -E02

The information about the Exchange Online rule appears in the column named- custom_data

  • Choose the column header custom_data and click on the small black arrow
  • Choose the menu Text Filters
  • In the submenu choose the menu Contains…
Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -E03

In the text box, add the GUID value of the specific Exchange Online rule that you are looking for.

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -E04

In the following screenshot, we can see the result.
The information that appears is a “filtered information” that includes only the relevant rows meaning, information rows that include the GUID number that we look for.

Analyzing the results of the Exchange spoofed E-mail rule using message trace CSV file -E05

Using Office 365 report to get information about the Exchange Online rule

In the following section, we will review an additional tool – the Office 365 report that can help us to get more information about the mail flow that is related to a specific Exchange Online rule.

The main character of the Office 365 report is that the information provided by using Graff, and the primary purpose of this report is to provide a “high level” view of a specific Office 365 infrastructure.

In our particular scenario, we will use a specific Office 365 report that will help us to get information about an Exchange Online Spoofed E-mail rule that we have created.

At the current time, the Office 365 report cannot be used for providing detailed information about each specific mail transaction, but instead, provide us general inquiries about the “trend” of the particular Exchange Online rule and the number of time in which the Exchange Online rule was “triggered”.

The name of the Exchange Online report that we use for getting information about particular Exchange Online rule is – Rule match for all mail

To be able to view the report that display information about “match” to existing Exchange Online transport rules, use the following steps:

Using Office 365 report to get information about Exchange Online rule -01

In the window that appears, we can see a Graff that describes the mail flow that is related to all the existing Exchange Online rules.

In our scenario, we are interested in a particular Exchange Online rule.
To be able to view information about a specific Exchange Online rule, click on the small arrow in the Rule section box

Using Office 365 report to get information about Exchange Online rule -02

In our specific scenario, we will choose the display information about the Exchange Online spoof rule that we have created named –
Detect Spoofed E-mail – Delete the E-mail + Send an incident report

Using Office 365 report to get information about Exchange Online rule -03

In the following screenshot, we can see the result.
We choose the default time range of “last seven days.”

We can see that the Exchange rule was “activated” or “triggered” in the dates 06/12 between 08/12

In case that we need more detailed information such as the number of time in which the specific Exchange Online rule was activated, we can use the option of – View table

Using Office 365 report to get information about Exchange Online rule -04

In the following screenshot, we can see the “table view“.

We can see that the Exchange Online rule triggered on the date 07/12

Using Office 365 report to get information about Exchange Online rule -05

Verifying if Exchange Online rule was used using the Exchange Online message trace

An additional tool that we can use for getting information about a specific “mail transaction” is by using the option of Exchange Online message trace.

Let’s assume that we have configured an Exchange Online Spoof E-mail rule that should identify and delete mail items; that appear to be spoofed E-mail messages.

The “issue” is that when Exchange Online identifies a spoofed E-mail, the mail will be deleted, but we cannot know about this event, and we are not entirely sure that the Exchange Online rule is “forcing” the required policy.

To verify what happened to a “Spoof E-mail meager” we will simulate a Spoof E-mail attack and then use the Exchange Online message to trace to check if the spoofed E-mail message was “captured” by the Exchange Online Spoof E-mail rule.

In our specific scenario, we have sent again a spoof E-mail message from a un-authenticated recipient who will use the identity of Suzan@o365pilot.com that we try sending email messages to the recipient named bob that uses the E-mail address – Bob@o365pilot.com

In our scenario, we want to test the following Exchange Online rule:

The rule should identify a scene in which host tries to impersonate himself to a legitimate organization recipient

In this case, the rule should

  1. Delete the E-mail message
  2. Generate + send an incident report to a designated recipient

In the following screenshot, we can see the content of our specific Exchange Online
Spoof E-mail rule:

verifying if Exchange Online rule was used using Exchange Online message trace -01

We will simulate the scenario of Spoof E-mail that will be sent by Suzan@o365pilot.com and in the next step, use the Exchange Online message trace to get more information about the flow of the specific spoofed E-mail message that was sent the destination recipient – Bob@o365pilot.com

If you want to read more information about how to simulate spoofed E-mail attack, read the article –How to Simulate E-mail Spoof Attack |Part 11#12

To be able to “track” the mail flow of the spoofed E-mail message that we sent to the destination recipient – Bob@o365pilot.com, we will use the Exchange Online message trace

  • Log in to the Exchange admin portal
  • On the left menu bar, choose –mail flow
  • On the top menu bar, choose – message trace

In our specific scenario, we are looking for E-mail message that was sent to the recipient Bob@o365pilot.com

verifying if Exchange Online rule was used using Exchange Online message trace -02

In the following screenshot, we can see the result. The result detailed all of the E-mail messages that was sent to Bob@o365pilot.com

We are interested in an E-mail message that was sent by Suzan@o365pilot.com

The status of the E-mail is “Failed

At first glance, it looks like a problem, but the translation of the term “Failed” is that the
E-mail message didn’t reach to the destination recipient mailbox.

In our scenario, this is the desired result because we expect that the Exchange Online Spoofed E-mail rule will identify, block and delete the spoofed E-mail message.

verifying if Exchange Online rule was used using Exchange Online message trace -03

In case that we want to get more details about the mail flow, we need to double click on the log row.

Now we can see the “chain of events” that occurred in the E-mail message.

We can see the Exchange Online accept the spoofed E-mail message (appear as RECEIVE)

Because Exchange Online found “match” to the particular rule that created, Exchange Online will follow the instructions that appear in the rule:

  • The spoofed E-mail message will be deleted (number 1).
  • An incident report will be created by Exchange Online and sent to the designated recipient (number 2).
verifying if Exchange Online rule was used using Exchange Online message trace -04

The next article in the current article series

In the next article – Simulating E-mail Spoof Attack |Part 10#12, we will review how to Simulate E-mail Spoof Attack.

o365info Team

o365info Team

This article was written by our team of experienced IT architects, consultants, and engineers.

This Post Has 0 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *