In the current article, we will review the options that are available for analyzing the result of the Exchange Online Spoof E-mail rule that we have created.
The term “analyze” relate to our ability to answer fundamental questions such as:
How many times the Exchange Online rule that we have created was “triggered”?
What is the scenario in which the Exchange Online rule that we have created was “triggered”?
Table of content | Click to expand
Dealing with spoofed E-mail office 365 | Article Series
This information is relevant for us for many different purposes, but for me, there are main three goals:
The be able to identify false-positive scenarios – the meaning is a scene in which a legitimate mail is mistaken identified as Spoof E-mail and deleted or blocks instead of reaching into the organization user mailbox.
Verifying that the Exchange Online Spoof E-mail rule fulfills its purpose, meaning – manage to identify a Spoof E-mail event and manage to execute that required “actions” that configured.
Identify a particular “trend” of Spoof E-mail attack such as a Spoof E-mail attack that is sent from a specific recipient or aimed to a specific organization recipient etc.
The big question is – does Exchange Online include built reporting tools that can provide us this type of information?
The answer is “Yes” and “No”.
“Yes,” Exchange Online includes a built-in reporting tool that enables us to display “usage report” of a particular Exchange Online rule such as – the number of times that the Exchange on-Premises Exchange Online rule was “triggered.”
“No,” at the current time, the information that is provided by the Exchange Online reports doesn’t include information about each of the specific events in which the Exchange Online rule was “triggered.”
For example, by using the Exchange Online built report, we can understand that a particular Exchange Online rule was “trigger” 20 times in the last month, but the report doesn’t tell us for which “source recipient” or “destination recipient” the rule was triggered.
The good news is that there is a workaround solution for these Inabilities of the Exchange Online to provide a detailed report about each separate event in which the Exchange Online rule was triggered.
The Online message trace tool and the historical search
The solution that we will review in the current article is implemented by using the Exchange Online message trace tool.
Exchange Online message trace tool, can provide us a very detailed information about each incoming mail and for each outgoing mail that was sent to our Office 365 recipients and from our Office 365 recipients.
Exchange Online will keep this information for 90 days.
Also, Exchange Online includes a very useful feature, but at the same time Unknown and an unfamiliar feature that described as historical search.
The Exchange Online message trace feature – historical search, enable us to export all the information to a CSV file (CSV stand for comma separated values).
When we have the information in a CSV file, we can use tools such as Microsoft Excel that have capacities that will help us to analyze the data stored in the CSV file.
The information that Exchange Online export using the CSV can consider as very detailed information and the good news is that each time that a particular Exchange Online rule is triggered, the information will be saved and displayed in the CSV file.
The less good news is that the information about the Exchange Online rule that was triggered doesn’t relate to “name” of the Exchange Online rule, but instead, to the GUID (globally unique identifier) of the Exchange Online rule.
The GUID implemented as a numerical value.
In other words, when we need to analyze the data that stored in the CSV file, we will need to look for the GUID number the particular Exchange Online rule.
The question now is – how can we know what is the GUID number of a specific Exchange Online rule?
And the answer is – by using PowerShell command that will “reveal” this information
To summarize – the option of analyzing information about a particular Exchange Online rule exists, but we will need to a little effort to reap the fruits of success
Using the Exchange Online message trace tool and the historical search option
The default Date range is 48 hours.
To be able to “activates” the option of Exchange Online message tracehistorical search we will need to use a custom time range (more than 7 days).
In the *Date range box, choose the option of Custom
As mentioned, to be able to activate the option of historical search, we will need to define date range that is equal or bigger than 7 days.
In the following steps, we will need to define the “search filter” such as – sender, recipient, etc.
The requirement is to specify at least one filter such as sender; the recipient is a mandatory requirement.
Pay attention that in our particular scenario, this “mandatory requirement” is a problem because our wish is to get information about all the mail transactions that occurs in a specific date range, so we will be able to locate a specific event in which our Exchange Online Spoofed E-mail rule triggered.
In our scenario, we don’t know if the spoofed E-mail attack performed by a specific “source recipient” (the sender), and we don’t know if the spoofed E-mail sent to a specific destination recipient (our organization recipient).
To overcome this obstacle, we will use a little trick in which we define a “search term” that relates to a particular domain name instead of a specific destination recipient name.
In our scenario, the organization domain name is – o365pilot.com
To be able to look for all the E-mail messages that were sent to our organization recipients (recipients whom their E-mail address includes the domain name o365pilot.com), we will look for a “wild card recipient” using the search term – *@<Domain name>
In our example, the search term is *@o365pilot.com
In the following screenshot, we can see additional options that are available for us.
The most prominent and important option described as – Include message events and details with the report (number 2).
When checking this option, we ask from Exchange Online to provide us a detailed report that will include all of the “additional information” that not included by default in the standard message trace results.
Other options are:
Report title (number 3). Exchange Online will generate a default report name who will include the current date and time; we can change this default name to another name who will suit our needs.
Notification E-mail address (number 4) – The process of generating the report and save the information to a CSV file can take 2 -6~ hours.
To be able to know when to process end, and to get a notification that informs us that the report (the CSV file) is ready for us, we can add an E-mail address of a designated recipient such as the E-mail address of the Exchange Online administrator.
After we “hit” the search button, the Exchange Online message trace – historical search option is “triggered”.
A notification message appears that inform us that the request for generating the required report was successfully submitted and that we need to check when the process completed.
In the following screenshot, we can see the result – the CSV file that included a detailed report about all the required mail transactions.
Notice that the task of generating this report can take up to 2-6~ hours.
In our specific example, we can see that the report includes 126 “rows of information” and we can download the report by choosing the link named- Download this report
Getting the GUID of the Exchange Online rule
In the next sections, we will review how to use the CSV file that we will get from performing the Exchange Online message trace historical search to analyze the data about a mail flow that is realities to a specific Exchange Online rule.
The pre-requirement for this operation is the information about the GUID value if the particular Exchange Online rule.
As mentioned, the information about the Exchange Online rule doesn’t appear using the “name” of the Exchange Online rule, but instead, by using the Exchange Online GUID value.
To get the required Exchange Online rule GUID, we need to use PowerShell command
The PowerShell command that we use for displaying the GUID value of the existing Exchange Online rules is
Get-TransportRule |FL name, guid
In our specific scenario, we want to get information about a specific Exchange Online rule that is named – Detect Spoofed E-mail + Send an incident report
The GUID for this rule is – 41b16365-5e3d-4476-b25b-468772545c01
Using Microsoft excel for analyzing the data in the CSV file
Before we begin with the specific instruction about how to “extract” the required information about a mail flow that is related to a specific Exchange Online rule, two “tips
that will enable us to manage “big chunk of data” when using excel to view the content of a file.
In the following screenshot, we can see the content of the CSV that was created by the Exchange Online message trace historical search
It’s clear to see that the “amount of data” can very easily cause headaches.
By default, when we scroll down, the data table headers will “disappear”.
Our task is to “fixate” the table headers, so when we scroll down, it will be clear to what row the particular cell is “belong”.
To “fixate” the table headers choose the menu View and click on the icon – Freeze Panes
Another option that we should consider using is to copy the content of the Excel sheet that includes the report to an additional Excel sheet so that we will be can execute different Excel option such as filtering and more without “damaging” or “corrupting” the original data.
To be able to copy the content of a specific Excel sheet, right click on the TAB name and choose the menu Move or Copy…
In the option box, choose the option – Create a copy
In the following screenshot, we can see the copy of the “original Excel sheet.” We can rename the copy to any suitable name.
Using Excel find option
In the former section, we learn how to get the required GUID of the specific Exchange Online rule that we look for.
The simplest procedure that we can use to look for information about “mail transactions” that priced by the particular Exchange Online rule is to use a “simple find” option.
Use the keyword key combination CTRL + F
In the following screenshot, we can see that the result appears as “ruleid=”
in other words, Exchange Online provides information about a specific rule as Rule Id (identification) and relates to the rule by using a GUID.
Using the Find all search option
More suitable option for our needs is the opting of “Find all” because the information about the specific rows that include the value that we look for will appear many times.
When using the option of the “Find”, click on the button – Find all
In the following screenshot, we can see that the “Find BOX” include all the “instances” in which the particular keyword found.
Using the Excel Filter option
An additional option that we can use is – the Excel Filter option.
To be able to “filter out” the specific rows that include the value that we look for, use the following steps:
Position the mouse cursor somewhere in the data table.
Choose the Data menu and click on the Filter icon
In the following screenshot, we can see that a “small arrow ” appears on the right side of each column header.
The information about the Exchange Online rule appears in the column named- custom_data
Choose the column header custom_data and click on the small black arrow
Choose the menu Text Filters
In the submenu choose the menu Contains…
In the text box, add the GUID value of the specific Exchange Online rule that you are looking for.
In the following screenshot, we can see the result.
The information that appears is a “filtered information” that includes only the relevant rows meaning, information rows that include the GUID number that we look for.
Using Office 365 report to get information about the Exchange Online rule
In the following section, we will review an additional tool – the Office 365 report that can help us to get more information about the mail flow that is related to a specific Exchange Online rule.
The main character of the Office 365 report is that the information provided by using Graff, and the primary purpose of this report is to provide a “high level” view of a specific Office 365 infrastructure.
In our particular scenario, we will use a specific Office 365 report that will help us to get information about an Exchange Online Spoofed E-mail rule that we have created.
At the current time, the Office 365 report cannot be used for providing detailed information about each specific mail transaction, but instead, provide us general inquiries about the “trend” of the particular Exchange Online rule and the number of time in which the Exchange Online rule was “triggered”.
The name of the Exchange Online report that we use for getting information about particular Exchange Online rule is – Rule match for all mail
To be able to view the report that display information about “match” to existing Exchange Online transport rules, use the following steps:
In the window that appears, we can see a Graff that describes the mail flow that is related to all the existing Exchange Online rules.
In our scenario, we are interested in a particular Exchange Online rule.
To be able to view information about a specific Exchange Online rule, click on the small arrow in the Rule section box
In our specific scenario, we will choose the display information about the Exchange Online spoof rule that we have created named – Detect Spoofed E-mail – Delete the E-mail + Send an incident report
In the following screenshot, we can see the result.
We choose the default time range of “last seven days.”
We can see that the Exchange rule was “activated” or “triggered” in the dates 06/12 between 08/12
In case that we need more detailed information such as the number of time in which the specific Exchange Online rule was activated, we can use the option of – View table
In the following screenshot, we can see the “table view“.
We can see that the Exchange Online rule triggered on the date 07/12
Verifying if Exchange Online rule was used using the Exchange Online message trace
An additional tool that we can use for getting information about a specific “mail transaction” is by using the option of Exchange Online message trace.
Let’s assume that we have configured an Exchange Online Spoof E-mail rule that should identify and delete mail items; that appear to be spoofed E-mail messages.
The “issue” is that when Exchange Online identifies a spoofed E-mail, the mail will be deleted, but we cannot know about this event, and we are not entirely sure that the Exchange Online rule is “forcing” the required policy.
To verify what happened to a “Spoof E-mail meager” we will simulate a Spoof E-mail attack and then use the Exchange Online message to trace to check if the spoofed E-mail message was “captured” by the Exchange Online Spoof E-mail rule.
In our specific scenario, we have sent again a spoof E-mail message from a un-authenticated recipient who will use the identity of Suzan@o365pilot.com that we try sending email messages to the recipient named bob that uses the E-mail address – Bob@o365pilot.com
In our scenario, we want to test the following Exchange Online rule:
The rule should identify a scene in which host tries to impersonate himself to a legitimate organization recipient
In this case, the rule should
Delete the E-mail message
Generate + send an incident report to a designated recipient
In the following screenshot, we can see the content of our specific Exchange Online
Spoof E-mail rule:
We will simulate the scenario of Spoof E-mail that will be sent by Suzan@o365pilot.com and in the next step, use the Exchange Online message trace to get more information about the flow of the specific spoofed E-mail message that was sent the destination recipient – Bob@o365pilot.com