Alerting on changes to Exchange Online threat policies

The Microsoft 365 landscape provides a significant number of security features that an organisation can configure to protect themselves against constantly evolving threats. These protections are often available based on the licence version you have purchased from Microsoft as opposed to being available by default to all customers. In the case of Exchange Online, the following threat policies are available to all customers, without requiring an extra licence (e.g. Defender for Office 365 Plan 1/2):

• Anti-malware protection. Capability that identifies malware in Exchange Online mailboxes. It is composed of layered defences including ability to directly block attachments that are of a certain file type (e.g. block all inbound emails with .HTML file attachments). This is especially useful for preventing application file types from executing when double-clicked in Windows, by ensuring they are never delivered in the first place. A good example here is a .js file, which runs automatically in Windows when double clicked, similar to EXE files. While EXE files are by default blocked by the policy, JS files are not.
• Anti-spam protection. Capability that identifies junk messages (which continuously improves based on emails flagged by users, including the consumer emails on outlook.com). It blocks not just single spam messages but also bulk delivered spam across multiple mailboxes. You can configure these policies to be quite aggressive, in fact you can configure it in a way that all emails from certain countries are automatically flagged as spam.
• Anti-Phishing protection. Capability that helps against spoofed (forged) senders. Spoofed messages appear to originate from someone or somewhere other than the actual source.
• Outbound spam protection. Capability that may deny email sending if certain suspicious sending patterns occur or email sending reaches a limit. This policy also controls whether email forwarding in the organisation is allowed or not (e.g. you may want to disable this feature to avoid GDPR-related data from being leaked outside the organisation).
• Quarantine and its associated policies. Capability that holds malicious/dangerous or unwanted messages. For example, if the anti-malware protection flags a message as malware, it will be held in this quarantine and not delivered to the user (unless released by the user/admin depending on configuration).
• Allow and block domains / URLs / senders. This is a capability that gives admins an ability to override Exchange Online's behaviour as to whether to allow or block emails depending on certain characteristics, such as sender address.

If you also have Defender for Office 365 Plan 1/2 in your licence, you will also get threat policies for:

• Safe attachments. Capability that provides an additional layer of protection for email attachments that have already been scanned by the anti-malware protection but were not flagged as malicious. Safe Attachments uses a virtual environment to check attachments in email messages for harmful attachments (eg. malware and phishing) before they're delivered to recipients (a process known as detonation).
• Safe links. A capability that provides a layer of protection against malicious links that are used in phishing attacks. Safe Links provides URL scanning and rewriting of inbound email messages during mail flow, and time-of-click verification of URLs and links in email messages, Teams, and supported Office 365 apps. Safe Links scanning occurs in addition to regular anti-spam and anti-malware protection.
• Added features in the anti-phishing policies (such as impersonation protection).

These policies are available in the Microsoft Defender portal at https://security.microsoft.com under 'Email & collaboration' > 'Policies & rules' > 'Threat Policies'.

When these policies are configured and/or modified, an audit log is generated. This log can be queried for in the Microsoft 365 unified logs as well as the "Advanced Hunting" section. In the Advanced Hunting section, they are stored in the table CloudAppEvents.

Each policy is associated with a different action types. I have tried to provide an overview in the table below:

Important

Even with the correct licences, you are required to ensure that this activity is in fact stored. It can be configured in https://security.microsoft.com under Settings > Cloud apps > App Connectors. You need a connector for Microsoft 365, which includes all of this activity:

Once enabled, it may take 24-72 hours before you see logs coming in (so be patient).

Querying logs

To query the logs for changes to the Outbound Anti-Spam policy, we would execute the following query:

The query is:

CloudAppEvents
| where ActionType in ("New-HostedOutboundSpamFilterRule", "Enable-HostedOutboundSpamFilterRule", "Disable-HostedOutboundSpamFilterRule", "Remove-HostedOutboundSpamFilterRule", "Set-HostedOutboundSpamFilterRule", "New-HostedOutboundSpamFilterPolicy", "Set-HostedOutboundSpamFilterPolicy", "Remove-HostedOutboundSpamFilterPolicy")

The field ActivityObjects in the output contains the actual settings that the policy has now been configured with. It is not visible in the default column layout, so you need to scroll further left to reach it. You can see a preview below: for example, NotifyOutboundSpamRecipients is set to an address, which is exactly the kind of change worth knowing about.

By using the above mentioned action types, we can either query the Advanced Hunting logs, or we can create and schedule custom detections to run e.g. daily. Custom detection is particularly useful as it will ensure that for any changes, an alert is generated.

Conclusion

Alerting on policy changes can help identify unwanted/unauthorized changes. While a certain Exchange Online licence is required for some of the logs, they are not collected by default once the licence is upgraded. A manual action is required to begin collecting these logs.

If you have the required licences, it is highly advised to ensure that logs are in fact collected, and alerts are configured.