- Release Notes
- Installation and Configuration
- Using the App
Mimecast for Splunk allows a Splunk Enterprise administrator to ingest events derived from data generated by the Mimecast platform i.e. audit, email, and Targeted Threat Protection, in addition to a service health overview using pre-built dashboards. Data is mapped to the Common Information Model where relevant, to allow users to correlate Mimecast events with other data sources.
Mimecast for Splunk can be used for many reasons. For example to ingest:
- Secure Email Gateway events into Splunk to feed the Splunk Enterprise Security or User Behavior Analytics products.
- Mimecast events to create custom reports on email traffic, security, and usage.
App Version: 3.1.1
Vendor Products: Mimecast (S1, D1, M2, M2A)
Has Index Operations: False, no requirement to install on index layer.
Create an Index: True, impacts storage, the app requires a dedicated index (Mimecast) but does not use report, search or data model acceleration. The app uses summary indexing to create single line CIM compliant events from multi-lined input data sources.
Splunk Enterprise Versions: 7.2, 7.1, 7.0, 6.6, 6.5, 6.4
Platforms: Platform independent
CIM Versions: 4.x
This version includes the following changes:
Support for the new SIEM log format.
Support for TTP Impersonation Protect logs.
Support for TTP Attachment Protect logs.
Support for adding multiple Mimecast tenants by making an Application key and Application ID per input.
Support for better filtering of data by the Mimecast tenant has been added. A new "splunkAccountCode" field has been added to all logs prior to being ingested into Splunk.
Support and Resources
Mimecast for Splunk 3.1.1 has been tested on Splunk Enterprise versions 7.0.1 hosted on Windows, Linux, and Mac OSX. Other host operating systems and Splunk versions later than 6.4.x should also work but have not been explicitly tested.
Installation and Configuration
The following System Requirements must be met:
- The app does not require any other software, other than a working Splunk installation.
- As this add-on runs on Splunk Enterprise, Splunk Enterprise System Requirements apply.
This version cannot upgrade from previous versions. Data ingested by previous versions remain in place, but won't be migrated to the new index. Do not remove previous versions until v3 has successfully been configured and is running as expected in your environment. Once v3 is running, disable previous versions until a migration path becomes available in a future release.
The application can either be downloaded directly from SplunkBase or installed from inside Splunk:
- Log on to the Splunk Web Application.
- Select Manage Apps from the top left-hand menu. The Apps page displays.
- Click on the Browse More Apps button.
- Search for Mimecast for Splunk.
- Follow the wizard steps to install the app.
Configuring Your Network
The app uses the Mimecast API to collect data and events. You must ensure that the server hosting Splunk and the Mimecast application has outbound HTTPS access (TCP port 443) to the following hosts, depending on the region where your Mimecast account is hosted:
|Europe (excluding Germany)||api.mimecast.com AND eu-api.mimecast.com|
|Germany||api.mimecast.com AND de-api.mimecast.com|
|United States||api.mimecast.com AND us-api.mimecast.com|
|South Africa||api.mimecast.com AND za-api.mimecast.com|
|Australia||api.mimecast.com AND au-api.mimecast.com|
|Offshore||api.mimecast.com AND je-api.mimecast.com|
Configuring Mimecast Permissions
The app uses the Mimecast API to collect data and transforms these into events. See the table below for the endpoints used, and the Mimecast administrator permissions required. For convenience, all permissions are included in the Basic Administrator role.
|/api/email/get-email-queues||Dashboard | Read|
|/api/directory/get-connections||Services | Directory Sync | Read|
|/api/journaling/get-service||Services | Journaling | Read|
|/api/audit/get-audit-events||Account | Logs | Read|
|/api/audit/get-siem-logs||Gateway | Tracking | Read|
|/api/ttp/url/get-logs||Monitoring | URL Protection | Read|
Configuring Authentication Tokens
The Mimecast for Splunk app requires a Mimecast Administrator authentication token's (Access Key and Secret Key) to be entered during the configuration of the app. By default an authentication token expires after three days, meaning the app stops collecting data and events after three days without manual intervention. To prevent this, create a user and Authentication Profile that defines a long-lived authentication token as described below.
Advanced Account Administration customers must repeat this process, and use a dedicated administrative user for each Mail Processing account. To additionally collect data from the Master account and any Grouping accounts configured in the Advanced Account Administration structure:
- Federated Administration must be enabled throughout the structure as described in the Federated Administration guide.
- The process below should be repeated for an administrator from the Master account using the Federated Administration domain.
Configuring the Mimecast Administration Console
To configure authentication tokens in Mimecast's Administration Console:
- Enable logging:
- Click on the Administration toolbar menu item.
- Select the Account | Account Settings menu item.
- Select the Enhanced Logging section.
- Select the types of logs you want to enable:
- Inbound: logs for messages from external senders to internal recipients.
- Outbound: logs for messages from internal senders to external recipients.
- Internal: logs for messages between internal domains.
- Click on the Save button. Once settings are saved, the Mimecast MTA starts logging data for your account. Logs are available to Mimecast for Splunk 30 minutes later.
- Create a User. See the "Creating a User" section of the Creating / Editing Mimecast Users page for further details. Keep a note of the password set, as you'll use this to get your Authentication Token.
- Add the user to a Basic Administrator Role. See the "Adding Users to a Role" section of the Managing Administrator Roles page for further details.
- Create a Profile Group. See the "Creating a Group" section of the Managing Groups page for further details.
- Add the User to the Profile Group. See the "Adding Email Addresses / Domains to a Group" section of the Managing Groups page for further details.
- Select the Add Email Addresses setting.
- Add the User created above.
- Create an Authentication Profile. See the Configuring an Authentication Profile page for further details.
- Set the Authentication TTL option to "Never Expires" to ensure the Authentication Token won't expire.
- Leave all other settings as the default values.
- Create an Application Setting. See the Configuring Application Settings page for full details.
- Select the Profile group created above in the Group option.
- Select the Authentication Profile created above by clicking on the Lookup button.
- Leave all other settings as the default values.
Configuring the Mimecast for Splunk App
To generate an authentication token:
- Download the login_scripts_v3.zip file and extract the contents.
- On Windows OS:
This process has been tested in PowerShell version 4 and 5. When copying / pasting from the PowerShell window, remove any line breaks in the access key and secret key values to prevent later issues.
- Start a PowerShell command prompt. See the Starting Windows PowerShell page of the Microsoft documentation for full details.
- Navigate to the location where the contents of login_scripts_v3.zip have been extracted.
- Execute the login_v3.ps1 script to be guided through the steps required to receive an access key and secret key. See the How to Write and Run Scripts in the Windows PowerShell page of the Microsoft documentation for full details.
- The script outputs the access key and secret key.
- Keep the values to hand as they are required when setting up Data Inputs.
- On Mac OSX or *Nix
This python script is dependent on python v2.7
- Open a Terminal application.
- Navigate to the location of the extracted login_scripts_v3.zip file.
- Execute the login_v3.py script to be guided through the steps required to receive an access key and secret key:
- The script outputs the access key and secret key.
- Keep the values to hand, as they are required when setting up Data Inputs.
To configure authentication tokens for the Mimecast for Splunk app:
- Log on to the Splunk Web Console.
- Ensure you've selected the Mimecast for Splunk app from the Apps dropdown.
- Click on the Configure Data Input link.
If you've already configured a data input, you must copy and paste the Application Id and Application Key for all existing data inputs.
- Configure one or more Data Inputs to enable the app to collect data from Mimecast: Before configuring data inputs, create a dedicated index named "mimecast" entered in lower case. See the Create Custom Indexes page in the Splunk documentation for further details.
- Click on the Configure Data Input link from the home page. The Inputs screen displays.
- Click on the Create New Input button.
- Select an Input Type.
- Add a Name to identify the input.
- Set Interval to 300.
- Set Index to mimecast.
- Paste in the Application Id and Application Key values.
- Paste in the Access Key and Secret Key values obtained above.
- Set the Account Code to your Mimecast account code. This can be found on the home page of the Mimecast Administration Console.
- Set the Base URL to the regions API URL (e.g. https://eu-api.mimecast.com). See the "Configuring Your Network" section above.
- Click on the Add button to save the new input. Advanced Account Administration Customers can use this process to configure a data input for each of their mail processing accounts, using an administrator from the mail processing account itself. However, to collect events from master and / or grouping accounts, contact our Support Team. They will be able to guide you through the process of enabling federated administration in your Mimecast hierarchy, and the process of configuring a data input for these account types.
Using the App
The app is comprised of a number of dashboards displaying the available data. See the table below for a description of the dashboards and the expected update frequencies.
|Dashboard||Description||Update Frequency||Available Data||Data Retention|
|Email Activity > Email Activity Summary||Displays messages received over time by route and spark lines for Rejections, Bounces, and Held Messages.||Every 30 minutes||From when you enabled Enhanced Logging.||7 days|
|Email Activity > Email Delivery||Displays visualizations for messages delivered by the Mimecast MTA.||Every 30 minutes||From when you enabled Enhanced Logging.||7 days|
|Email Activity > Email Receipt||Displays visualizations for messages received by the Mimecast MTA.||Every 30 minutes||From when you enabled Enhanced Logging.||7 days|
|Email Activity > TLS||Displays visualizations detailing the secure delivery and receipt of messages processed by the Mimecast MTA.||Every 30 minutes||From when you enabled Enhanced Logging.||7 days|
|Email Activity > AV / AS||Displays visualizations for messages detected as Spam or carrying a virus.||Every 30 minutes||From when you enabled Enhanced Logging.||7 days|
|Targeted Threat Protection > Attachment Protect||Displays visualizations for data logged when a potentially malicious attachment is identified by the Mimecast sandbox.||Every 30 minutes||From when you enabled Enhanced Logging.||7 days|
|Targeted Threat Protection > Impersonation Protect||Displays visualizations for Targeted Threat Protection Impersonation Protect message characteristics detected in a message. |
These visualizations do not necessarily indicate that a policy action was triggered, but that one of the characteristics was detected.
|Every 30 minutes||From when you enabled Enhanced Logging.||7 days|
|Targeted Threat Protection > URL Protect||Displays visualizations for data logged when a user clicks on a potentially malicious link in an email.||Every 30 minutes||From when you enabled Enhanced Logging.||7 days|
|Audit and Access > Audit Log||Displays a feed of administrator and authentication activity.||Real time||From ten minutes before you configured the Splunk Data Input.||The lifetime of your Mimecast account.|
|Audit and Access > Access Attempts||Displays visualizations for access attempts made to the Mimecast account.||Real time||From ten minutes before you configured the Splunk Data Input.||The lifetime of your Mimecast account.|
|Service Health||A dashboard displaying inbound and outbound message queue totals, as well as the status of Directory and Journal integrations if configured,||Real time||From one hour before you configured the Splunk Data Input.||n/a|
|Troubleshoot||Mimecast specific events from the splunkd.log file.||Real time||n/a||n/a|
The Modular Input script writes to the splunkd.log file. The Troubleshoot dashboard is the first place to check if you're experiencing any problems or errors. It is provided as part of the Mimecast for Splunk application and filters the splunkd.log for Mimecast specific events. It can also be used to view additional data contained in the Splunkd log. To achieve this you will need to enable the Include platform logs option on the Troubleshoot dashboard. Additionally, the log level can be set for the Mimecast for Splunk app via the Configuration | Logging menu.
Dashboards Aren't Updating
If your dashboards aren't updating correctly:
- Check you have a valid Splunk Data Input configured.
- Check the date filters are set to a time period where you're expecting to see data.
- Check the log file to confirm events are being added to Splunk.
Request Returned with Status Code 418 ()
If you see "Request returned with status code 418 ()" in the app logs, the access key and secret key used in your Data Input has expired. Ensure you have followed the configuration steps above, to create a user with an Authentication Token that doesn't expire.