Mimecast for Splunk Administrators Guide

Document created by user.zL0FB6L9lN Employee on Jul 25, 2016Last modified by user.oxriBaJeN4 on Jun 28, 2019
Version 40Show Document
  • View in full screen mode

Mimecast for Splunk allows a Splunk Enterprise administrator to ingest events derived from data generated by the Mimecast platform i.e. audit, email, and Targeted Threat Protection, in addition to a service health overview using pre-built dashboards. Data is mapped to the Common Information Model where relevant, to allow users to correlate Mimecast events with other data sources.

 

Mimecast for Splunk can be used for many reasons. For example to ingest:

  • Secure Email Gateway events into Splunk to feed the Splunk Enterprise Security or User Behavior Analytics products.
  • Mimecast events to create custom reports on email traffic, security, and usage.

 

Release Notes

 

Author: Mimecast

App Version: 3.1.1

Vendor Products: Mimecast (S1, D1, M2, M2A)

Has Index Operations: False, no requirement to install on index layer.

Create an Index: True, impacts storage, the app requires a dedicated index (Mimecast) but does not use report, search or data model acceleration. The app uses summary indexing to create single line CIM compliant events from multi-lined input data sources.

Splunk Enterprise Versions: 7.2, 7.1, 7.0, 6.6, 6.5, 6.4

Platforms: Platform independent

CIM Versions: 4.x

 

v3.1.1 Changes

 

This version includes the following changes:

  • Support for the new SIEM log format.

  • Support for TTP Impersonation Protect logs.

  • Support for TTP Attachment Protect logs.

  • Support for adding multiple Mimecast tenants by making an Application key and Application ID per input.

  • Support for better filtering of data by the Mimecast tenant has been added. A new "splunkAccountCode" field has been added to all logs prior to being ingested into Splunk.

 

Support and Resources

 

Mimecast for Splunk 3.1.1 has been tested on Splunk Enterprise versions 7.0.1 hosted on Windows, Linux, and Mac OSX. Other host operating systems and Splunk versions later than 6.4.x should also work but have not been explicitly tested.

 

You can ask questions specific to Mimecast for Splunk on the Community Forums. You can also contact Mimecast's Technical Support team.

 

Installation and Configuration

 

Considerations

 

The following System Requirements must be met:

  • The app does not require any other software, other than a working Splunk installation.

 

This version cannot upgrade from previous versions. Data ingested by previous versions remain in place, but won't be migrated to the new index. Do not remove previous versions until v3 has successfully been configured and is running as expected in your environment. Once v3 is running, disable previous versions until a migration path becomes available in a future release.

 

Installation

 

The application can either be downloaded directly from SplunkBase or installed from inside Splunk:

  1. Log on to the Splunk Web Application.
  2. Select Manage Apps from the top left-hand menu. The Apps page displays.
  3. Click on the Browse More Apps button.
  4. Search for Mimecast for Splunk.
  5. Follow the wizard steps to install the app.

 

Configuring Your Network

 

The app uses the Mimecast API to collect data and events. You must ensure that the server hosting Splunk and the Mimecast application has outbound HTTPS access (TCP port 443) to the following hosts, depending on the region where your Mimecast account is hosted:

 

RegionHost(s)
Europe (excluding Germany)api.mimecast.com AND eu-api.mimecast.com
Germanyapi.mimecast.com AND de-api.mimecast.com
United Statesapi.mimecast.com AND us-api.mimecast.com
South Africaapi.mimecast.com AND za-api.mimecast.com
Australiaapi.mimecast.com AND au-api.mimecast.com
Offshoreapi.mimecast.com AND je-api.mimecast.com

 

Configuring Mimecast Permissions

 

The app uses the Mimecast API to collect data and transforms these into events. See the table below for the endpoints used, and the Mimecast administrator permissions required. For convenience, all permissions are included in the Basic Administrator role.

 

EndpointPermission Required
/api/login/discover-authenticationN/A
/api/email/get-email-queuesDashboard | Read
/api/directory/get-connectionsServices | Directory Sync | Read
/api/journaling/get-serviceServices | Journaling | Read
/api/audit/get-audit-eventsAccount | Logs | Read
/api/audit/get-siem-logsGateway | Tracking | Read
/api/ttp/url/get-logsMonitoring | URL Protection | Read
The Admin IP Ranges specified in the Administration Console are respected when attempting to retrieve SIEM logs. Admin IP Ranges are entered in the "User Access and Permissions section of your Account Settings. See the Your Mimecast Account Settings page for further details.

Configuring Authentication Tokens

 

The Mimecast for Splunk app requires a Mimecast Administrator authentication token's (Access Key and Secret Key) to be entered during the configuration of the app. By default an authentication token expires after three days, meaning the app stops collecting data and events after three days without manual intervention. To prevent this, create a user and Authentication Profile that defines a long-lived authentication token as described below.

Advanced Account Administration customers must repeat this process, and use a dedicated administrative user for each Mail Processing account. To additionally collect data from the Master account and any Grouping accounts configured in the Advanced Account Administration structure:

  • Federated Administration must be enabled throughout the structure as described in the Federated Administration guide.
  • The process below should be repeated for an administrator from the Master account using the Federated Administration domain.

Configuring the Mimecast Administration Console

 

To configure authentication tokens in Mimecast's Administration Console:

  1. Enable logging:
    1. Click on the Administration toolbar menu item.
    2. Select the Account | Account Settings menu item. 
    3. Select the Enhanced Logging section.
    4. Select the types of logs you want to enable:
      • Inbound: logs for messages from external senders to internal recipients.
      • Outbound: logs for messages from internal senders to external recipients.
      • Internal: logs for messages between internal domains.
    5. Click on the Save button.
      Once settings are saved, the Mimecast MTA starts logging data for your account. Logs are available to Mimecast for Splunk 30 minutes later.
  2. Create a User. See the  "Creating a User" section of the Creating / Editing Mimecast Users page for further details. Keep a note of the password set, as you'll use this to get your Authentication Token.
  3. Add the user to a Basic Administrator Role. See the "Adding Users to a Role" section of the Managing Administrator Roles page for further details.
  4. Create a Profile Group. See the "Creating a Group" section of the Managing Groups page for further details.
  5. Add the User to the Profile Group. See the "Adding Email Addresses / Domains to a Group" section of the Managing Groups page for further details.
    1. Select the Add Email Addresses setting.
    2. Add the User created above.
  6. Create an Authentication Profile. See the Configuring an Authentication Profile page for further details.
    1. Set the Authentication TTL option to "Never Expires" to ensure the Authentication Token won't expire.
    2. Leave all other settings as the default values.
  7. Create an Application Setting. See the Configuring Application Settings page for full details.
    1. Select the Profile group created above in the Group option.
    2. Select the Authentication Profile created above by clicking on the Lookup button.
    3. Leave all other settings as the default values.

 

Configuring the Mimecast for Splunk App

 

To create an authentication token, follow the steps in the Managing API Applications page. Specify an application name of “Mimecast for Splunk v3” to obtain an Application ID, Application Key, Access Key and Secret Key.

 

To configure authentication tokens for the Mimecast for Splunk app:

  1. Log on to the Splunk Web Console.
  2. Ensure you've selected the Mimecast for Splunk app from the Apps dropdown.
  3. Click on the Configure Data Input link.

    If you've already configured a data input, you must copy and paste the Application Id and Application Key for all existing data inputs.

  4. Configure one or more Data Inputs to enable the app to collect data from Mimecast:
    Before configuring data inputs, create a dedicated index named "mimecast" entered in lower case. See the Create Custom Indexes page in the Splunk documentation for further details. 
    1. Click on the Configure Data Input link from the home page. The Inputs screen displays.
    2. Click on the Create New Input button.
    3. Select an Input Type.
    4. Add a Name to identify the input.
    5. Set Interval to 300.
    6. Set Index to mimecast.
    7. Paste in the Application Id and Application Key values.
    8. Paste in the Access Key and Secret Key values obtained above.
    9. Set the Account Code to your Mimecast account code. This can be found on the home page of the Mimecast Administration Console.
    10. Set the Base URL to the regions API URL (e.g. https://eu-api.mimecast.com). See the "Configuring Your Network" section above.
    11. Click on the Add button to save the new input.
      Advanced Account Administration Customers can use this process to configure a data input for each of their mail processing accounts, using an administrator from the mail processing account itself. However, to collect events from master and / or grouping accounts, contact our Support Team. They will be able to guide you through the process of enabling federated administration in your Mimecast hierarchy, and the process of configuring a data input for these account types.

Using the App

 

The app is comprised of a number of dashboards displaying the available data. See the table below for a description of the dashboards and the expected update frequencies.

 

DashboardDescriptionUpdate FrequencyAvailable DataData Retention
Email Activity > Email Activity SummaryDisplays messages received over time by route and spark lines for Rejections, Bounces, and Held Messages.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Email Activity > Email DeliveryDisplays visualizations for messages delivered by the Mimecast MTA.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Email Activity > Email ReceiptDisplays visualizations for messages received by the Mimecast MTA.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Email Activity > TLSDisplays visualizations detailing the secure delivery and receipt of messages processed by the Mimecast MTA.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Email Activity > AV / ASDisplays visualizations for messages detected as Spam or carrying a virus.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Targeted Threat Protection > Attachment ProtectDisplays visualizations for data logged when a potentially malicious attachment is identified by the Mimecast sandbox. Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Targeted Threat Protection > Impersonation ProtectDisplays visualizations for Targeted Threat Protection Impersonation Protect message characteristics detected in a message.
These visualizations do not necessarily indicate that a policy action was triggered, but that one of the characteristics was detected.
Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Targeted Threat Protection > URL ProtectDisplays visualizations for data logged when a user clicks on a potentially malicious link in an email.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Audit and Access > Audit LogDisplays a feed of administrator and authentication activity.Real timeFrom ten minutes before you configured the Splunk Data Input.The lifetime of your Mimecast account.
Audit and Access > Access AttemptsDisplays visualizations for access attempts made to the Mimecast account.Real timeFrom ten minutes before you configured the Splunk Data Input.The lifetime of your Mimecast account.
Service HealthA dashboard displaying inbound and outbound message queue totals, as well as the status of Directory and Journal integrations if configured,Real timeFrom one hour before you configured the Splunk Data Input.n/a
TroubleshootMimecast specific events from the splunkd.log file.Real timen/an/a
The current app design does not include the subject field in Journal logs.

Troubleshooting

 

The Modular Input script writes to the splunkd.log file. The Troubleshoot dashboard is the first place to check if you're experiencing any problems or errors. It is provided as part of the Mimecast for Splunk application and filters the splunkd.log for Mimecast specific events. It can also be used to view additional data contained in the Splunkd log. To achieve this you will need to enable the Include platform logs option on the Troubleshoot dashboard. Additionally, the log level can be set for the Mimecast for Splunk app via the Configuration | Logging menu.

 

Dashboards Aren't Updating

 

If your dashboards aren't updating correctly:

  1. Check you have a valid Splunk Data Input configured.
  2. Check the date filters are set to a time period where you're expecting to see data.
  3. Check the log file to confirm events are being added to Splunk.

 

Request Returned with Status Code 418 ()

 

If you see "Request returned with status code 418 ()" in the app logs, the access key and secret key used in your Data Input has expired. Ensure you have followed the configuration steps above, to create a user with an Authentication Token that doesn't expire.

Attachments

    Outcomes