Mimecast for Splunk Administrators Guide

Document created by user.zL0FB6L9lN Expert on Jul 25, 2016Last modified by user.oxriBaJeN4 on Oct 25, 2018
Version 35Show Document
  • View in full screen mode

Mimecast for Splunk allows a Splunk Enterprise administrator to ingest events derived from data generated by the Mimecast platform i.e. audit, email, and Targeted Threat Protection, in addition to a service health overview using pre-built dashboards. Data is mapped to the Common Information Model where relevant, to allow users to correlate Mimecast events with other data sources.

 

Mimecast for Splunk can be used for many reasons. For example to ingest:

  • Secure Email Gateway events into Splunk to feed the Splunk Enterprise Security or User Behavior Analytics products.
  • Mimecast events to create custom reports on email traffic, security, and usage.

 

Release Notes

 

Author: Mimecast

App Version: 3.0.1

Vendor Products: Mimecast (S1, D1, M2, M2A)

Has index operations: False, no requirement to install on index layer

Create an index: True, impacts storage, the app requires a dedicated index (Mimecast) but does not use report, search or data model acceleration. The app uses summary indexing to create single line CIM compliant events from multi-lined input data sources.

Splunk Enterprise Versions: 7.2, 7.1, 7.0, 6.6, 6.5, 6.4

Platforms: Platform independent

CIM Versions: 4.9, 4.10

 

v3.0.1 Changes

 

This version includes the following changes:

  • Added distinct input types (SIEM, Email, Directory, Journal, Audit, and Targeted Threat Protection URL).
  • Changed the data source and enhanced the Targeted Threat Protection URL dashboard.
  • Optimized and enhanced the performance of query generation and data collection.
  • Complies to Common Information Model (CIM) v4.10.
  • Dashboards align with the new architecture.
  • Mapping data model to CIM properties.

 

Support and Resources

 

Mimecast for Splunk 3.0.1 has been tested on Splunk Enterprise versions 7.0.1 hosted on Windows, Linux, and Mac OSX. Other host operating systems and Splunk versions later than 6.4.x should also work but have not been explicitly tested.

 

You can ask questions specific to Mimecast for Splunk on the Community Forums. You can also contact Mimecast's Technical Support team.

 

Installation and Configuration

 

Considerations

 

The following System Requirements must be met:

  • The app does not require any other software, other than a working Splunk installation.

 

This version cannot upgrade from previous versions. Data ingested by previous versions remain in place, but won't be migrated to the new index. Do not remove previous versions until v3 has successfully been configured and is running as expected in your environment. Once v3 is running, disable previous versions until a migration path becomes available in a future release.

 

Installation

 

The application can either be downloaded directly from SplunkBase or installed from inside Splunk:

  1. Log on to the Splunk Web Application.
  2. Select Manage Apps from the top left-hand menu. The Apps page displays.
  3. Click on the Browse More Apps button.
  4. Search for Mimecast for Splunk.
  5. Follow the wizard steps to install the app.

 

Configuring Your Network

 

The app uses the Mimecast API to collect data and events. You must ensure that the server hosting Splunk and the Mimecast application has outbound HTTPS access (TCP port 443) to the following hosts, depending on the region where your Mimecast account is hosted:

 

RegionHost(s)
Europe (excluding Germany)api.mimecast.com AND eu-api.mimecast.com
Germanyapi.mimecast.com AND de-api.mimecast.com
United Statesapi.mimecast.com AND us-api.mimecast.com
South Africaapi.mimecast.com AND za-api.mimecast.com
Australiaapi.mimecast.com AND au-api.mimecast.com
Offshoreapi.mimecast.com AND je-api.mimecast.com

 

Configuring Mimecast Permissions

 

The app uses the Mimecast API to collect data and transforms these into events. See the table below for the endpoints used, and the Mimecast administrator permissions required. For convenience, all permissions are included in the Basic Administrator role.

 

EndpointPermission Required
/api/login/discover-authenticationN/A
/api/email/get-email-queuesDashboard | Read
/api/directory/get-connectionsServices | Directory Sync | Read
/api/journaling/get-serviceServices | Journaling | Read
/api/audit/get-audit-eventsAccount | Logs | Read
/api/audit/get-siem-logsGateway | Tracking | Read
/api/ttp/url/get-logsMonitoring | URL Protection | Read
The Admin IP Ranges specified in the Administration Console are respected when attempting to retrieve SIEM logs. Admin IP Ranges are entered in the "User Access and Permissions section of your Account Settings. See the Your Mimecast Account Settings page for further details.

Configuring Authentication Tokens

 

The Mimecast for Splunk app requires a Mimecast Administrator authentication token's (Access Key and Secret Key) to be entered during the configuration of the app. By default an authentication token expires after three days, meaning the app stops collecting data and events after three days without manual intervention. To prevent this, create a user and Authentication Profile that defines a long-lived authentication token as described below.

Advanced Account Administration customers must repeat this process, and use a dedicated administrative user for each Mail Processing account. To additionally collect data from the Master account and any Grouping accounts configured in the Advanced Account Administration structure:

 

  • Federated Administration must be enabled throughout the structure as described in the Federated Administration guide.
  • The process below should be repeated for an administrator from the Master account using the Federated Administration domain.

Configuring the Mimecast Administration Console

 

To configure authentication tokens in Mimecast's Administration Console:

  1. Enable logging:
    1. Click on the Administration toolbar menu item. A drop down menu displays.
    2. Select the Account | Account Settings menu item. 
    3. Select the Enhanced Logging section.
    4. Select the types of logs you want to enable:
      • Inbound: logs for messages from external senders to internal recipients.
      • Outbound: logs for messages from internal senders to external recipients.
      • Internal: logs for messages between internal domains.
    5. Click on the Save button.
      Once settings are saved, the Mimecast MTA starts logging data for your account. Logs are available to Mimecast for Splunk 30 minutes later.
  2. Create a User. See the  "Creating a User" section of the Creating / Editing Mimecast Users page for further details. Keep a note of the password set, as you'll use this to get your Authentication Token.
  3. Add the user to a Basic Administrator Role. See the "Adding Users to a Role" section of the Managing Administrator Roles page for further details.
  4. Create a Profile Group. See the "Creating a Group" section of the Managing Groups page for further details.
  5. Add the User to the Profile Group. See the "Adding Email Addresses / Domains to a Group" section of the Managing Groups page for further details.
    1. Select the Add Email Addresses setting.
    2. Add the User created above.
  6. Create an Authentication Profile. See the Configuring an Authentication Profile page for further details.
    1. Set the Authentication TTL option to "Never Expires" to ensure the Authentication Token won't expire.
    2. Leave all other settings as the default values.
  7. Create an Application Setting. See the Configuring Application Settings page for full details.
    1. Select the Profile group created above in the Group option.
    2. Select the Authentication Profile created above by clicking on the Lookup button.
    3. Leave all other settings as the default values.

 

Configuring the Splunk Web Console

This step requires access to the Services | API Applications menu in the Mimecast Administration Console. See the Adding an API Application page for full details. Set the Application Name to "Mimecast for Splunk v3" for auditing and identification purposes.

To configure authentication tokens in Splunk's Web Console:

  1. Log on to the Splunk Web Console.
  2. Click on the Mimecast for Splunk app in the top left hand menu.
  3. Configure the Application Id and Application Key;
    1. Click on Configure Add On Variables.
    2. Paste the Application ID and Application Key values into the respective fields.
    3. Click on the Save button.
  4. Generate the Authentication Token.
    1. Download the login_scripts_v3.zip file and extract the contents.
    2. On Windows OS:
      1. Start a PowerShell command prompt. See the Starting Windows PowerShell page of the Microsoft documentation for full details.
      2. Navigate to the location where the contents of login_scripts_v3.zip have been extracted to.
      3. Execute the login_v3.ps1 script to be guided through the steps required to receive an access key and secret key. See the How to Write and Run Scripts in the Windows PowerShell page of the Microsoft documentation for full details.
      4. .\login_v3.ps1
      5. The script outputs the access key and secret key.
      6. Keep the values to hand as they are required when setting up Data Inputs.
      This process has been tested in PowerShell version 4 and 5. When copying / pasting from the PowerShell window, remove any line breaks in the access key and secret key values to prevent later issues.
    3. On Mac OSX or *Nix
      1. Open a Terminal application.
      2. Navigate to the location of the extracted login_scripts_v3.zip file.
      3. Execute the login_v3.py script to be guided through the steps required to receive an access key and secret key:
        python login_v3.py
      4. The script outputs the access key and secret key.
      5. Keep the values to hand, as they are required when setting up Data Inputs.
      This python script is dependent on python v2.7
  5. Configure one or more Data Inputs to enable the app to collect data from Mimecast:
    Before configuring Data Inputs, create a dedicated index named "mimecast" entered in lower case. See the Create Custom Indexes page in the Splunk documentation for further details. 
    1. Click on the Configure Data Input link from the home page. The Inputs screen displays.
    2. Click on the Create New Input button.
    3. Select an Input Type.
    4. Add a Name to identify the input.
    5. Set Interval to 300.
    6. Set Index to mimecast.
    7. Paste the Access Key and Secret Key values obtained above.
    8. Set the Account Code to your Mimecast account code. This can be found on the home page of the Mimecast Administration Console.
    9. Set the Base URL to the regions API URL (e.g. https://eu-api.mimecast.com). See the "Configuring Your Network" section above.
    10. Click on the Add button to save the new input.
      Advanced Account Administration Customers can use this process to configure a data input for each of their mail processing accounts, using an administrator from the mail processing account itself. However, to collect events from master and / or grouping accounts, contact our Support Team. They will be able to guide you through the process of enabling federated administration in your Mimecast hierarchy, and the process of configuring a data input for these account types.

Using the App

 

The app is comprised of a number of dashboards displaying the available data. See the table below for a description of the dashboards and the expected update frequencies.

 

DashboardDescriptionUpdate FrequencyAvailable DataData Retention
Email Activity > Email Activity SummaryDisplays messages received over time by route and spark lines for Rejections, Bounces, and Held Messages.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Email Activity > Email DeliveryDisplays visualizations for messages delivered by the Mimecast MTA.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Email Activity > Email ReceiptDisplays visualizations for messages received by the Mimecast MTA.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Email Activity > TLSDisplays visualizations detailing the secure delivery and receipt of messages processed by the Mimecast MTA.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Email Activity > AV / ASDisplays visualizations for messages detected as Spam or carrying a virus.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Targeted Threat Protection > Attachment ProtectDisplays visualizations for data logged when a potentially malicious attachment is identified by the Mimecast sandbox. Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Targeted Threat Protection > Impersonation ProtectDisplays visualizations for Targeted Threat Protection Impersonation Protect message characteristics detected in a message.
These visualizations do not necessarily indicate that a policy action was triggered, but that one of the characteristics was detected.
Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Targeted Threat Protection > URL ProtectDisplays visualizations for data logged when a user clicks on a potentially malicious link in an email.Every 30 minutesFrom when you enabled Enhanced Logging.7 days
Audit and Access > Audit LogDisplays a feed of administrator and authentication activity.Real timeFrom ten minutes before you configured the Splunk Data Input.The lifetime of your Mimecast account.
Audit and Access > Access AttemptsDisplays visualizations for access attempts made to the Mimecast account.Real timeFrom ten minutes before you configured the Splunk Data Input.The lifetime of your Mimecast account.
Service HealthA dashboard displaying inbound and outbound message queue totals, as well as the status of Directory and Journal integrations if configured,Real timeFrom one hour before you configured the Splunk Data Input.n/a
TroubleshootMimecast specific events from the splunkd.log file.Real timen/an/a
The current app design does not include the subject field in Journal logs.

Troubleshooting

 

The Modular Input script writes to the splunkd.log file. The Troubleshoot dashboard is the first place to check if you're experiencing any problems or errors. It is provided as part of the Mimecast for Splunk application and filters the splunkd.log for Mimecast specific events. It can also be used to view additional data contained in the Splunkd log. To achieve this you will need to enable the Include platform logs option on the Troubleshoot dashboard. Additionally, the log level can be set for the Mimecast for Splunk app via the Configuration | Logging menu.

 

Dashboards Aren't Updating

 

If your dashboards aren't updating correctly:

  1. Check you have a valid Splunk Data Input configured.
  2. Check the date filters are set to a time period where you're expecting to see data.
  3. Check the log file to confirm events are being added to Splunk.

 

Request Returned with Status Code 418 ()

 

If you see "Request returned with status code 418 ()" in the app logs, the access key and secret key used in your Data Input has expired. Ensure you have followed the configuration steps above, to create a user with an Authentication Token that doesn't expire.

9 people found this helpful

Attachments

    Outcomes