Pushing Mule Application Logs to Splunk

Pushing Mule Application Logs to Splunk
Added 2 months ago
Summary: MuleSoft provides its logging mechanism for storing application logs. In this article, we will discuss the typical applications of MuleSoft Logs in Splunk.
Sentiment: positive
4 minute read
9 people have read this
Stocks: $SPLK
Cryptos: $RAISE

Join the DZone community and get the full member experience.

Join For Free

Introduction

Logging is an essential part of monitoring and troubleshooting issues and any production errors or visualizing the data. Logging must be consistent and reliable so we can use that information for discovering relevant data. Some external logging tools, including ELK and Splunk
MuleSoft, provide its logging mechanism for storing application logs. Although CloudHub has a limitation of 100 MB of logs or 30 days of logs. The blog typically talks MuleSoft Splunk Integration.

For a robust logging mechanism, it is essential to have an external log analytic tool to further monitor the application. Today we will be using Splunk as an external logging tool and integrating it with MuleSoft using Log4j2 HTTP appender to send mule application logs to Splunk. Logging to Splunk can be enabled on Cloud Hub and On-Premise.

First things first, we need to create a token in Splunk.

1. Go to Settings > Data > Data Inputs
Settings > Data > Data Inputs

2. Go to Settings > Data > Data Inputs > New Token
> New Token

3. After Clicking on New Token, Click on HTTP Event Collector and add log4j as the source, since we will be sending logs from log4j to Splunk
Add Log4j as a source

4. Complete all the steps, and you will get the token value. The token value will be used to connect to Splunk from the log4j file in the MuleSoft application. The next steps will involve configuring the HTTP Appender in the log4j file to connect to Splunk.
Configuring the HTTP Appender

5. Once you have created the token, make sure to enable the token by going to global settings. You can also enable SSL for this token and set the port. By default, the value is “8088”
Edit global settings

6. Add the following snippet in the log4j2.xml in the mule application. We can also add SSL if the URL is HTTPS.

7. For better log analysis and monitoring, it is recommended to use JSON logs. For this purpose, we can either use JSON logs or add log information in JSON format. Here is a snippet for the application that will be sending logs to Splunk.
Sending logs to Splunk

8. Once you start the application, you can see logs flowing to Splunk. To check that, click on the “Search and Monitoring” option.
Search and monitoring option

9. After that, click on “Data Summary” and click on “Source Types” and search for Log4j (see Step 3), and select log4j.
Data summary

10. On selecting that, you can see the logs being pushed to Splunk
Logs being pushed to Splunk

Pushing MuleSoft Anypoint Platform Logs to Splunk

Sending MuleSoft Anypoint Platform (CloudHub) logs will require a slightly different process. CloudHub uses its default logging mechanism. To use our logging, we need to make specific changes to the log4j file so that we can override the default log4j configuration for CloudHub. Below are the few steps that need to be followed.

1. Raise a support ticket with MuleSoft to disable CloudHub application logs. Once that is done, you will have an option to disable logs “Disable Application Logs” at runtime while deploying the application
Disable CloudHub Application Logins

2. The next step is to add CloudHub log appenders to the log4j2.xml file in the mule application.
Add Cloudhub Appenders
An example log4j2 file with a custom cloudhub appender.

3. Once you deploy the application to MuleSoft Anypoint Platform CloudHub and disable CloudHub logs, it will use the log4j2 configuration which we have created.

Feel free to drop your questions below!

Topics:

mulesoft, loggging, anypoint platform, splunk