Comprehensive Splunk Series: Part 1 — Installation and Configuration on Linux with Forwarder Agent Setup
Welcome to the Comprehensive Splunk Series! In this multi-part guide, I will take you on a journey through the world of Splunk, a powerful platform for analyzing, monitoring, and visualizing data.
Part 1 of this series will focus on the fundamental steps of starting with Splunk on Linux. We’ll walk you through the installation process and show you how to set up a forwarder agent. With the help of detailed instructions and screenshots, you’ll be able to follow along easily, even if you’re new to Splunk.
In the upcoming parts of the series, we’ll dive deeper into Splunk’s features and capabilities, showcasing how to ingest, search, and visualize data effectively, enabling you to gain valuable insights for your organization.
Prerequisites:
- A Linux Virtual Machine (VM) where you will install Splunk.
- A secondary Linux VM on which you will set up the Splunk Forwarder.
I have chosen Kali Linux as my primary Linux distribution to install Splunk Enterprise. Additionally, I plan to utilize an Ubuntu virtual machine (VM) in the future, both as a forwarder and as a platform for web hosting.
Implementation on a Linux Server
Visit the official website, sign up for an account, and download the required installation package for you.
On your primary Linux instance:
tar -xvf <Installation_package_name>Use the tar command to extract the contents of the package.
I prefer to store all my packages within the /opt/ directory.
Upon extracting the package, you’ll notice a new ‘splunk’ folder, which I’ve relocated to the /opt/ directory.
Once you’ve reached the /opt/splunk/bin/ directory, you can initiate Splunk by executing the following command.
You must set up an administrator username and password during the initial launch. Allow Splunk to perform its operations, and if everything is configured correctly, you should observe the following messages.
You can access the interface by navigating to localhost:8000 in your web browser.
Log in using your provided username and password, and you can explore and appreciate the delightful interface.
Exploring Splunk via the Command Line: An Overview
While the user-friendly web interface is a popular choice for many tasks, Splunk offers an equally robust command-line interface (CLI) that provides a more efficient and scriptable way to interact with the platform.
You can find some fundamental commands outlined here, but for a more extensive list and detailed information, consider utilizing the “splunk help” command to explore further options and capabilities.
Configuring a Universal Forwarder
Universal Forwarders in Splunk are lightweight agents or components responsible for forwarding data from various sources to a central Splunk instance (typically a Splunk Enterprise deployment). These forwarders are crucial in Splunk’s data collection and indexing process.
Access the provided URL and obtain the necessary installation package for your secondary Linux VM, which, in my case, is Ubuntu.
Following the steps you’ve executed previously, utilize the ‘tar’ command, as illustrated, to extract the package. Next, transfer the extracted contents to the /opt/ directory. Finally, commence the instance using the command sudo ./splunk start — accept-license, and during this process, create a username and password as prompted.
Setting Up the Forwarder: Configuration Guide
On your primary Linux machine (Kali), go to “Settings” and then access “Forwarding and Receiving.”
We aim to receive data from the Linux endpoint, click “Configure Receiving,” and then proceed with configuring a new receiving port.
Port 9997 is a common choice for receiving data in Splunk due to its dedicated nature, ease of configuration, and compatibility with Universal Forwarders. It ensures efficient data flow from forwarders to the central Splunk instance, making it a practical choice for data ingestion and analysis.
The crucial next action is establishing an index where all incoming data will be stored. Data will be stored in the default index, the “main index, without specifying an index.”
Access the “Settings” menu and proceed to the “Indexes” section.
Select “New Index,” input your desired Index name, and save your changes. In this instance, I have created an index named “ubuntu_vm.”
Exploring the Splunk Help Command.
Configure the forwarder to ensure it correctly routes and sends data to the intended destination. On the Ubuntu VM, utilize the provided command. Keep in mind that your IP address will differ from mine, so be sure to make the necessary adjustments.
Ingesting Linux Log Sources into Splunk
Linux logs are typically stored in the /var/log/ directory and categorized into various log files or log directories based on different components, services, and subsystems of the operating system.
Utilize the following command to specify the particular log file you wish to monitor.
In Splunk, the inputs.conf file is a configuration file used to define data inputs and how Splunk should collect and index data from various sources. This file is a crucial part of configuring data ingestion in Splunk.
Add the other log files or data sources as needed by creating additional stanzas in the inputs.conf file. Each stanza specifies the data source, source type, and other relevant configurations.
The logger command in Linux adds log messages to the system log (syslog) or other log files. It is a command-line utility that facilitates logging messages for various purposes, including system monitoring, debugging, and information tracking.
If all configurations have been properly set up, you will observe the log data displayed within the Splunk web interface on your primary Linux instance.
Congratulations, you’ve successfully established a Splunk instance on your local hardware and configured a Splunk forwarder.
Additionally, feel free to explore Splunk further and embark on your journey of discovery. Happy learning, and stay tuned for Part Two of our comprehensive Splunk series!
