What is ELK and Installing ELK stack (elasticsearch, logstash, kibana) in Ubuntu

  

What is ELK Stack

The ELK stack is a collection of three open-source tools — Elasticsearch, Logstash, and Kibana — that work together to collect, store, and analyze data. Here is how each tool works:

  1. Elasticsearch: Elasticsearch is a search and analytics engine that provides a distributed, real-time search and analytics platform. It stores data in a distributed index, which allows for fast, real-time search and analysis of data. Elasticsearch is highly scalable, and it can handle a large volume of data with ease.
  2. Logstash: Logstash is a data pipeline that collects, filters, and transforms data from different sources before sending it to Elasticsearch. It can collect data from different sources such as log files, databases, and message queues. Logstash can also perform data transformation and filtering to ensure that only the relevant data is sent to Elasticsearch.
  3. Kibana: Kibana is a data visualization tool that allows users to create dashboards, visualizations, and reports based on data stored in Elasticsearch. Kibana provides a web interface that allows users to interact with the data and visualize it in various formats such as tables, graphs, and maps.

When used together, Elasticsearch, Logstash, and Kibana form a complete data analysis solution. Logstash collects data from different sources and filters and transforms it before sending it to Elasticsearch for storage.

Elasticsearch stores the data and provides fast, real-time search and analysis capabilities. Kibana provides a user-friendly interface to visualize and analyze the data stored in Elasticsearch.

Overall, the ELK stack is a powerful tool for collecting, storing, and analyzing data in real-time. It can be used for various use cases such as log analysis, security analysis, and business intelligence.

Installation and Configuration of ELK Stack on Ubuntu

Note: In the process of installation you can supply ubuntu machine IP in place of localhost or else you can also use localhost.

Install Java environment packages by using the below command

sudo su
apt install default-jdk default-jre -y

Add the elasticsearch APT repository key by using the below command

curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

Add the elastic to the APT source list by using the below command

echo “deb https://artifacts.elastic.co/packages/7.x/apt stable main” > /etc/apt/sources.list.d/elastic-7.x.list

Update the APT source list by using the below command

apt update

Install the Elastic Search by using the below command

apt install elasticsearch -y

Configure the elasticsearch by using the below command

nano /etc/elasticsearch/elasticsearch.yml

Change the network.host and http.port as per the screenshot(network.host is the IP of the Ubuntu machine)

Add “discovery.type: single-node”

Configure the JVM heap memory by using the below command

vim /etc/elasticsearch/jvm.options

Restart and enable elasticsearch.

systemctl restart elasticsearch
systemctl enable elasticsearch

To verify the working of elasticsearch use curl command as given below

curl -X GET “<IP>:9200”

Install the Logstash using apt

apt install logstash

Start and enable Logstash services

systemctl start logstash
systemctl enable logstash

Check the status of the Logstash Service

systemctl status logstash

Now install Kibana

apt install kibana

Configure kibana.yml file in /etc/kibana

nano /etc/kibana/kibana.yml

Start and enable kibana service

systemctl start kibana
systemctl enable kibana

Check the status of the kibana service

systemctl status kibana

Ping the http://<IP>:5601 or http://localhost:5601 in browser to view the Dashboard of the kibana as show in the below image.

You have successfully installed ELK stack in Ubuntu. In the upcoming blog I will share how to configure xpack security to enable minimal security.

You can find the installation video below:

Subscribe to our YouTube channel. https://www.youtube.com/@CyberToolGuardian/featured

Follow us on Instagram.
https://instagram.com/cybertoolguardian

Comments

Post a Comment

Popular posts from this blog

Zeek Installation in Ubuntu

Image
What is Zeek? An open-source protocol analyzer and network security monitoring tool, Zeek was once known as Bro. It is intended to assist enterprises with real-time network traffic monitoring and analysis, offering information on network activity, potential security risks, and performance concerns. Due to its effectiveness in swiftly capturing and processing network data, Zeek is especially well-liked among cybersecurity experts and network managers. How to install Zeek in Ubuntu? Update and upgrade the Ubuntu using apt. sudo apt-get update sudo apt-get upgrade Download the Zeek source code from the official website ( https://zeek.org/get-zeek/ ). Zeek offical download page Install dependencies using the below command. sudo apt-get install cmake make gcc g++ flex bison libpcap-dev libssl-dev python3-dev swig zlib1g-dev Dependencies installation Once all the dependencies are installed change the directory to the path where the Zeek source code file is downloaded and unzip the file. cd D...
Read more

Sending Zeek logs to ELK using Filebeats

Image
Before sending logs we must modify local.zeek file and add the below line at the end of the file. @load policy/tuning/json-logs.zeek Use the find command to find local.zeek file. find / -name local.zeek local.zeek path Now copy the path and edit the file using nano and add the line. local.zeek Save and exit from editor, now check the configuration of zeekctl sudo zeekctl check zeekctl check Once the you get zeek scripts are ok, deploy zeekctl sudo zeekctl deploy zeekctl deploy Go to the zeek logs path directory which is /usr/local/zeek/logs/current. Check if logs are generated properly. logs Go to Kibana dashboard head over to integration section and search for zeek, in the integration page on bottom right go to “Also available in Beats”. You will be redirected to Filebeats Zeek integration page. Copy the commands from the Step1 and open new terminal window and run the commands. Which will download and extract the filebeats. Now edit the filebeat.yml file, change the paths to the zeek ...
Read more