←Project_Database

ELK Stack

Data Analytics // Log Management // Threat Detection

ElasticsearchLogstashKibana

Overview

The ELK stack is a group of three software programs. People use it to collect and read large amounts of data. You can use it to monitor computer networks and find security threats. The name comes from the three main tools inside the stack.

Elasticsearch

The storage unit

  • Acts as a massive storage unit.
  • Keeps all your data in one central place.
  • Helps you search through millions of records very fast.

Logstash

The delivery worker

  • Acts as the delivery worker.
  • Gathers data from many different computers.
  • Cleans the data and sends it directly to Elasticsearch.

Kibana

The display screen

  • Acts as the display screen.
  • Takes the raw data and turns it into pictures.
  • Build charts and graphs to understand your information easily.

System Implementations

Project One

The Security Alarm System

Mission_Goal
To build a system that acts like a security camera for a computer network.
Execution_Steps
I set up the stack to watch for people typing the wrong password. When someone tries to log in and fails, Logstash grabs that event. It sends the event to Elasticsearch for safe keeping. Finally, I used Kibana to build a live dashboard. This dashboard shows a red warning light and a world map whenever a failed login happens.
System_Impact
This helps stop hackers. If a hacker tries to guess a password one hundred times, my dashboard catches them instantly before they can break in.
Project Two

Digital Data Analytics and Threat Detection

Mission_Goal
To find hidden viruses and bad network traffic by reading computer logs.
Execution_Steps
Computers generate thousands of lines of text logs every minute, and it is impossible for a human to read them all. I connected an Intrusion Detection System to my ELK stack. I captured data packets and sent them into Elasticsearch. I then used Kibana to turn those boring text logs into beautiful bar charts that spike up when suspicious activity occurs.
System_Impact
It takes messy data and makes it visual. A security team can look at my charts and mitigate a cyber threat in seconds instead of spending hours reading text files.
Project Three

The Website Health Monitor

Mission_Goal
To track how well a website is running and fix errors before users notice them.
Execution_Steps
I connected web server records to the ELK stack. Every time a user clicks a broken link on a website, an error code is created. Logstash cleans up these error codes. Kibana then displays a pie chart showing the exact percentage of users experiencing issues versus successful visits.
System_Impact
It keeps businesses running smoothly. Developers can look at the Kibana dashboard and instantly know which part of their website is broken.
SEQUENCE_01

Windows Installation

  1. 01

    Install Elasticsearch

    • Download the Elasticsearch zip file from the official Elastic website.
    • Extract the downloaded zip file into a folder on your main drive.
    • Open that folder and navigate inside the bin folder.
    • Find the file named elasticsearch.bat.
    • Double click that file to run the server.
    • A command prompt window will open and stay open.
    • Look at the text in the window to find the auto-generated password for the elastic user.
    • Save that password in a safe text file.
    • Open your web browser and go to http://localhost:9200 to verify it works.
  2. 02

    Install Kibana

    • Download the Kibana zip file from the official website.
    • Extract the folder onto your computer.
    • Open the config folder inside Kibana.
    • Open the kibana.yml file using Notepad.
    • Find the line for elasticsearch.hosts and ensure it points to http://localhost:9200.
    • Save and close the file.
    • Go to the bin folder inside Kibana.
    • Double click the kibana.bat file.
    • Open your web browser and go to http://localhost:5601 to see the dashboard.
  3. 03

    Install Logstash

    • Download the Logstash zip file and unzip it.
    • Create a new text file inside the config folder named logstash.conf.
    • Add an input block, a filter block, and an output block to this configuration file.
    • Set the output block to point to your localhost Elasticsearch address.
    • Open your command prompt as an administrator.
    • Navigate to the Logstash bin folder.
    • Run the command logstash.bat followed by the path to your config file.
SEQUENCE_02

Ubuntu Installation

  1. 01

    Install Elasticsearch

    • Open your terminal window.
    • Run the update command to refresh your package list.
    • Install the apt-transport-https package to allow secure downloads.
    • Import the Elastic public signing key into your system.
    • Add the Elastic repository definition to your sources list.
    • Run the install command for elasticsearch.
    • Start the service using the systemctl start command.
    • Enable the service to start automatically when the computer boots up.
  2. 02

    Install Kibana

    • Use the apt install command followed by kibana in your terminal.
    • Open the configuration file located at /etc/kibana/kibana.yml.
    • Uncomment the server.port line to allow access.
    • Change the server.host setting to your local IP address or localhost.
    • Save the file and exit the text editor.
    • Start the Kibana service using the systemctl start command.
    • Check the status to ensure the service runs without issues.
  3. 03

    Install Logstash

    • Run the apt install command followed by logstash in your terminal.
    • Create your pipeline configuration file inside the /etc/logstash/conf.d directory.
    • Name your file logstash.conf.
    • Define your data input sources and processing filters inside this file.
    • Point the output section to your secure Elasticsearch node.
    • Start the Logstash service using the systemctl start tool.
    • Your pipeline will now process data and send it to Kibana automatically.
←Project Archive