elk-stack
Deploy and manage the ELK Stack (Elasticsearch, Logstash, Kibana) for log aggregation and analysis. Configure log pipelines, create visualizations, and implement log-based monitoring
Install / Use
npx skills add BagelHole/DevOps-Security-Agent-Skills --skill elk-stackInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
AutomationSupported Platforms
Tags
Our assessment of elk-stack
elk-stack scores 87/100 on our quality scale, 1698th of 2,790 Automation skills we index.
Its SKILL.md is 9.1 KB long, well organised into 31 sections with 9 code examples: a thorough specification that gives an agent plenty to work with.
With 1,113 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 6 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
- Our last check on 2026-10-11 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 98/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-07. Automated pattern scan on 2026-10-07. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
elk-stack compared with similar skills
All 4 of these similar skills score higher than elk-stack; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| elk-stack (this skill)by BagelHole | 87 | 1.1k | 6mo ago | SKILL.md |
| Agent-Reachby Panniantong | 100 | 112.3k | 8d ago | CLAUDE.md |
| rufloby ruvnet | 100 | 81.9k | today | CLAUDE.md |
| Claude-Code-Game-Studiosby Donchitos | 100 | 39.7k | 7d ago | CLAUDE.md |
| Scraplingby D4Vinci | 100 | 103.6k | 2d ago | MCP Server |
Frequently asked questions
- How do I install elk-stack?
- Run
npx skills add BagelHole/DevOps-Security-Agent-Skills --skill elk-stack. The install tabs above show the steps for each supported agent. - Which AI agents does elk-stack work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is elk-stack safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 98/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is elk-stack still maintained?
- The repository was last updated about 6 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
Skill content
View source on GitHubname: elk-stack description: Deploy and manage the ELK Stack (Elasticsearch, Logstash, Kibana) for log aggregation and analysis. Configure log pipelines, create visualizations, and implement log-based monitoring. Use when centralizing logs, implementing search functionality, or building log analytics platforms. license: MIT metadata: author: devops-skills version: "1.0"
ELK Stack
Centralize and analyze logs with Elasticsearch, Logstash, and Kibana.
When to Use This Skill
Use this skill when:
- Centralizing logs from multiple sources
- Building log search and analytics platforms
- Creating log-based dashboards and alerts
- Implementing full-text search for logs
- Processing and transforming log data
Prerequisites
- Docker or server infrastructure
- Sufficient disk space for log storage
- Network access from log sources
Docker Deployment
# docker-compose.yml
version: '3.8'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
environment:
- discovery.type=single-node
- xpack.security.enabled=false
- "ES_JAVA_OPTS=-Xms1g -Xmx1g"
ports:
- "9200:9200"
volumes:
- elasticsearch-data:/usr/share/elasticsearch/data
logstash:
image: docker.elastic.co/logstash/logstash:8.11.0
volumes:
- ./logstash/pipeline:/usr/share/logstash/pipeline
- ./logstash/config:/usr/share/logstash/config
ports:
- "5044:5044"
- "5000:5000"
depends_on:
- elasticsearch
kibana:
image: docker.elastic.co/kibana/kibana:8.11.0
ports:
- "5601:5601"
environment:
- ELASTICSEARCH_HOSTS=http://elasticsearch:9200
depends_on:
- elasticsearch
filebeat:
image: docker.elastic.co/beats/filebeat:8.11.0
user: root
volumes:
- ./filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
- /var/lib/docker/containers:/var/lib/docker/containers:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
depends_on:
- logstash
volumes:
elasticsearch-data:
Elasticsearch Configuration
Index Templates
PUT _index_template/logs-template
{
"index_patterns": ["logs-*"],
"template": {
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1,
"index.lifecycle.name": "logs-policy"
},
"mappings": {
"properties": {
"@timestamp": { "type": "date" },
"message": { "type": "text" },
"level": { "type": "keyword" },
"service": { "type": "keyword" },
"host": { "type": "keyword" },
"trace_id": { "type": "keyword" }
}
}
}
}
Index Lifecycle Management
PUT _ilm/policy/logs-policy
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"rollover": {
"max_size": "50GB",
"max_age": "1d"
}
}
},
"warm": {
"min_age": "7d",
"actions": {
"shrink": { "number_of_shards": 1 },
"forcemerge": { "max_num_segments": 1 }
}
},
"cold": {
"min_age": "30d",
"actions": {
"freeze": {}
}
},
"delete": {
"min_age": "90d",
"actions": {
"delete": {}
}
}
}
}
}
Logstash Pipeline
Basic Pipeline
# logstash/pipeline/main.conf
input {
beats {
port => 5044
}
tcp {
port => 5000
codec => json_lines
}
}
filter {
# Parse JSON logs
if [message] =~ /^\{/ {
json {
source => "message"
}
}
# Parse timestamp
date {
match => ["timestamp", "ISO8601", "yyyy-MM-dd HH:mm:ss"]
target => "@timestamp"
}
# Add environment tag
mutate {
add_field => { "environment" => "production" }
}
# Grok pattern for nginx logs
if [type] == "nginx" {
grok {
match => {
"message" => '%{IPORHOST:client_ip} - %{USER:user} \[%{HTTPDATE:timestamp}\] "%{WORD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:http_version}" %{NUMBER:status} %{NUMBER:bytes}'
}
}
}
}
output {
elasticsearch {
hosts => ["elasticsearch:9200"]
index => "logs-%{+YYYY.MM.dd}"
}
}
Advanced Filtering
filter {
# Parse application logs
grok {
match => {
"message" => "%{TIMESTAMP_ISO8601:timestamp} %{LOGLEVEL:level} \[%{DATA:service}\] %{GREEDYDATA:log_message}"
}
}
# Extract trace ID from message
if [log_message] =~ /trace_id=/ {
grok {
match => { "log_message" => "trace_id=%{UUID:trace_id}" }
}
}
# GeoIP lookup
if [client_ip] {
geoip {
source => "client_ip"
target => "geoip"
}
}
# Drop debug logs in production
if [level] == "DEBUG" and [environment] == "production" {
drop {}
}
# Enrich with lookup
translate {
field => "status"
destination => "status_description"
dictionary => {
"200" => "OK"
"404" => "Not Found"
"500" => "Internal Server Error"
}
}
}
Filebeat Configuration
# filebeat/filebeat.yml
filebeat.inputs:
- type: container
paths:
- '/var/lib/docker/containers/*/*.log'
processors:
- add_docker_metadata:
host: "unix:///var/run/docker.sock"
- type: log
enabled: true
paths:
- /var/log/nginx/*.log
tags: ["nginx"]
fields:
type: nginx
output.logstash:
hosts: ["logstash:5044"]
logging.level: info
logging.to_files: true
logging.files:
path: /var/log/filebeat
name: filebeat
keepfiles: 7
Elasticsearch Queries
Basic Queries
// Search all logs
GET logs-*/_search
{
"query": {
"match_all": {}
}
}
// Search by keyword
GET logs-*/_search
{
"query": {
"match": {
"message": "error"
}
}
}
// Filter by field
GET logs-*/_search
{
"query": {
"bool": {
"must": [
{ "match": { "level": "ERROR" } },
{ "range": { "@timestamp": { "gte": "now-1h" } } }
],
"filter": [
{ "term": { "service": "api-gateway" } }
]
}
}
}
Aggregations
// Count by log level
GET logs-*/_search
{
"size": 0,
"aggs": {
"log_levels": {
"terms": { "field": "level" }
}
}
}
// Error rate over time
GET logs-*/_search
{
"size": 0,
"aggs": {
"errors_over_time": {
"date_histogram": {
"field": "@timestamp",
"fixed_interval": "5m"
},
"aggs": {
"error_count": {
"filter": { "term": { "level": "ERROR" } }
}
}
}
}
}
Kibana Setup
Index Patterns
- Go to Stack Management → Index Patterns
- Create pattern:
logs-* - Set time field:
@timestamp
Saved Searches
Create saved searches for common queries:
level:ERROR- All errorsservice:api-gateway AND level:ERROR- API gateway errorsresponse_time:>1000- Slow requests
Visualizations
Common visualization types:
- Line Chart: Error rate over time
- Pie Chart: Distribution by log level
- Data Table: Top error messages
- Metric: Total error count
Dashboard Example
Create dashboard with:
- Total log count (Metric)
- Error rate trend (Line chart)
- Logs by service (Pie chart)
- Recent errors (Data table)
- Log stream (Discover panel)
Alerting
Watcher (X-Pack)
PUT _watcher/watch/error_alert
{
"trigger": {
"schedule": { "interval": "5m" }
},
"input": {
"search": {
"request": {
"indices": ["logs-*"],
"body": {
"query": {
"bool": {
"must": [
{ "match": { "level": "ERROR" } },
{ "range": { "@timestamp": { "gte": "now-5m" } } }
]
}
}
}
}
}
},
"condition": {
"compare": { "ctx.payload.hits.total.value": { "gt": 100 } }
},
"actions": {
"notify_slack": {
"webhook": {
"scheme": "https",
"host": "hooks.slack.com",
"port": 443,
"method": "post",
"path": "/services/xxx",
"body": "{\"text\": \"High error rate detected: {{ctx.payload.hits.total.value}} errors in last 5 minutes\"}"
}
}
}
}
Common Issues
Issue: High Disk Usage
Problem: Elasticsearch consuming too much disk Solution: Implement ILM policies, reduce retention
Issue: Slow Searches
Problem: Queries taking too long Solution: Optimize index settings, add more shards, use filters
Issue: Log Parsing Failures
Problem: Logs not parsed correctly Solution: Test grok patterns, check for log format changes
Issue: Memory Pressure
Problem: Elasticsearch OOM errors Solution: Increase heap size (max 50% of RAM), limit field data
Best Practices
- Implement index lifecycle management
- Use index templates for consistent mappings
- Parse logs at ingestion time
- Limit stored fields to reduce storage
- Use data streams for time-series data
- Monitor cluster health
- Implement proper security (X-Pack)
- Regular index maintenance
Related Skills
- loki-logging - Alternative logging stack
- prometheus-grafana - Metrics monitoring
- audit-logging - Compliance logging
Related Skills
Agent-Reach
112.3kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
ruflo
81.9k🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
Claude-Code-Game-Studios
39.7kTurn Claude Code into a full game dev studio — 49 AI agents, 72 workflow skills, and a complete coordination system mirroring real studio hierarchy.
Scrapling
103.6k🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
