gpu-server-management
Set up and manage NVIDIA GPU servers for AI workloads — driver installation, CUDA toolkit, container toolkit, MIG partitioning, GPU health monitoring, and multi-GPU configuration for LLM inference and training.
Install / Use
npx skills add BagelHole/DevOps-Security-Agent-Skills --skill gpu-server-managementInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
AI & Machine LearningSupported Platforms
Tags
Our assessment of gpu-server-management
gpu-server-management scores 87/100 on our quality scale, 512th of 950 AI & Machine Learning skills we index.
Its SKILL.md is 7.3 KB long, well organised into 56 sections with 7 code examples: a thorough specification that gives an agent plenty to work with.
With 1,113 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 6 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
- Our last check on 2026-11-02 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 98/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-07. Automated pattern scan on 2026-10-07. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
gpu-server-management compared with similar skills
All 4 of these similar skills score higher than gpu-server-management; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| gpu-server-management (this skill)by BagelHole | 87 | 1.1k | 6mo ago | SKILL.md |
| claude-memby thedotmack | 100 | 117.6k | today | CLAUDE.md |
| Understand-Anythingby Egonex-AI | 100 | 92.8k | 1d ago | CLAUDE.md |
| headroomby headroomlabs-ai | 100 | 79.8k | today | CLAUDE.md |
| CowAgentby zhayujie | 100 | 48.0k | today | CLAUDE.md |
Frequently asked questions
- How do I install gpu-server-management?
- Run
npx skills add BagelHole/DevOps-Security-Agent-Skills --skill gpu-server-management. The install tabs above show the steps for each supported agent. - Which AI agents does gpu-server-management work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is gpu-server-management safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 98/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is gpu-server-management still maintained?
- The repository was last updated about 6 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
Skill content
View source on GitHubname: gpu-server-management description: Set up and manage NVIDIA GPU servers for AI workloads — driver installation, CUDA toolkit, container toolkit, MIG partitioning, GPU health monitoring, and multi-GPU configuration for LLM inference and training. license: MIT metadata: author: devops-skills version: "1.0"
GPU Server Management
Provision, configure, and monitor NVIDIA GPU servers for AI inference and training workloads.
When to Use This Skill
Use this skill when:
- Setting up a new GPU server for LLM inference or model training
- Installing or upgrading NVIDIA drivers and CUDA toolkit
- Configuring Docker with NVIDIA Container Toolkit for GPU workloads
- Partitioning A100/H100 GPUs with MIG for multi-tenant workloads
- Troubleshooting GPU errors, driver issues, or thermal throttling
Prerequisites
- Ubuntu 22.04 LTS (recommended) or RHEL 8/9
- NVIDIA GPU (A10G, A100, H100, RTX 4090, or L40S recommended)
- Root or sudo access
- Internet access for package downloads
Driver Installation (Ubuntu)
# Remove old drivers
sudo apt purge -y 'nvidia*' 'cuda*' 'libcuda*'
sudo apt autoremove -y
# Add NVIDIA package repository
distribution=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \
sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
# Install latest driver (560.x as of 2025)
sudo apt install -y nvidia-driver-560 cuda-toolkit-12-6
# Install NVIDIA Container Toolkit (Docker GPU support)
sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
# Verify
nvidia-smi
nvcc --version
docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu22.04 nvidia-smi
Post-Install Configuration
# Enable persistence mode (reduces driver initialization latency)
sudo nvidia-smi -pm 1
# Set power limit (reduce heat/noise on inference servers)
sudo nvidia-smi -pl 350 # watts; check TDP for your GPU model
# Disable ECC on inference servers (frees ~6% VRAM, less safe)
sudo nvidia-smi --ecc-config=0 # requires reboot
# Enable P2P for multi-GPU NVLink training
sudo nvidia-smi topo -m # check NVLink topology
GPU Health Monitoring
# Real-time monitoring (like htop for GPUs)
watch -n 1 nvidia-smi
# Detailed stats
nvidia-smi --query-gpu=index,name,temperature.gpu,utilization.gpu,\
utilization.memory,memory.used,memory.free,power.draw,clocks.current.graphics \
--format=csv --loop=1
# DCGM — production monitoring daemon (for clusters)
sudo apt install -y datacenter-gpu-manager
sudo systemctl start dcgm
dcgmi discovery -l # list GPUs
dcgmi diag -r 1 # quick health check
dcgmi diag -r 3 # full diagnostic (takes ~20 min)
# Check GPU errors (XID errors — important for stability)
sudo dmesg | grep -i "NVRM\|nvidia\|XID"
nvidia-smi --query-gpu=ecc.errors.corrected.volatile.total \
--format=csv,noheader
Prometheus GPU Metrics (DCGM Exporter)
# Deploy DCGM Exporter for Prometheus scraping
docker run -d \
--name dcgm-exporter \
--gpus all \
--cap-add SYS_ADMIN \
-p 9400:9400 \
--restart unless-stopped \
nvcr.io/nvidia/k8s/dcgm-exporter:latest
# Key metrics exposed:
# DCGM_FI_DEV_GPU_UTIL - GPU utilization %
# DCGM_FI_DEV_MEM_COPY_UTIL - Memory bandwidth utilization
# DCGM_FI_DEV_FB_USED - Framebuffer memory used (MB)
# DCGM_FI_DEV_SM_CLOCK - SM clock speed (MHz)
# DCGM_FI_DEV_GPU_TEMP - Temperature (°C)
# DCGM_FI_DEV_POWER_USAGE - Power draw (W)
# DCGM_FI_DEV_XID_ERRORS - XID error count (0 = healthy)
MIG Partitioning (A100/H100)
MIG (Multi-Instance GPU) allows slicing one GPU into isolated smaller GPUs.
# Enable MIG mode (requires reboot or restart of all processes)
sudo nvidia-smi -mig 1
sudo systemctl restart nvidia-persistenced
# List available MIG profiles (A100 80GB example)
nvidia-smi mig -lgip
# 1g.10gb — 1 slice, 10GB (max 7 instances)
# 2g.20gb — 2 slices, 20GB (max 3 instances)
# 3g.40gb — 3 slices, 40GB (max 2 instances)
# 7g.80gb — full GPU, 80GB (max 1 instance)
# Create MIG instances (e.g., 3× 2g.20gb + 1× 2g.20gb = multi-tenant)
sudo nvidia-smi mig -cgi 2g.20gb,2g.20gb,2g.20gb,2g.20gb -C
# List created instances
nvidia-smi mig -lgi
nvidia-smi mig -lcgi
# Use in Docker
docker run --gpus '"device=MIG-GPU-xxx/0/0"' ...
# Disable MIG
sudo nvidia-smi mig -i 0 -dci
sudo nvidia-smi mig -i 0 -dgi
sudo nvidia-smi -mig 0
Kernel & OS Tuning for GPU Servers
# Increase file descriptor limits
echo '* soft nofile 1048576' | sudo tee -a /etc/security/limits.conf
echo '* hard nofile 1048576' | sudo tee -a /etc/security/limits.conf
# Disable transparent huge pages (reduces latency jitter)
echo never | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
echo never | sudo tee /sys/kernel/mm/transparent_hugepage/defrag
# Persist via rc.local or systemd unit:
cat <<'EOF' | sudo tee /etc/rc.local
#!/bin/bash
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
nvidia-smi -pm 1
exit 0
EOF
sudo chmod +x /etc/rc.local
# PCIe performance mode
sudo nvidia-smi --auto-boost-default=0
sudo nvidia-smi --auto-boost-permission=0
Multi-GPU Topology Check
# Check NVLink and PCIe topology
nvidia-smi topo -m
# Output shows interconnect type:
# NV4 = NVLink 4.0 (H100 SXM)
# NV2 = NVLink 2.0 (A100 SXM)
# PHB = PCIe bus (slower; avoid for tensor parallel training)
# PIX = same PCIe switch (fast)
# Bandwidth test between GPUs
/usr/local/cuda/samples/bin/x86_64/linux/release/p2pBandwidthLatencyTest
Common Issues
| Issue | Cause | Fix |
|-------|-------|-----|
| nvidia-smi: command not found | Driver not installed | Follow driver installation steps above |
| Driver version mismatch | CUDA/driver incompatibility | Check compatibility matrix at developer.nvidia.com |
| GPU temperature >85°C | Poor airflow or fan failure | Check nvidia-smi -q -d TEMPERATURE; reseat cooler |
| XID 79 errors | GPU hardware error | Run dcgmi diag -r 3; may need GPU replacement |
| failed to open device in container | Container toolkit not configured | Run nvidia-ctk runtime configure --runtime=docker |
| Low PCIe bandwidth | Wrong slot or power limit | Check nvidia-smi -q | grep PCIe; use x16 slot |
Best Practices
- Always enable persistence mode (
nvidia-smi -pm 1) — reduces first-request latency. - Monitor XID errors; persistent XID 79/94 indicates hardware failure.
- For training: use NVLink-connected GPUs; for inference: PCIe is usually fine.
- Set up DCGM alerts on temperature >80°C and power draw near TDP.
- Use MIG for multi-tenant inference to provide GPU isolation between models.
Related Skills
- vllm-server - LLM inference on GPUs
- llm-fine-tuning - GPU training setup
- linux-hardening - Secure the host OS
- prometheus-grafana - Metrics dashboards
Related Skills
claude-mem
117.6kPersistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + More
Understand-Anything
92.8kGraphs that teach > graphs that impress. Turn any code into an interactive knowledge graph you can explore, search, and ask questions about. Works with Claude Code, Codex, Cursor, Copilot, Gemini CLI, and more.
headroom
79.8kCompress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
CowAgent
48.0kOpen-source personal AI assistant & Agent Harness. Plans tasks, runs tools and skills, self-evolves with memory and knowledge. Multi-agent, multi-model, multi-channel. Lightweight, extensible, one-line install.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
