llmops-platform-engineering
Build production LLMOps platforms with CI/CD, model promotion workflows, evaluation gates, rollback, and governance across cloud and self-hosted inference.
Install / Use
npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llmops-platform-engineeringInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
AutomationSupported Platforms
Our assessment of llmops-platform-engineering
llmops-platform-engineering scores 88/100 on our quality scale, 1554th of 2,790 Automation skills we index.
Its SKILL.md is 12 KB long, well organised into 20 sections with 4 code examples: a thorough specification that gives an agent plenty to work with.
With 1,113 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 6 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
- Our last check on 2026-11-01 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 98/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-08. Automated pattern scan on 2026-10-08. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
llmops-platform-engineering compared with similar skills
All 4 of these similar skills score higher than llmops-platform-engineering; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| llmops-platform-engineering (this skill)by BagelHole | 88 | 1.1k | 6mo ago | SKILL.md |
| taste-skillby Leonxlnx | 100 | 116.8k | 8d ago | CLAUDE.md |
| Agent-Reachby Panniantong | 100 | 112.3k | 8d ago | CLAUDE.md |
| rufloby ruvnet | 100 | 81.9k | today | CLAUDE.md |
| headroomby headroomlabs-ai | 100 | 79.8k | today | CLAUDE.md |
Frequently asked questions
- How do I install llmops-platform-engineering?
- Run
npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llmops-platform-engineering. The install tabs above show the steps for each supported agent. - Which AI agents does llmops-platform-engineering work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is llmops-platform-engineering safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 98/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is llmops-platform-engineering still maintained?
- The repository was last updated about 6 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
Skill content
View source on GitHubname: llmops-platform-engineering description: Build production LLMOps platforms with CI/CD, model promotion workflows, evaluation gates, rollback, and governance across cloud and self-hosted inference. license: MIT metadata: author: devops-skills version: "1.0"
LLMOps Platform Engineering
Design and operate an internal LLM platform that supports rapid experimentation without compromising reliability, cost, or compliance.
When to Use This Skill
- Building an internal platform for teams to deploy and manage LLM-powered features
- Designing CI/CD pipelines that include model evaluation gates
- Setting up A/B testing infrastructure for model versions
- Creating Kubernetes-based model serving infrastructure
- Establishing governance workflows for model promotion
Prerequisites
- Kubernetes cluster with GPU node pools (or cloud inference API access)
- Container registry (Harbor, ECR, GCR, or ACR)
- CI/CD system (GitHub Actions, GitLab CI, or Argo Workflows)
- Observability stack (Prometheus + Grafana + OpenTelemetry)
- Model registry (MLflow or custom metadata store)
Outcomes
- Standardized path from experiment to production
- Safe model rollout with quality and safety gates
- Repeatable infra modules for inference, vector DB, and observability
- Clear ownership model across platform, app, and security teams
Reference Architecture
- Control Plane: model registry, prompt/version catalog, policy checks, eval pipeline.
- Data Plane: inference gateway, vector database, cache, feature store.
- Ops Plane: telemetry, alerting, SLO dashboards, cost analytics.
- Security Plane: IAM boundaries, secret rotation, content filters, audit logs.
Model Promotion Pipeline
# .github/workflows/model-promotion.yaml
name: Model Promotion Pipeline
on:
workflow_dispatch:
inputs:
model_name:
description: "Model identifier"
required: true
model_version:
description: "Model version to promote"
required: true
target_env:
description: "Target environment"
required: true
type: choice
options: [staging, production]
jobs:
evaluate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run quality evaluation suite
run: |
python -m evals.run \
--model "${{ inputs.model_name }}:${{ inputs.model_version }}" \
--suite quality \
--output results/quality.json
- name: Run safety evaluation suite
run: |
python -m evals.run \
--model "${{ inputs.model_name }}:${{ inputs.model_version }}" \
--suite safety \
--output results/safety.json
- name: Run latency benchmark
run: |
python -m evals.benchmark \
--model "${{ inputs.model_name }}:${{ inputs.model_version }}" \
--concurrent-users 50 \
--duration 300 \
--output results/latency.json
- name: Gate check - quality
run: |
python -m evals.gate_check \
--results results/quality.json \
--threshold-file thresholds/quality.yaml
- name: Gate check - safety
run: |
python -m evals.gate_check \
--results results/safety.json \
--threshold-file thresholds/safety.yaml
- name: Gate check - latency
run: |
python -m evals.gate_check \
--results results/latency.json \
--threshold-file thresholds/latency.yaml
- name: Upload eval evidence
uses: actions/upload-artifact@v4
with:
name: eval-results-${{ inputs.model_version }}
path: results/
approve:
needs: evaluate
runs-on: ubuntu-latest
environment: ${{ inputs.target_env }}
steps:
- name: Record approval
run: |
echo "Approved by: ${{ github.actor }}"
echo "Model: ${{ inputs.model_name }}:${{ inputs.model_version }}"
echo "Target: ${{ inputs.target_env }}"
echo "Time: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
deploy:
needs: approve
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Deploy canary
run: |
kubectl set image deployment/${{ inputs.model_name }}-canary \
model=${{ inputs.model_name }}:${{ inputs.model_version }} \
-n ai-${{ inputs.target_env }}
- name: Wait for canary validation (15 min)
run: |
python -m canary.validate \
--deployment ${{ inputs.model_name }}-canary \
--namespace ai-${{ inputs.target_env }} \
--duration 900 \
--quality-threshold 0.85 \
--error-rate-threshold 0.02
- name: Promote to full rollout
run: |
kubectl set image deployment/${{ inputs.model_name }} \
model=${{ inputs.model_name }}:${{ inputs.model_version }} \
-n ai-${{ inputs.target_env }}
kubectl rollout status deployment/${{ inputs.model_name }} \
-n ai-${{ inputs.target_env }} --timeout=300s
Evaluation Gate Thresholds
# thresholds/quality.yaml
gates:
groundedness:
metric: groundedness_score
min: 0.85
comparison: gte
task_success:
metric: task_success_rate
min: 0.90
comparison: gte
hallucination:
metric: hallucination_rate
max: 0.08
comparison: lte
regression:
metric: quality_delta_vs_baseline
min: -0.02
comparison: gte
description: "Must not regress more than 2% vs current production"
# thresholds/latency.yaml
gates:
p50_latency:
metric: latency_p50_ms
max: 800
comparison: lte
p95_latency:
metric: latency_p95_ms
max: 2000
comparison: lte
p99_latency:
metric: latency_p99_ms
max: 5000
comparison: lte
throughput:
metric: requests_per_second
min: 50
comparison: gte
A/B Testing Configuration
# ab-test-config.yaml
apiVersion: gateway.ai/v1
kind: ABTest
metadata:
name: model-comparison-q1
namespace: ai-production
spec:
duration: 7d
traffic_split:
control:
model: gpt-4o-2024-08-06
weight: 70
treatment:
model: gpt-4o-2025-01-15
weight: 30
metrics:
primary:
- task_success_rate
- user_satisfaction_score
secondary:
- latency_p95
- cost_per_request
- hallucination_rate
guardrails:
auto_rollback_if:
- metric: task_success_rate
threshold: 0.80
window: 1h
- metric: hallucination_rate
threshold: 0.15
window: 30m
assignment:
strategy: sticky_user
hash_key: user_id
Kubernetes Model Serving Deployment
# model-serving-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: llm-inference
namespace: ai-production
labels:
app: llm-inference
model: gpt-4o
version: "2025-01"
spec:
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
selector:
matchLabels:
app: llm-inference
template:
metadata:
labels:
app: llm-inference
model: gpt-4o
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "8080"
prometheus.io/path: "/metrics"
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: llm-inference
containers:
- name: model
image: registry.internal/vllm-server:0.4.1
args:
- "--model=/models/current"
- "--tensor-parallel-size=1"
- "--max-model-len=8192"
- "--gpu-memory-utilization=0.90"
ports:
- containerPort: 8000
name: inference
- containerPort: 8080
name: metrics
resources:
requests:
cpu: "4"
memory: "16Gi"
nvidia.com/gpu: "1"
limits:
cpu: "8"
memory: "32Gi"
nvidia.com/gpu: "1"
readinessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 60
periodSeconds: 10
livenessProbe:
httpGet:
path: /health
port: 8000
initialDelaySeconds: 120
periodSeconds: 30
volumeMounts:
- name: model-weights
mountPath: /models
readOnly: true
- name: config
mountPath: /etc/vllm
volumes:
- name: model-weights
persistentVolumeClaim:
claimName: model-weights-pvc
- name: config
configMap:
name: vllm-config
tolerations:
- key: nvidia.com/gpu
operator: Exists
effect: NoSchedule
nodeSelector:
gpu-type: a100
---
apiVersion: v1
kind: Service
metadata:
name: llm-inference
namespace: ai-production
spec:
selector:
app: llm-inference
ports:
- name: inference
port: 8000
targetPort: 8000
- name: metrics
port: 8080
targetPort: 8080
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: llm-inference-hpa
namespace: ai-production
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: llm-inference
minReplicas: 2
maxReplicas: 10
metrics:
- type: Pods
pods:
metric:
name: llm_queue_depth
target:
type: AverageValue
averageValue: "5"
- type: Pods
pods:
metric:
name: gpu_utilization_percent
target:
type: AverageValue
averageValue: "75"
behavior:
scaleUp:
stabilizationWindowSeconds: 60
policies:
- type: Pods
value: 2
periodSeconds: 120
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Pods
value: 1
periodSeconds: 300
CI/CD Design for AI Services
- Build immutable containers with pinned dependencies and model hashes.
- Use environment promotion:
dev -> stage -> prod. - Fail deployment if:
- regression evals drop below baseline,
- safety tests exceed risk threshold,
- p95 latency exceeds SLO budget.
- Store deployment evidence for audits (commit SHA, eval report, approver).
Operational SLOs
| Signal | Target | Measurement Window | |--------|--------|--------------------| | Availability | 99.9% | 30-day rolling | | p95 Latency | < 1200ms | 5-min buckets | | Cost per request | < $0.05 | 1-hour average | | Task success rate | > 90% | 24-hour rolling | | Groundedness | > 85% | 24-hour rolling |
Platform Guardrails
- Enforce tenant quotas and model allow-lists.
- Require structured output contracts for automation paths.
- Default to low-risk model settings for critical workflows.
- Disable unconstrained tool execution in production.
Tooling Stack (Example)
| Layer | Tools | |-------|-------| | Orchestration | Argo Workflows, GitHub Actions, Airflow | | Model Registry | MLflow, custom metadata DB | | Gateway | LiteLLM, Envoy-based API gateway | | Observability | OpenTelemetry + Prometheus + Grafana + Langfuse | | Policy | OPA/Rego for deployment and runtime checks | | Evaluation | RAGAS, custom eval harness, Promptfoo | | Serving | vLLM, TGI, Triton Inference Server |
Troubleshooting
| Issue | Diagnosis | Resolution | |-------|-----------|------------| | Canary fails quality gate | Compare eval results with baseline | Adjust model config or revert version | | Deployment stuck in rollout | Check pod events and resource quotas | Fix resource limits or node availability | | A/B test shows no significant difference
Truncated for display — read the full file on GitHub.
Related Skills
taste-skill
116.8kAgent skills that give coding agents taste: art direction, reference images and a browser check instead of generic AI pages
Agent-Reach
112.3kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
ruflo
81.9k🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
headroom
79.8kCompress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
