SkillAgentSearch skills...

ai-observability

Use when adding Spring AI-specific model observations, token usage, latency, externally configured cost attribution, advisor telemetry, or protected prompt and completion logging. Use production-observability for general service metrics, health, logs, and OTLP setup.

Install / Use

npx skills add rrezartprebreza/spring-boot-skills --skill ai-observability

Installs into whichever agent you are using.

About this skill
šŸ“„

SKILL.md

Installable skill definition

Quality Score

89/100

Supported Platforms

Universal

Tags

Our assessment of ai-observability

ai-observability scores 89/100 on our quality scale, 1736th of 4,698 Development & Engineering skills we index (top 37%).

Its SKILL.md is 7.1 KB long, well organised into 9 sections with 6 code examples: a thorough specification that gives an agent plenty to work with.

It has 334 GitHub stars, a meaningful sign that others use it.

Substance
29/30
Structure
20/20
Description
15/15
Adoption
11/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated 18 days ago, so ai-observability is actively maintained.
  • It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
  • Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-11-12. Automated pattern scan on 2026-11-12. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

ai-observability compared with similar skills

All 4 of these similar skills score higher than ai-observability; compare them before choosing.

SkillScoreStarsUpdatedFormat
ai-observability (this skill)by rrezartprebreza8933418d agoSKILL.md
taste-skillby Leonxlnx100116.8k8d agoCLAUDE.md
ai-job-searchby ai-job-search-dev10072.1ktodayCLAUDE.md
claude-howtoby luongnv8910047.3k2d agoCLAUDE.md
lobehubby lobehub10085.6ktodayMCP Server

Frequently asked questions

How do I install ai-observability?
Run npx skills add rrezartprebreza/spring-boot-skills --skill ai-observability. The install tabs above show the steps for each supported agent.
Which AI agents does ai-observability work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is ai-observability safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is ai-observability still maintained?
The repository was last updated 18 days ago, so ai-observability is actively maintained.

name: ai-observability description: > Use when adding Spring AI-specific model observations, token usage, latency, externally configured cost attribution, advisor telemetry, or protected prompt and completion logging. Use production-observability for general service metrics, health, logs, and OTLP setup.

AI Observability

Dependencies

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

Spring AI Built-in Observability

Spring AI 1.0+ includes built-in Micrometer instrumentation:

spring:
  ai:
    chat:
      observations:
        log-prompt: false      # GA renamed include-prompt → log-prompt. true only in local dev (PII)
        log-completion: false  # GA renamed include-completion → log-completion
management:
  metrics:
    tags:
      application: order-service
  endpoints:
    web:
      exposure:
        include: health,prometheus,metrics

Auto-generated metrics (OpenTelemetry GenAI semantic conventions):

  • gen_ai.client.operation — model call latency, tagged with provider and model
  • gen_ai.client.token.usage — token counts (input/output/total)
  • spring.ai.chat.client — ChatClient-level operation timer/span

Custom AI Metrics

@Component
@RequiredArgsConstructor
public class AiMetrics {

    private final MeterRegistry meterRegistry;

    private final Timer.Builder promptTimer = Timer.builder("ai.prompt.latency")
        .description("LLM prompt latency");

    private final Counter.Builder tokenCounter = Counter.builder("ai.tokens.used")
        .description("Total tokens consumed");

    public <T> T track(String operation, String model, Supplier<T> call) {
        return Timer.builder("ai.prompt.latency")
            .tag("operation", operation)
            .tag("model", model)
            .register(meterRegistry)
            .record(call); // record(Supplier) — recordCallable throws checked Exception
    }

    public void recordTokens(String operation, String model, int inputTokens, int outputTokens) {
        Counter.builder("ai.tokens.used")
            .tag("operation", operation)
            .tag("model", model)
            .tag("type", "input")
            .register(meterRegistry)
            .increment(inputTokens);

        Counter.builder("ai.tokens.used")
            .tag("operation", operation)
            .tag("model", model)
            .tag("type", "output")
            .register(meterRegistry)
            .increment(outputTokens);
    }
}

Prompt/Response Logging Advisor

GA replaced the whole advisor API: CallAroundAdvisor → CallAdvisor, AdvisedRequest → ChatClientRequest, AdvisedResponse → ChatClientResponse, and Usage.getGenerationTokens() → getCompletionTokens(). Agents reliably generate the old one — it does not compile on 1.0.

@Component
public class AiAuditAdvisor implements CallAdvisor {

    private static final Logger log = LoggerFactory.getLogger(AiAuditAdvisor.class);

    @Override
    public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain) {
        String requestId = UUID.randomUUID().toString();
        long start = System.currentTimeMillis();

        log.info("[AI-AUDIT] requestId={} promptLength={}",
            requestId, request.prompt().getUserMessage().getText().length());

        try {
            ChatClientResponse response = chain.nextCall(request);
            long latency = System.currentTimeMillis() - start;

            ChatResponse chatResponse = response.chatResponse();
            if (chatResponse != null && chatResponse.getMetadata() != null) {
                Usage usage = chatResponse.getMetadata().getUsage();
                log.info("[AI-AUDIT] requestId={} latencyMs={} inputTokens={} outputTokens={}",
                    requestId, latency,
                    usage.getPromptTokens(), usage.getCompletionTokens()); // GA: not getGenerationTokens()
            }
            return response;
        } catch (Exception e) {
            log.error("[AI-AUDIT] requestId={} FAILED after {}ms", requestId,
                System.currentTimeMillis() - start, e);
            throw e;
        }
    }

    @Override
    public String getName() { return "AiAuditAdvisor"; }

    @Override
    public int getOrder() { return Ordered.LOWEST_PRECEDENCE; }
}

Cost attribution

  • Keep provider prices in externally managed configuration with an effective date and currency.
  • Key prices by the exact provider model identifier returned in usage metadata.
  • Reject an unknown model instead of silently applying a default price.
  • Preserve the raw token usage so historical costs can be recalculated after pricing changes.
  • Prefer provider billing exports for invoices; application estimates are operational signals only.

Structured AI Audit Log (DB)

The audit advisor template treats telemetry as best-effort. Catch submission failures around the call to the separate async bean, as well as persistence failures inside it: a full or stopped executor can reject a task before its method runs. Use a bounded executor and monitor rejected audit tasks; do not use caller-runs when response latency must remain independent of audit storage. Store sanitized error types, not raw provider exception messages. Mandatory compliance audit records need durable delivery, not this best-effort design.

See Spring task execution and rejection policies.

@Entity
@Table(name = "ai_audit_log")
public class AiAuditLog {
    @Id @GeneratedValue(strategy = GenerationType.UUID)
    private UUID id;
    private String operation;
    private String model;
    private int inputTokens;
    private int outputTokens;
    private double estimatedCostUsd;
    private long latencyMs;
    private boolean success;
    private Instant createdAt;
}

// Async to avoid blocking main flow
@Async
public void saveAuditLog(AiAuditLog log) {
    auditLogRepository.save(log);
}

application.yml — Full Observability

management:
  endpoints:
    web:
      exposure:
        include: health,prometheus,metrics,info
  metrics:
    distribution:
      percentiles-histogram:
        ai.prompt.latency: true  # enables P50/P95/P99
  tracing:
    sampling:
      probability: 1.0  # 100% trace sampling in dev, reduce in prod

logging:
  level:
    org.springframework.ai: DEBUG  # enable in dev only

Gotchas

  • Agent implements CallAroundAdvisor/AdvisedRequest — removed in GA; use CallAdvisor/ChatClientRequest
  • Agent calls usage.getGenerationTokens() — GA renamed it to getCompletionTokens()
  • Agent logs full prompts in production — keep log-prompt: false for PII safety
  • Agent skips async on audit saves — always @Async to avoid latency impact, and put the @Async method on a separate bean; calling it on this bypasses the proxy and runs synchronously
  • Agent hardcodes token pricing — extract to config, prices change
  • Agent misses failed calls in metrics — track errors separately with error tag

Related Skills

View on GitHub
GitHub Stars334
CategoryDevelopment
Updated18d ago
Forks54

Languages

Java

Trust signals

100/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

No cautions
ai-observability — Universal Skill: Install & Safety Check | SkillAgent