AI可观测性平台：LLM调用的专属Metrics、Tracing与Logging设计

AI可观测性平台：LLM调用的专属Metrics、Tracing与Logging设计

一、为什么传统可观测性不适用于LLM应用

微服务的可观测性体系（Metrics/Tracing/Logging三支柱）已经非常成熟——Prometheus + Grafana监控QPS和延迟，Jaeger/Zipkin追踪调用链路，ELK/Loki收集日志。但将这些方案直接套用到LLM应用上时，会出现明显的能力缺口。

核心差异在于：LLM调用引入了传统API调用不存在的维度——Token消耗、模型推理时间、缓存命中率、Prompt/Response的内容质量。这些维度不仅是性能指标，更直接关联到成本和用户体验。

一个LLM请求的生命周期包含多个专属观测点：

graph LR
subgraph 请求生命周期
A[用户请求] –> B[Prompt组装<br/>含RAG检索结果]
B –> C{命中语义缓存?}
C –>|是| D[返回缓存结果<br/>Token消耗=0]
C –>|否| E[模型推理]
E –> F[流式输出<br/>Token-by-Token]
F –> G[Response后处理]
G –> H[返回用户]
end

subgraph 观测点
O1[Prompt Token数]
O2[缓存命中率]
O3[TTFT首Token延迟]
O4[Token生成速率]
O5[总Token消耗/$成本]
O6[Response质量指标]
end

B -.-> O1
C -.-> O2
E -.-> O3
F -.-> O4
F -.-> O5
G -.-> O6

二、AI专属指标体系

2.1 核心指标定义

与传统微服务相比，LLM应用需要监控以下专属指标：

public class LLMMetrics {
// === Token相关 ===
private final Counter promptTokensTotal; // 累计Prompt Token消耗
private final Counter completionTokensTotal; // 累计Completion Token消耗
private final Histogram tokensPerRequest; // 每次请求的总Token数

// === 延迟相关 ===
private final Histogram timeToFirstToken; // 首Token延迟（TTFT）
private final Histogram timePerOutputToken; // 每个输出Token的生成延迟
private final Histogram totalRequestLatency; // 请求总延迟

// === 缓存相关 ===
private final Counter cacheHits; // 语义缓存命中次数
private final Counter cacheMisses; // 语义缓存未命中次数
private final Gauge cacheHitRate; // 缓存命中率

// === 成本相关 ===
private final Counter totalCost; // 累计费用（美元）
private final Histogram costPerRequest; // 每次请求的费用

// === 质量相关 ===
private final Histogram responseTokenCount; // Response token数量
private final Counter streamingConnections; // 活跃的流式连接数
private final Counter rateLimitErrors; // 限流错误次数
}

2.2 指标的Prometheus采集实现

@Configuration
public class LLMMetricsConfig {

@Bean
public MeterRegistry meterRegistry() {
return new PrometheusMeterRegistry(PrometheusConfig.DEFAULT);
}

@Bean
public LLMObservabilityInterceptor llmInterceptor(MeterRegistry registry) {
return new LLMObservabilityInterceptor(registry);
}
}

@Aspect
@Component
public class LLMObservabilityInterceptor {
private final MeterRegistry registry;

@Around("@annotation(llmCall)")
public Object observeLLMCall(ProceedingJoinPoint pjp, LLMCall llmCall) {
Timer.Sample sample = Timer.start(registry);
long startTime = System.nanoTime();

LLMResponse response;
try {
response = (LLMResponse) pjp.proceed();
recordSuccess(llmCall.model(), response, sample, startTime);
} catch (Exception e) {
recordFailure(llmCall.model(), e);
throw e;
}
return response;
}

private void recordSuccess(String model, LLMResponse resp,
Timer.Sample sample, long startTime) {
// Token指标
registry.counter("llm.tokens.prompt", "model", model)
.increment(resp.getPromptTokens());
registry.counter("llm.tokens.completion", "model", model)
.increment(resp.getCompletionTokens());
registry.counter("llm.cost.total", "model", model)
.increment(calculateCost(model, resp));

// 延迟指标
double ttftMs = (resp.getFirstTokenTime() – startTime) / 1_000_000.0;
registry.timer("llm.latency.ttft", "model", model)
.record((long) ttftMs, TimeUnit.MILLISECONDS);

registry.timer("llm.latency.total", "model", model)
.record(System.nanoTime() – startTime, TimeUnit.NANOSECONDS);

// 输出速率
double tps = resp.getCompletionTokens() /
((System.nanoTime() – resp.getFirstTokenTime()) / 1_000_000_000.0);
registry.summary("llm.tokens_per_second", "model", model)
.record(tps);
}

private double calculateCost(String model, LLMResponse resp) {
// GPT-4: prompt $0.03/1K, completion $0.06/1K
return switch (model) {
case "gpt-4" -> resp.getPromptTokens() * 0.03 / 1000
+ resp.getCompletionTokens() * 0.06 / 1000;
case "gpt-3.5-turbo" -> resp.getPromptTokens() * 0.0015 / 1000
+ resp.getCompletionTokens() * 0.002 / 1000;
default -> 0;
};
}
}

2.3 Grafana告警规则

groups:
– name: llm_alerts
rules:
– alert: LLMCostSpike
expr: rate(llm_cost_total[5m]) > 10 # 每分钟费用超过$10
for: 5m
annotations:
summary: "LLM cost spike: {{ $value }}/min"

– alert: HighTTFT
expr: histogram_quantile(0.99, rate(llm_latency_ttft_bucket[5m])) > 3000
for: 5m
annotations:
summary: "P99 TTFT exceeds 3s"

– alert: CacheHitRateDrop
expr: rate(llm_cache_hits_total[5m]) /
(rate(llm_cache_hits_total[5m]) + rate(llm_cache_misses_total[5m])) < 0.5
for: 10m
annotations:
summary: "Cache hit rate dropped below 50%"

– alert: RateLimitErrors
expr: rate(llm_rate_limit_errors_total[5m]) > 1
for: 2m
annotations:
summary: "Rate limit errors detected"

三、Prompt→Response完整链路追踪

3.1 Tracing设计

LLM调用的链路追踪需要记录从用户请求到模型响应的完整路径，包括Prompt的组装过程（含RAG检索的文档块）：

public class LLMTracer {
private final Tracer tracer;

public LLMResponse traceLLMCall(LLMRequest request) {
Span span = tracer.spanBuilder("llm.completion")
.setAttribute("model.name", request.getModel())
.setAttribute("model.provider", request.getProvider())
.setAttribute("llm.prompt.tokens", request.getPromptTokens())
.setAttribute("llm.max_tokens", request.getMaxTokens())
.setAttribute("llm.temperature", request.getTemperature())
.startSpan();

try (Scope scope = span.makeCurrent()) {
// 如果有RAG检索，记录子Span
if (request.hasRAGContext()) {
Span retrievalSpan = tracer.spanBuilder("rag.retrieval")
.setAttribute("rag.documents.retrieved",
request.getRAGDocuments().size())
.setAttribute("rag.top_k", request.getRAGTopK())
.startSpan();
retrievalSpan.end();
}

LLMResponse response = callModel(request);

span.setAttribute("llm.completion.tokens", response.getCompletionTokens());
span.setAttribute("llm.stop_reason", response.getStopReason());
span.setAttribute("llm.cost", calculateCost(request.getModel(), response));
span.setStatus(StatusCode.OK);

return response;
} catch (Exception e) {
span.setStatus(StatusCode.ERROR, e.getMessage());
span.recordException(e);
throw e;
} finally {
span.end();
}
}
}

3.2 链路追踪的可视化

在Jaeger中，一次LLM请求的完整链路如下：

llm.completion (3.2s) ─────────────────────────
├── rag.retrieval (120ms) ──
│ ├── vector_db.query (95ms) ──
│ └── rerank (23ms) ──
├── llm.prefill (680ms) ───────
└── llm.decode (2.4s) ─────────────────────────
├── token_1 (80ms)
├── token_2 (78ms)
├── …
└── token_30 (72ms)

每个Span携带的Tags：

• llm.prompt.tokens: 输入token数
• llm.completion.tokens: 输出token数
• llm.ttft_ms: 首Token延迟
• llm.cost_usd: 本次调用费用
• rag.documents.retrieved: 检索文档数

四、成本归因与开源方案对比

4.1 多维度成本归因

graph TB
subgraph 成本归因维度
D1[按模型<br/>GPT-4 vs Claude vs Llama]
D2[按业务线<br/>客服 vs 文档 vs 代码]
D3[按用户<br/>VIP用户 vs 免费用户]
D4[按功能<br/>RAG检索 vs 纯生成]
end

subgraph 成本分析
C1[日/周/月消费趋势]
C2[单次请求成本分布]
C3[Token使用效率<br/>输出/输入比]
C4[缓存节省的成本]
end

D1 –> C1
D2 –> C1
D3 –> C2
D4 –> C3
D4 –> C4

成本归因SQL示例（基于ClickHouse存储）：

— 按业务线统计每日LLM成本
SELECT
toDate(timestamp) AS date,
business_line,
model,
sum(prompt_tokens) AS total_prompt_tokens,
sum(completion_tokens) AS total_completion_tokens,
sum(cost_usd) AS total_cost_usd,
count() AS request_count,
avg(ttft_ms) AS avg_ttft_ms
FROM llm_observability
WHERE date >= today() – 7
GROUP BY date, business_line, model
ORDER BY date DESC, total_cost_usd DESC;

— 识别高成本低价值调用
SELECT
user_id,
business_line,
avg(completion_tokens) AS avg_output_tokens,
sum(cost_usd) AS total_cost,
count() AS call_count,
sum(cost_usd) / count() AS avg_cost_per_call
FROM llm_observability
WHERE timestamp >= now() – INTERVAL 24 HOUR
AND completion_tokens < 10 — 输出极短但调用了昂贵模型
AND model LIKE 'gpt-4%'
GROUP BY user_id, business_line
HAVING call_count > 100
ORDER BY total_cost DESC
LIMIT 20;

4.2 开源方案对比

4.3 Langfuse集成示例

from langfuse import Langfuse
from langfuse.decorators import observe, langfuse_context

langfuse = Langfuse(
secret_key="sk-lf-…",
public_key="pk-lf-…",
host="https://cloud.langfuse.com"
)

@observe(as_type="generation")
async def llm_completion(prompt: str, model: str = "gpt-4"):
langfuse_context.update_current_observation(
model=model,
usage_details={"prompt_tokens": len(prompt) // 4}
)

response = await openai_client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}]
)

langfuse_context.update_current_observation(
usage_details={
"completion_tokens": response.usage.completion_tokens,
"total_tokens": response.usage.total_tokens,
},
cost_details={
"total": calculate_cost(model, response.usage)
}
)

return response

@observe()
async def rag_pipeline(query: str):
# 自动追踪RAG检索Span
documents = await vector_search(query, top_k=5)

langfuse_context.update_current_observation(
metadata={"retrieved_docs": len(documents)}
)

prompt = build_prompt(query, documents)
return await llm_completion(prompt)

4.4 自建 vs 开源的选型决策

选择自建的条件：

选择开源的条件：

推荐策略：先用Langfuse快速建立可观测能力（1周内完成），再逐步将核心指标接入自建的Prometheus体系（1~2个月渐进迁移）。

五、总结

LLM应用的可观测性是一个仍在快速演进的领域。与已经标准化的微服务可观测性不同，LLM的可观测性需要覆盖新的维度：

• Token级别指标：不仅衡量延迟和QPS，更要追踪Token消耗与成本的实时变化
• 语义缓存命中率：直接影响模型调用量和响应延迟，是需要重点监控的核心指标
• Prompt→Response全链路Tracing：从RAG检索文档到最终生成的每个token，都应该被追踪
• 多维度成本归因：按模型/业务线/用户/功能维度的成本分析，是控制LLM支出的前提
• 开源方案的合理利用：Langfuse适合快速启动，但长期的指标沉淀和告警体系应建立在自建基础设施上

一个实用的启动路径：第一周接入Langfuse建立Tracing，第二周在Prometheus中补齐AI专属指标，第三周搭建成本Dashboard和告警规则。可观测性不是一次性的工程，而是在每次模型升级、Prompt迭代和流量增长中持续演进的能力。