C#_日志与监控实战

C# 系列教程导航

本文属于《C# 企业级开发实战》系列第 13 篇

C# 日志与监控实战：从 ILogger 到 ELK，生产级可观测性方案

前言：为什么需要可观测性？

在微服务架构和分布式系统中，当生产环境出现问题时，传统的调试方式往往束手无策。你无法在服务器上打断点，也无法简单地复现问题。这时候，可观测性（Observability） 就成为了救命稻草。

可观测性包含三大支柱：

┌─────────────────────────────────────────────────────────────┐
│ 可观测性三支柱 │
├─────────────────┬─────────────────┬─────────────────────────┤
│ 日志 Logs │ 指标 Metrics │ 追踪 Traces │
├─────────────────┼─────────────────┼─────────────────────────┤
│ • 发生了什么 │ • 系统健康状况 │ • 请求完整路径 │
│ • 错误详情 │ • 性能数据 │ • 服务间调用关系 │
│ • 业务上下文 │ • 资源使用 │ • 耗时分析 │
└─────────────────┴─────────────────┴─────────────────────────┘

本文将从 .NET Core 内置的 ILogger 开始，逐步深入到 Serilog 结构化日志、请求追踪、健康检查，最终构建一个集成 Prometheus + Grafana 的完整监控体系。

第一部分：ILogger 基础使用

1.1 ILogger 简介

ILogger 是 .NET Core 内置的日志抽象接口，位于 Microsoft.Extensions.Logging 命名空间。它提供了统一的日志记录 API，支持多种日志提供程序（Provider）。

// ILogger 接口定义
public interface ILogger
{
// 核心日志方法
void Log<TState>(
LogLevel logLevel,
EventId eventId,
TState state,
Exception? exception,
Func<TState, Exception?, string> formatter);

// 检查是否启用某日志级别
bool IsEnabled(LogLevel logLevel);

// 创建日志范围（用于关联一组日志）
IDisposable BeginScope<TState>(TState state);
}

1.2 基本配置与使用

Step 1：安装必要包

<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Logging" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Console" Version="8.0.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="8.0.0" />
</ItemGroup>

Step 2：配置日志服务

using Microsoft.Extensions.Logging;

// 创建日志工厂
using var loggerFactory = LoggerFactory.Create(builder =>
{
builder
.AddFilter("Microsoft", LogLevel.Warning) // 过滤框架日志
.AddFilter("System", LogLevel.Warning)
.AddConsole() // 控制台输出
.AddDebug(); // 调试输出
});

// 获取 logger 实例
var logger = loggerFactory.CreateLogger<Program>();

// 记录不同级别的日志
logger.LogTrace("这是 Trace 级别日志，最详细的信息");
logger.LogDebug("这是 Debug 级别日志，调试信息");
logger.LogInformation("应用启动成功");
logger.LogWarning("这是一个警告：配置项缺失，使用默认值");
logger.LogError("发生错误：数据库连接失败");
logger.LogCritical("严重错误：系统即将停止运行");

1.3 在 ASP.NET Core 中使用

在 ASP.NET Core 项目中，日志服务已自动注册，可通过依赖注入使用：

// Program.cs
var builder = WebApplication.CreateBuilder(args);

// 配置日志（可选，appsettings.json 也可配置）
builder.Logging.ClearProviders();
builder.Logging.AddConsole();
builder.Logging.AddJsonConsole(); // JSON 格式输出，适合容器环境

var app = builder.Build();
app.Run();

在 Controller 中注入使用：

[ApiController]
[Route("api/[controller]")]
public class ProductsController : ControllerBase
{
private readonly ILogger<ProductsController> _logger;
private readonly IProductService _productService;

public ProductsController(
ILogger<ProductsController> logger,
IProductService productService)
{
_logger = logger;
_productService = productService;
}

[HttpGet("{id}")]
public async Task<ActionResult<Product>> GetProduct(int id)
{
_logger.LogInformation("获取产品信息，ProductId: {ProductId}", id);

try
{
var product = await _productService.GetByIdAsync(id);

if (product == null)
{
_logger.LogWarning("产品不存在，ProductId: {ProductId}", id);
return NotFound();
}

_logger.LogInformation("成功获取产品：{ProductName}", product.Name);
return Ok(product);
}
catch (Exception ex)
{
_logger.LogError(ex, "获取产品失败，ProductId: {ProductId}", id);
throw;
}
}
}

1.4 使用 LoggerMessage 源生成器（高性能方案）

.NET 6 引入了 LoggerMessage 源生成器，提供编译时日志消息生成，性能更优：

public partial class OrderService
{
private readonly ILogger<OrderService> _logger;

// 定义日志消息模板（编译时生成）
[LoggerMessage(
Level = LogLevel.Information,
Message = "订单创建成功，订单号: {OrderId}，金额: {Amount}")]
public partial void LogOrderCreated(int orderId, decimal amount);

[LoggerMessage(
Level = LogLevel.Warning,
Message = "库存不足，商品: {ProductName}，需要: {Required}，现有: {Available}")]
public partial void LogInsufficientStock(string productName, int required, int available);

[LoggerMessage(
Level = LogLevel.Error,
Message = "订单处理失败，订单号: {OrderId}")]
public partial void LogOrderFailed(int orderId, Exception ex);

public async Task CreateOrderAsync(OrderRequest request)
{
try
{
// 业务逻辑…
var orderId = 12345;
LogOrderCreated(orderId, request.TotalAmount);
}
catch (Exception ex)
{
LogOrderFailed(0, ex);
throw;
}
}
}

第二部分：结构化日志（Serilog）

2.1 为什么选择 Serilog？

传统日志是纯文本格式，难以查询和分析。结构化日志 将日志保存为结构化数据（如 JSON），支持强大的查询能力。

传统日志：
2024-01-15 10:30:45 [INFO] 用户 user123 登录成功，IP: 192.168.1.100

结构化日志（JSON）：
{
"timestamp": "2024-01-15T10:30:45.123Z",
"level": "Information",
"message": "用户登录成功",
"userId": "user123",
"ipAddress": "192.168.1.100",
"machineName": "web-server-01",
"correlationId": "abc-123-def"
}

2.2 Serilog 基础配置

安装包：

<ItemGroup>
<PackageReference Include="Serilog" Version="3.1.1" />
<PackageReference Include="Serilog.Extensions.Hosting" Version="8.0.0" />
<PackageReference Include="Serilog.Sinks.Console" Version="5.0.0" />
<PackageReference Include="Serilog.Sinks.File" Version="5.0.0" />
<PackageReference Include="Serilog.Sinks.Seq" Version="6.0.0" />
<PackageReference Include="Serilog.Enrichers.Environment" Version="2.3.0" />
<PackageReference Include="Serilog.Enrichers.Thread" Version="3.1.0" />
</ItemGroup>

配置 Serilog：

using Serilog;

// 创建 Serilog logger
Log.Logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
.MinimumLevel.Override("System", LogEventLevel.Warning)
.Enrich.FromLogContext()
.Enrich.WithMachineName()
.Enrich.WithThreadId()
.Enrich.WithProperty("Application", "MyApi")
.Enrich.WithProperty("Environment", "Production")
.WriteTo.Console(
outputTemplate: "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj}{NewLine}{Exception}")
.WriteTo.File(
path: "logs/myapi-.log",
rollingInterval: RollingInterval.Day,
retainedFileCountLimit: 30,
outputTemplate: "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level:u3}] {Message:lj}{NewLine}{Exception}")
.WriteTo.Seq("http://localhost:5341") // Seq 日志服务器
.CreateLogger();

try
{
Log.Information("应用启动中…");
// 运行应用…
}
catch (Exception ex)
{
Log.Fatal(ex, "应用启动失败");
}
finally
{
Log.CloseAndFlush();
}

2.3 集成到 ASP.NET Core

// Program.cs
using Serilog;

var builder = WebApplication.CreateBuilder(args);

// 使用 Serilog 替换默认日志
builder.Host.UseSerilog((context, services, configuration) => configuration
.ReadFrom.Configuration(context.Configuration)
.ReadFrom.Services(services)
.Enrich.FromLogContext()
.Enrich.WithMachineName()
.Enrich.WithProperty("Application", context.HostingEnvironment.ApplicationName)
.WriteTo.Console()
.WriteTo.File("logs/log-.txt", rollingInterval: RollingInterval.Day)
.WriteTo.Seq(context.Configuration["Seq:ServerUrl"] ?? "http://localhost:5341"));

var app = builder.Build();
app.Run();

appsettings.json 配置方式：

{
"Serilog": {
"MinimumLevel": {
"Default": "Information",
"Override": {
"Microsoft": "Warning",
"Microsoft.Hosting.Lifetime": "Information",
"System": "Warning"
}
},
"WriteTo": [
{
"Name": "Console",
"Args": {
"outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj} {Properties:j}{NewLine}{Exception}"
}
},
{
"Name": "File",
"Args": {
"path": "logs/log-.json",
"rollingInterval": "Day",
"formatter": "Serilog.Formatting.Json.JsonFormatter, Serilog"
}
}
],
"Enrich": ["FromLogContext", "WithMachineName", "WithThreadId"]
}
}

2.4 结构化日志最佳实践

使用属性而非字符串拼接：

// ❌ 错误方式：字符串拼接，无法查询
_logger.LogInformation($"用户 {userId} 购买了商品 {productId}，数量 {quantity}");

// ✅ 正确方式：结构化日志，属性可查询
_logger.LogInformation("用户 {UserId} 购买了商品 {ProductId}，数量 {Quantity}",
userId, productId, quantity);

记录业务上下文：

public class PaymentService
{
private readonly ILogger<PaymentService> _logger;

public async Task<PaymentResult> ProcessPaymentAsync(PaymentRequest request)
{
// 使用 LogContext 添加上下文属性
using var _ = LogContext.PushProperty("OrderId", request.OrderId);
using var __ = LogContext.PushProperty("UserId", request.UserId);
using var ___ = LogContext.PushProperty("PaymentMethod", request.Method);

_logger.LogInformation("开始处理支付请求，金额: {Amount} {Currency}",
request.Amount, request.Currency);

try
{
var result = await _gateway.ProcessAsync(request);

_logger.LogInformation("支付成功，交易号: {TransactionId}",
result.TransactionId);

return result;
}
catch (PaymentException ex)
{
_logger.LogError(ex, "支付失败，错误码: {ErrorCode}", ex.ErrorCode);
throw;
}
}
}

自定义 Enricher：

// 添加租户信息的 Enricher
public class TenantEnricher : ILogEventEnricher
{
private readonly IHttpContextAccessor _httpContextAccessor;

public TenantEnricher(IHttpContextAccessor httpContextAccessor)
{
_httpContextAccessor = httpContextAccessor;
}

public void Enrich(LogEvent logEvent, ILogEventPropertyFactory propertyFactory)
{
var httpContext = _httpContextAccessor.HttpContext;
if (httpContext?.User?.Identity?.IsAuthenticated == true)
{
var tenantId = httpContext.User.FindFirst("TenantId")?.Value;
if (!string.IsNullOrEmpty(tenantId))
{
logEvent.AddPropertyIfAbsent(
propertyFactory.CreateProperty("TenantId", tenantId));
}
}
}
}

// 注册
builder.Host.UseSerilog((context, services, configuration) => configuration
.Enrich.With(new TenantEnricher(services.GetRequiredService<IHttpContextAccessor>())));

第三部分：日志级别与过滤

3.1 日志级别详解

.NET 定义了 6 个日志级别，从低到高：

public enum LogLevel
{
Trace = 0, // 最详细，仅开发调试使用
Debug = 1, // 调试信息，开发环境
Information = 2, // 一般信息，生产环境可见
Warning = 3, // 警告，潜在问题
Error = 4, // 错误，需要关注
Critical = 5, // 严重错误，系统级故障
None = 6 // 禁用日志
}

各级别使用场景：

3.2 日志过滤配置

appsettings.json 配置：

{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning",
"Microsoft.EntityFrameworkCore": "Warning",
"MyApp.Services": "Debug",
"MyApp.Services.ExternalApi": "Information"
}
}
}

代码方式配置过滤器：

// 自定义过滤器：只记录包含特定关键字的日志
public class KeywordFilter : ILogFilter
{
private readonly string[] _keywords;

public KeywordFilter(params string[] keywords)
{
_keywords = keywords;
}

public bool IsEnabled(LogLevel level, string category, EventId eventId)
{
return true; // 在 Log 方法中实现过滤
}
}

// 使用 Lambda 过滤
builder.Logging.AddFilter((provider, category, logLevel) =>
{
// 过滤掉健康检查的日志（太频繁）
if (category.Contains("HealthCheck"))
{
return logLevel >= LogLevel.Warning;
}

// 过滤掉 SignalR 心跳日志
if (category.StartsWith("Microsoft.AspNetCore.SignalR"))
{
return logLevel >= LogLevel.Warning;
}

return true;
});

3.3 动态日志级别控制

生产环境中，有时需要动态调整日志级别而不重启应用：

// 自定义配置源
public class DynamicLogLevelProvider : ILoggerProvider
{
private readonly ConcurrentDictionary<string, DynamicLogger> _loggers = new();
private volatile LogLevel _globalMinimumLevel = LogLevel.Information;
private readonly ConcurrentDictionary<string, LogLevel> _categoryLevels = new();

public void SetGlobalLevel(LogLevel level)
{
_globalMinimumLevel = level;
}

public void SetCategoryLevel(string category, LogLevel level)
{
_categoryLevels[category] = level;
}

public ILogger CreateLogger(string categoryName)
{
return _loggers.GetOrAdd(categoryName,
name => new DynamicLogger(this, name));
}

public bool IsEnabled(string category, LogLevel level)
{
if (_categoryLevels.TryGetValue(category, out var categoryLevel))
{
return level >= categoryLevel;
}
return level >= _globalMinimumLevel;
}

public void Dispose() { }
}

// API 端点控制日志级别
[ApiController]
[Route("api/[controller]")]
public class LogLevelController : ControllerBase
{
private readonly DynamicLogLevelProvider _logProvider;

[HttpPut("global")]
public IActionResult SetGlobalLevel([FromBody] SetLogLevelRequest request)
{
_logProvider.SetGlobalLevel(request.Level);
return Ok(new { Message = $"全局日志级别已设置为 {request.Level}" });
}

[HttpPut("category")]
public IActionResult SetCategoryLevel([FromBody] SetCategoryLogLevelRequest request)
{
_logProvider.SetCategoryLevel(request.Category, request.Level);
return Ok(new { Message = $"分类 {request.Category} 日志级别已设置为 {request.Level}" });
}
}

第四部分：请求追踪与 CorrelationId

4.1 为什么需要请求追踪？

在微服务架构中，一个请求可能经过多个服务：

用户请求 → API Gateway → 订单服务 → 库存服务 → 支付服务 → 通知服务

当请求失败时，我们需要追踪整个调用链。CorrelationId（关联 ID） 就是连接所有日志的纽带。

4.2 实现 CorrelationId 中间件

public class CorrelationIdMiddleware
{
private const string CorrelationIdHeader = "X-Correlation-Id";
private readonly RequestDelegate _next;

public CorrelationIdMiddleware(RequestDelegate next)
{
_next = next;
}

public async Task Invoke(HttpContext context)
{
// 从请求头获取或生成新的 CorrelationId
string correlationId = context.Request.Headers.TryGetValue(
CorrelationIdHeader, out var headerValue) && !string.IsNullOrEmpty(headerValue)
? headerValue.ToString()
: Guid.NewGuid().ToString("N");

// 设置响应头
context.Response.Headers[CorrelationIdHeader] = correlationId;

// 存储到 HttpContext.Items
context.Items[CorrelationIdHeader] = correlationId;

// 使用 LogContext 使所有日志自动包含 CorrelationId
using (LogContext.PushProperty("CorrelationId", correlationId))
{
await _next(context);
}
}
}

// 注册中间件
app.UseMiddleware<CorrelationIdMiddleware>();

4.3 HTTP 客户端自动传递 CorrelationId

public class CorrelationIdDelegatingHandler : DelegatingHandler
{
private const string CorrelationIdHeader = "X-Correlation-Id";
private readonly IHttpContextAccessor _httpContextAccessor;

public CorrelationIdDelegatingHandler(IHttpContextAccessor httpContextAccessor)
{
_httpContextAccessor = httpContextAccessor;
}

protected override async Task<HttpResponseMessage> SendAsync(
HttpRequestMessage request,
CancellationToken cancellationToken)
{
var correlationId = _httpContextAccessor.HttpContext?.Items["X-Correlation-Id"]?.ToString();

if (!string.IsNullOrEmpty(correlationId))
{
request.Headers.Add(CorrelationIdHeader, correlationId);
}

return await base.SendAsync(request, cancellationToken);
}
}

// 注册 HttpClient
builder.Services.AddHttpClient<IExternalApiService, ExternalApiService>()
.AddHttpMessageHandler<CorrelationIdDelegatingHandler>();

4.4 完整的请求日志中间件

public class RequestLoggingMiddleware
{
private readonly RequestDelegate _next;
private readonly ILogger<RequestLoggingMiddleware> _logger;

public RequestLoggingMiddleware(
RequestDelegate next,
ILogger<RequestLoggingMiddleware> logger)
{
_next = next;
_logger = logger;
}

public async Task Invoke(HttpContext context)
{
var stopwatch = Stopwatch.StartNew();
var requestId = Guid.NewGuid().ToString("N");

// 请求信息
var requestInfo = new
{
RequestId = requestId,
Method = context.Request.Method,
Path = context.Request.Path.Value,
QueryString = context.Request.QueryString.Value,
UserAgent = context.Request.Headers["User-Agent"].ToString(),
RemoteIp = context.Connection.RemoteIpAddress?.ToString(),
UserId = context.User?.FindFirst(ClaimTypes.NameIdentifier)?.Value
};

_logger.LogInformation("HTTP 请求开始: {@RequestInfo}", requestInfo);

// 捕获响应
var originalBodyStream = context.Response.Body;
using var memoryStream = new MemoryStream();
context.Response.Body = memoryStream;

try
{
await _next(context);
}
catch (Exception ex)
{
_logger.LogError(ex, "HTTP 请求异常: {RequestId}", requestId);
throw;
}
finally
{
stopwatch.Stop();

memoryStream.Position = 0;
await memoryStream.CopyToAsync(originalBodyStream);
context.Response.Body = originalBodyStream;

var responseInfo = new
{
RequestId = requestId,
StatusCode = context.Response.StatusCode,
ElapsedMilliseconds = stopwatch.ElapsedMilliseconds,
ResponseSize = memoryStream.Length
};

var logLevel = context.Response.StatusCode >= 500
? LogLevel.Error
: context.Response.StatusCode >= 400
? LogLevel.Warning
: LogLevel.Information;

_logger.Log(logLevel, "HTTP 请求完成: {@ResponseInfo}", responseInfo);
}
}
}

第五部分：健康检查端点

5.1 配置健康检查

// Program.cs
builder.Services.AddHealthChecks()
// 数据库健康检查
.AddNpgSql(
builder.Configuration.GetConnectionString("DefaultConnection"),
name: "database",
tags: new[] { "db", "critical" })
// Redis 健康检查
.AddRedis(
builder.Configuration["Redis:ConnectionString"],
name: "redis",
tags: new[] { "cache" })
// 自定义健康检查
.AddCheck<ExternalApiHealthCheck>("external-api", tags: new[] { "external" })
.AddCheck<MemoryHealthCheck>("memory", tags: new[] { "system" });

// 配置健康检查端点
app.MapHealthChecks("/health", new HealthCheckOptions
{
ResponseWriter = WriteHealthCheckResponse
});

// 详细健康检查（带标签过滤）
app.MapHealthChecks("/health/ready", new HealthCheckOptions
{
Predicate = check => check.Tags.Contains("critical"),
ResponseWriter = WriteHealthCheckResponse
});

app.MapHealthChecks("/health/live", new HealthCheckOptions
{
Predicate = _ => false // 只检查应用是否运行
});

5.2 自定义健康检查

// 外部 API 健康检查
public class ExternalApiHealthCheck : IHealthCheck
{
private readonly IHttpClientFactory _httpClientFactory;
private readonly ILogger<ExternalApiHealthCheck> _logger;

public ExternalApiHealthCheck(
IHttpClientFactory httpClientFactory,
ILogger<ExternalApiHealthCheck> logger)
{
_httpClientFactory = httpClientFactory;
_logger = logger;
}

public async Task<HealthCheckResult> CheckHealthAsync(
HealthCheckContext context,
CancellationToken cancellationToken = default)
{
try
{
var client = _httpClientFactory.CreateClient("ExternalApi");
var response = await client.GetAsync("/health", cancellationToken);

if (response.IsSuccessStatusCode)
{
return HealthCheckResult.Healthy("外部 API 可用");
}

return HealthCheckResult.Degraded($"外部 API 返回 {response.StatusCode}");
}
catch (Exception ex)
{
_logger.LogError(ex, "外部 API 健康检查失败");
return HealthCheckResult.Unhealthy("外部 API 不可用", ex);
}
}
}

// 内存使用健康检查
public class MemoryHealthCheck : IHealthCheck
{
private readonly long _threshold = 1024 * 1024 * 1024; // 1GB

public Task<HealthCheckResult> CheckHealthAsync(
HealthCheckContext context,
CancellationToken cancellationToken = default)
{
var allocated = GC.GetTotalMemory(forceFullCollection: false);
var data = new Dictionary<string, object>
{
["AllocatedBytes"] = allocated,
["AllocatedMB"] = allocated / 1024 / 1024,
["ThresholdMB"] = _threshold / 1024 / 1024
};

if (allocated < _threshold * 0.8)
{
return Task.FromResult(HealthCheckResult.Healthy("内存使用正常", data));
}

if (allocated < _threshold)
{
return Task.FromResult(HealthCheckResult.Degraded("内存使用较高", data: data));
}

return Task.FromResult(HealthCheckResult.Unhealthy("内存使用过高", data: data));
}
}

5.3 自定义健康检查响应格式

private static Task WriteHealthCheckResponse(
HttpContext context,
HealthReport report)
{
context.Response.ContentType = "application/json";

var response = new
{
status = report.Status.ToString(),
totalDuration = report.TotalDuration.TotalMilliseconds,
checks = report.Entries.Select(entry => new
{
name = entry.Key,
status = entry.Value.Status.ToString(),
duration = entry.Value.Duration.TotalMilliseconds,
description = entry.Value.Description,
data = entry.Value.Data,
tags = entry.Value.Tags,
exception = entry.Value.Exception?.Message
}),
timestamp = DateTime.UtcNow
};

return context.Response.WriteAsJsonAsync(response);
}

响应示例：

{
"status": "Healthy",
"totalDuration": 156.32,
"checks": [
{
"name": "database",
"status": "Healthy",
"duration": 45.21,
"description": null,
"data": {},
"tags": ["db", "critical"]
},
{
"name": "redis",
"status": "Healthy",
"duration": 12.05,
"description": null,
"data": {},
"tags": ["cache"]
},
{
"name": "memory",
"status": "Degraded",
"duration": 0.12,
"description": "内存使用较高",
"data": {
"AllocatedMB": 856,
"ThresholdMB": 1024
},
"tags": ["system"]
}
],
"timestamp": "2024-01-15T10:30:45Z"
}

第六部分：集成 Prometheus + Grafana 监控

6.1 Prometheus 简介

Prometheus 是开源的监控告警系统，采用拉取模式收集指标，支持强大的 PromQL 查询语言。

┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Application │────▶│ Prometheus │────▶│ Grafana │
│ /metrics 端点 │ │ 时序数据库 │ │ 可视化 │
└─────────────────┘ └─────────────────┘ └─────────────────┘

6.2 添加 Prometheus 支持

安装包：

<ItemGroup>
<PackageReference Include="prometheus-net" Version="8.2.1" />
<PackageReference Include="prometheus-net.AspNetCore" Version="8.2.1" />
</ItemGroup>

配置 Prometheus：

// Program.cs
using Prometheus;

// 添加 Prometheus 指标
builder.Services.AddMetricServer(options =>
{
options.Port = 9090; // 指标端口
});

// 添加默认的 ASP.NET Core 指标
app.UseHttpMetrics(); // 自动记录请求计数、耗时、活跃请求等

// 指标端点
app.MapMetrics();

6.3 自定义业务指标

// Metrics/CustomMetrics.cs
using Prometheus;

public static class CustomMetrics
{
// 计数器：订单数量
public static readonly Counter OrdersTotal = Metrics
.CreateCounter("orders_total", "订单总数",
new CounterConfiguration
{
LabelNames = new[] { "status", "payment_method" }
});

// 直方图：请求耗时分布
public static readonly Histogram RequestDuration = Metrics
.CreateHistogram("request_duration_seconds", "请求耗时分布",
new HistogramConfiguration
{
LabelNames = new[] { "endpoint", "method" },
Buckets = Histogram.ExponentialBuckets(0.001, 2, 10) // 1ms 到 ~1s
});

// 仪表盘：当前活跃连接数
public static readonly Gauge ActiveConnections = Metrics
.CreateGauge("active_connections", "当前活跃连接数");

// 仪表盘：缓存命中率
public static readonly Gauge CacheHitRate = Metrics
.CreateGauge("cache_hit_rate", "缓存命中率",
new GaugeConfiguration
{
LabelNames = new[] { "cache_name" }
});
}

在业务代码中使用：

public class OrderService : IOrderService
{
private readonly ILogger<OrderService> _logger;

public async Task<OrderResult> CreateOrderAsync(CreateOrderRequest request)
{
using var timer = CustomMetrics.RequestDuration
.WithLabels("/api/orders", "POST")
.NewTimer();

try
{
// 业务逻辑…
var result = await ProcessOrderAsync(request);

CustomMetrics.OrdersTotal
.WithLabels("success", request.PaymentMethod)
.Inc();

return result;
}
catch (Exception)
{
CustomMetrics.OrdersTotal
.WithLabels("failed", request.PaymentMethod)
.Inc();
throw;
}
}
}

// 中间件记录活跃连接
public class ActiveConnectionsMiddleware
{
private readonly RequestDelegate _next;

public ActiveConnectionsMiddleware(RequestDelegate next)
{
_next = next;
}

public async Task Invoke(HttpContext context)
{
CustomMetrics.ActiveConnections.Inc();
try
{
await _next(context);
}
finally
{
CustomMetrics.ActiveConnections.Dec();
}
}
}

6.4 Prometheus 配置文件

# prometheus.yml
global:
scrape_interval: 15s
evaluation_interval: 15s

scrape_configs:
– job_name: 'my-api'
metrics_path: /metrics
static_configs:
– targets: ['my-api:9090']
relabel_configs:
– source_labels: [__address__]
target_label: instance

– job_name: 'kubernetes-pods'
kubernetes_sd_configs:
– role: pod
relabel_configs:
– source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: true

6.5 Grafana Dashboard 配置

常用查询示例：

# 每秒请求数 (QPS)
rate(http_requests_received_total[5m])

# 平均请求延迟
rate(http_request_duration_seconds_sum[5m]) / rate(http_request_duration_seconds_count[5m])

# P95 延迟
histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))

# 错误率
sum(rate(http_requests_received_total{status=~"5.."}[5m])) / sum(rate(http_requests_received_total[5m]))

# 订单成功率
sum(rate(orders_total{status="success"}[5m])) / sum(rate(orders_total[5m]))

Grafana Dashboard JSON 片段：

{
"panels": [
{
"title": "请求速率 (QPS)",
"type": "graph",
"targets": [
{
"expr": "sum(rate(http_requests_received_total[5m])) by (method)",
"legendFormat": "{{method}}"
}
]
},
{
"title": "P95 延迟",
"type": "graph",
"targets": [
{
"expr": "histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket[5m])) by (le))",
"legendFormat": "P95"
}
]
},
{
"title": "错误率",
"type": "stat",
"targets": [
{
"expr": "sum(rate(http_requests_received_total{status=~\\"5..\\"}[5m])) / sum(rate(http_requests_received_total[5m])) * 100",
"legendFormat": "Error Rate %"
}
],
"thresholds": "1,5",
"format": "percent"
}
]
}

第七部分：实战案例：完整的 API 监控体系

7.1 项目结构

MyApi/
├── Program.cs
├── appsettings.json
├── Metrics/
│ ├── CustomMetrics.cs
│ └── MetricsMiddleware.cs
├── Middleware/
│ ├── CorrelationIdMiddleware.cs
│ ├── RequestLoggingMiddleware.cs
│ └── ExceptionHandlingMiddleware.cs
├── HealthChecks/
│ ├── DatabaseHealthCheck.cs
│ ├── RedisHealthCheck.cs
│ └── ExternalApiHealthCheck.cs
├── Services/
│ └── OrderService.cs
└── Controllers/
└── OrdersController.cs

7.2 完整的 Program.cs

using Prometheus;
using Serilog;
using MyApi.Middleware;
using MyApi.HealthChecks;
using MyApi.Metrics;

var builder = WebApplication.CreateBuilder(args);

// ==================== 日志配置 ====================
builder.Host.UseSerilog((context, services, configuration) => configuration
.ReadFrom.Configuration(context.Configuration)
.ReadFrom.Services(services)
.MinimumLevel.Information()
.MinimumLevel.Override("Microsoft", Serilog.Events.LogEventLevel.Warning)
.MinimumLevel.Override("System", Serilog.Events.LogEventLevel.Warning)
.Enrich.FromLogContext()
.Enrich.WithMachineName()
.Enrich.WithProperty("Application", "MyApi")
.Enrich.WithProperty("Environment", context.HostingEnvironment.EnvironmentName)
.WriteTo.Console()
.WriteTo.File(
path: "logs/log-.json",
rollingInterval: RollingInterval.Day,
formatter: new Serilog.Formatting.Json.JsonFormatter())
.WriteTo.Seq(context.Configuration["Seq:Url"] ?? "http://localhost:5341"));

// ==================== 健康检查 ====================
builder.Services.AddHealthChecks()
.AddNpgSql(
builder.Configuration.GetConnectionString("DefaultConnection"),
name: "database",
tags: new[] { "critical" })
.AddRedis(
builder.Configuration["Redis:ConnectionString"],
name: "redis",
tags: new[] { "cache" })
.AddCheck<ExternalApiHealthCheck>("external-api", tags: new[] { "external" });

// ==================== Prometheus 指标 ====================
builder.Services.AddMetricServer();

// ==================== 服务注册 ====================
builder.Services.AddControllers();
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
builder.Services.AddHttpContextAccessor();

// HttpClient with correlation propagation
builder.Services.AddHttpClient<IExternalApiService, ExternalApiService>()
.AddHttpMessageHandler<CorrelationIdDelegatingHandler>();

var app = builder.Build();

// ==================== 中间件管道 ====================
// 全局异常处理
app.UseMiddleware<ExceptionHandlingMiddleware>();

// CorrelationId
app.UseMiddleware<CorrelationIdMiddleware>();

// 请求日志
app.UseMiddleware<RequestLoggingMiddleware>();

// Prometheus 指标
app.UseHttpMetrics();

// Swagger
if (app.Environment.IsDevelopment())
{
app.UseSwagger();
app.UseSwaggerUI();
}

// 健康检查端点
app.MapHealthChecks("/health", new HealthCheckOptions
{
ResponseWriter = HealthCheckResponseWriter.WriteResponse
});
app.MapHealthChecks("/health/ready", new HealthCheckOptions
{
Predicate = check => check.Tags.Contains("critical"),
ResponseWriter = HealthCheckResponseWriter.WriteResponse
});
app.MapHealthChecks("/health/live", new HealthCheckOptions
{
Predicate = _ => false
});

// Prometheus 指标端点
app.MapMetrics();

app.MapControllers();

Log.Information("应用启动成功，环境: {Environment}", app.Environment.EnvironmentName);

app.Run();

7.3 全局异常处理中间件

public class ExceptionHandlingMiddleware
{
private readonly RequestDelegate _next;
private readonly ILogger<ExceptionHandlingMiddleware> _logger;

public ExceptionHandlingMiddleware(
RequestDelegate next,
ILogger<ExceptionHandlingMiddleware> logger)
{
_next = next;
_logger = logger;
}

public async Task Invoke(HttpContext context)
{
try
{
await _next(context);
}
catch (ValidationException ex)
{
await HandleValidationExceptionAsync(context, ex);
}
catch (NotFoundException ex)
{
await HandleNotFoundExceptionAsync(context, ex);
}
catch (BusinessException ex)
{
await HandleBusinessExceptionAsync(context, ex);
}
catch (Exception ex)
{
await HandleUnknownExceptionAsync(context, ex);
}
}

private async Task HandleValidationExceptionAsync(HttpContext context, ValidationException ex)
{
_logger.LogWarning(ex, "验证失败: {Message}", ex.Message);

context.Response.StatusCode = StatusCodes.Status400BadRequest;
await context.Response.WriteAsJsonAsync(new
{
Error = "ValidationError",
Message = ex.Message,
Details = ex.Errors,
TraceId = Activity.Current?.Id ?? context.TraceIdentifier
});
}

private async Task HandleNotFoundExceptionAsync(HttpContext context, NotFoundException ex)
{
_logger.LogInformation("资源未找到: {ResourceType} {ResourceId}",
ex.ResourceType, ex.ResourceId);

context.Response.StatusCode = StatusCodes.Status404NotFound;
await context.Response.WriteAsJsonAsync(new
{
Error = "NotFound",
Message = ex.Message,
TraceId = Activity.Current?.Id ?? context.TraceIdentifier
});
}

private async Task HandleBusinessExceptionAsync(HttpContext context, BusinessException ex)
{
_logger.LogWarning(ex, "业务异常: {Code} – {Message}", ex.Code, ex.Message);

context.Response.StatusCode = StatusCodes.Status422UnprocessableEntity;
await context.Response.WriteAsJsonAsync(new
{
Error = ex.Code,
Message = ex.Message,
TraceId = Activity.Current?.Id ?? context.TraceIdentifier
});
}

private async Task HandleUnknownExceptionAsync(HttpContext context, Exception ex)
{
_logger.LogError(ex, "未处理的异常");

context.Response.StatusCode = StatusCodes.Status500InternalServerError;
await context.Response.WriteAsJsonAsync(new
{
Error = "InternalServerError",
Message = "服务器内部错误，请稍后重试",
TraceId = Activity.Current?.Id ?? context.TraceIdentifier
});
}
}

7.4 订单服务完整示例

public class OrderService : IOrderService
{
private readonly ILogger<OrderService> _logger;
private readonly IOrderRepository _orderRepository;
private readonly IProductService _productService;
private readonly IPaymentService _paymentService;
private readonly INotificationService _notificationService;

public OrderService(
ILogger<OrderService> logger,
IOrderRepository orderRepository,
IProductService productService,
IPaymentService paymentService,
INotificationService notificationService)
{
_logger = logger;
_orderRepository = orderRepository;
_productService = productService;
_paymentService = paymentService;
_notificationService = notificationService;
}

public async Task<OrderResult> CreateOrderAsync(CreateOrderRequest request)
{
using var timer = CustomMetrics.RequestDuration
.WithLabels("order", "create")
.NewTimer();

// 添加业务上下文到日志
using var _ = LogContext.PushProperty("OrderId", Guid.NewGuid());
using var __ = LogContext.PushProperty("UserId", request.UserId);

_logger.LogInformation(
"开始创建订单，用户: {UserId}，商品数量: {ItemCount}",
request.UserId, request.Items.Count);

try
{
// 1. 验证库存
_logger.LogDebug("开始验证库存");
await ValidateInventoryAsync(request.Items);

// 2. 计算订单金额
var totalAmount = await CalculateTotalAmountAsync(request.Items);
_logger.LogInformation("订单金额计算完成: {Amount:C}", totalAmount);

// 3. 创建订单
var order = new Order
{
Id = Guid.NewGuid(),
UserId = request.UserId,
Items = request.Items,
TotalAmount = totalAmount,
Status = OrderStatus.Pending,
CreatedAt = DateTime.UtcNow
};

await _orderRepository.AddAsync(order);
_logger.LogInformation("订单已创建: {OrderId}", order.Id);

// 4. 处理支付
var paymentResult = await _paymentService.ProcessPaymentAsync(
new PaymentRequest
{
OrderId = order.Id,
Amount = totalAmount,
Method = request.PaymentMethod
});

if (!paymentResult.Success)
{
order.Status = OrderStatus.PaymentFailed;
await _orderRepository.UpdateAsync(order);

CustomMetrics.OrdersTotal
.WithLabels("payment_failed", request.PaymentMethod)
.Inc();

throw new PaymentException(paymentResult.ErrorCode, paymentResult.Message);
}

// 5. 更新订单状态
order.Status = OrderStatus.Paid;
order.PaymentId = paymentResult.TransactionId;
await _orderRepository.UpdateAsync(order);

// 6. 发送通知（不阻塞主流程）
_ = Task.Run(async () =>
{
try
{
await _notificationService.SendOrderConfirmationAsync(order);
}
catch (Exception ex)
{
_logger.LogWarning(ex, "发送订单通知失败: {OrderId}", order.Id);
}
});

CustomMetrics.OrdersTotal
.WithLabels("success", request.PaymentMethod)
.Inc();

_logger.LogInformation(
"订单创建成功: {OrderId}，交易号: {TransactionId}",
order.Id, paymentResult.TransactionId);

return new OrderResult
{
OrderId = order.Id,
Status = order.Status,
TransactionId = paymentResult.TransactionId
};
}
catch (InsufficientInventoryException ex)
{
CustomMetrics.OrdersTotal
.WithLabels("inventory_failed", request.PaymentMethod)
.Inc();

_logger.LogWarning(ex, "库存不足: {ProductId}", ex.ProductId);
throw;
}
catch (Exception ex)
{
CustomMetrics.OrdersTotal
.WithLabels("error", request.PaymentMethod)
.Inc();

_logger.LogError(ex, "订单创建失败");
throw;
}
}
}

7.5 Docker Compose 完整配置

# docker-compose.yml
version: '3.8'

services:
my-api:
build: .
ports:
– "8080:80"
– "9090:9090"
environment:
– ASPNETCORE_ENVIRONMENT=Production
– ConnectionStrings__DefaultConnection=Host=postgres;Database=myapi;Username=postgres;Password=postgres
– Redis__ConnectionString=redis:6379
– Seq__Url=http://seq:5341
depends_on:
– postgres
– redis
– seq

postgres:
image: postgres:15
environment:
POSTGRES_DB: myapi
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
volumes:
– postgres_data:/var/lib/postgresql/data

redis:
image: redis:7–alpine
volumes:
– redis_data:/data

seq:
image: datalust/seq:latest
environment:
– ACCEPT_EULA=Y
ports:
– "5341:80"
volumes:
– seq_data:/data

prometheus:
image: prom/prometheus:latest
ports:
– "9091:9090"
volumes:
– ./prometheus.yml:/etc/prometheus/prometheus.yml
– prometheus_data:/prometheus

grafana:
image: grafana/grafana:latest
ports:
– "3000:3000"
environment:
– GF_SECURITY_ADMIN_PASSWORD=admin
volumes:
– grafana_data:/var/lib/grafana
– ./grafana/dashboards:/etc/grafana/provisioning/dashboards
– ./grafana/datasources:/etc/grafana/provisioning/datasources

volumes:
postgres_data:
redis_data:
seq_data:
prometheus_data:
grafana_data:

7.6 监控大盘效果

启动后可访问：

总结

本文从 .NET Core 内置的 ILogger 开始，逐步构建了一个完整的可观测性体系：

核心要点回顾

┌─────────────────────────────────────────────────────────────────────┐
│ 可观测性最佳实践 │
├─────────────────────────────────────────────────────────────────────┤
│ 1. 使用结构化日志，避免字符串拼接 │
│ 2. 为每条日志添加业务上下文（UserId、OrderId 等） │
│ 3. 使用 CorrelationId 追踪分布式请求 │
│ 4. 合理设置日志级别，生产环境过滤框架日志 │
│ 5. 配置健康检查端点，区分 liveness 和 readiness │
│ 6. 使用 Prometheus 指标监控业务关键指标 │
│ 7. Grafana 大盘集中展示系统健康状况 │
│ 8. 异常处理中间件统一处理错误响应 │
└─────────────────────────────────────────────────────────────────────┘

技术选型建议

参考资源

• Microsoft.Extensions.Logging 官方文档
• Serilog 官方文档
• Prometheus .NET 客户端
• ASP.NET Core 健康检查
• OpenTelemetry .NET
• Grafana Dashboard 最佳实践

关注引导

如果本文对你有帮助，欢迎：

• 点赞 👍：让更多人看到这篇文章
• 收藏 ⭐：方便日后查阅
• 评论 💬：分享你的实践经验或问题
• 关注 📢：获取更多 C# 企业级开发教程

系列文章持续更新中，关注不迷路！

下一篇预告

下一篇：配置管理实战

将深入讲解：

• IConfiguration 配置系统原理
• 多环境配置管理（Development、Staging、Production）
• 配置热更新与强类型配置
• Azure Key Vault 与配置加密
• 配置验证与默认值处理
• 自定义配置源实现

敬请期待！

标签： C# .NET Core 日志 监控 Serilog Prometheus Grafana 可观测性 微服务 企业级开发