[.NET] Aspire Dashboard: 云原生可观测性

Aspire Dashboard 遥测数据采集机制详解

概述

.NET Aspire Dashboard是一个专门为分布式应用程序设计的可观测性平台，它通过OpenTelemetry协议(OTLP)采集和展示应用程序的日志(Logs)、指标(Metrics)和追踪(Traces)三大类遥测数据。Dashboard同时支持gRPC和HTTP两种传输协议，为应用程序提供了灵活的数据上报方式。

整体架构

Aspire Dashboard的遥测数据采集架构主要包含以下几个核心组件：

graph TB
subgraph "应用程序/服务"
A1["日志生成"]
A2["指标生成"]
A3["追踪生成"]
A1 --> OTEL
A2 --> OTEL
A3 --> OTEL
OTEL["OpenTelemetry SDK<br/>(OTLP Export)"]
end

OTEL -->|HTTP/gRPC| DASHBOARD

subgraph DASHBOARD ["Aspire Dashboard"]
subgraph ENDPOINTS ["OTLP 接收端点"]
subgraph HTTP_EP ["HTTP 端点"]
H1["/v1/logs"]
H2["/v1/traces"]
H3["/v1/metrics"]
end

subgraph GRPC_EP ["gRPC 端点"]
G1["OtlpGrpcLogsService"]
G2["OtlpGrpcTraceService"]
G3["OtlpGrpcMetricsService"]
end
end

ENDPOINTS --> STORAGE

subgraph STORAGE ["数据处理与存储层"]
subgraph REPO ["TelemetryRepository"]
subgraph STORES ["存储组件"]
S1["Logs 存储<br/>CircularBuffer&lt;OtlpLogEntry&gt;"]
S2["Metrics 存储<br/>Resource Based Storage"]
S3["Traces 存储<br/>CircularBuffer&lt;OtlpTrace&gt;"]
end
end
end

STORAGE --> UI

subgraph UI ["Dashboard UI"]
U1["日志查看器"]
U2["指标图表"]
U3["追踪分析器"]
end
end

style A1 fill:#e1f5fe
style A2 fill:#e8f5e8
style A3 fill:#fff3e0
style OTEL fill:#f3e5f5
style S1 fill:#e1f5fe
style S2 fill:#e8f5e8
style S3 fill:#fff3e0

核心组件详解

1. OTLP接收端点

Dashboard提供两种接收遥测数据的方式：

HTTP端点 (OtlpHttpEndpointsBuilder)

• 路径: /v1/logs, /v1/traces, /v1/metrics
• 支持格式: Protocol Buffers (application/x-protobuf)
• CORS支持: 可配置跨域资源共享
• 认证: 支持API Key认证

// HTTP端点映射示例
group.MapPost("logs", static (MessageBindable<ExportLogsServiceRequest> request, OtlpLogsService service) =>
{
    if (request.Message == null)
    {
        return Results.Empty;
    }
    return OtlpResult.Response(service.Export(request.Message));
});

gRPC端点

• 服务: OtlpGrpcLogsService, OtlpGrpcTraceService, OtlpGrpcMetricsService
• 协议: OpenTelemetry标准gRPC服务
• 性能: 更高效的二进制传输

[Authorize(Policy = OtlpAuthorization.PolicyName)]
public class OtlpGrpcLogsService : LogsService.LogsServiceBase
{
    public override Task<ExportLogsServiceResponse> Export(ExportLogsServiceRequest request, ServerCallContext context)
    {
        return Task.FromResult(_logsService.Export(request));
    }
}

2. 数据处理服务

日志处理 (OtlpLogsService)

public ExportLogsServiceResponse Export(ExportLogsServiceRequest request)
{
    var addContext = new AddContext();
    _telemetryRepository.AddLogs(addContext, request.ResourceLogs);

    return new ExportLogsServiceResponse
    {
        PartialSuccess = new ExportLogsPartialSuccess
        {
            RejectedLogRecords = addContext.FailureCount
        }
    };
}

追踪处理 (OtlpTraceService)

public ExportTraceServiceResponse Export(ExportTraceServiceRequest request)
{
    var addContext = new AddContext();
    _telemetryRepository.AddTraces(addContext, request.ResourceSpans);

    return new ExportTraceServiceResponse
    {
        PartialSuccess = new ExportTracePartialSuccess
        {
            RejectedSpans = addContext.FailureCount
        }
    };
}

指标处理 (OtlpMetricsService)

public ExportMetricsServiceResponse Export(ExportMetricsServiceRequest request)
{
    var addContext = new AddContext();
    _telemetryRepository.AddMetrics(addContext, request.ResourceMetrics);

    return new ExportMetricsServiceResponse
    {
        PartialSuccess = new ExportMetricsPartialSuccess
        {
            RejectedDataPoints = addContext.FailureCount
        }
    };
}

3. 数据存储层 (TelemetryRepository)

TelemetryRepository是Dashboard的核心数据管理组件，负责：

存储结构

• 日志存储: CircularBuffer<OtlpLogEntry> - 循环缓冲区，FIFO方式管理
• 追踪存储: CircularBuffer<OtlpTrace> - 支持容量管理和自动清理
• 指标存储: 基于Resource的存储模式
• 资源管理: ConcurrentDictionary<ResourceKey, OtlpResource>

数据容量管理

public sealed class TelemetryLimitOptions
{
    public int MaxLogCount { get; set; } = 10_000;        // 最大日志条数
    public int MaxTraceCount { get; set; } = 10_000;      // 最大追踪条数
    public int MaxMetricsCount { get; set; } = 50_000;    // 最大指标点数
    public int MaxAttributeCount { get; set; } = 128;     // 最大属性数量
    public int MaxAttributeLength { get; set; } = int.MaxValue; // 最大属性长度
    public int MaxSpanEventCount { get; set; } = int.MaxValue;  // 最大Span事件数
}

数据插入机制

日志插入：支持乱序插入和时间戳排序

public void AddLogsCore(AddContext context, OtlpResourceView resourceView, RepeatedField<ScopeLogs> scopeLogs)
{
    _logsLock.EnterWriteLock();
    try
    {
        foreach (var record in sl.LogRecords)
        {
            var logEntry = new OtlpLogEntry(record, resourceView, scope, _otlpContext);

            // 基于时间戳插入到正确位置
            var added = false;
            for (var i = _logs.Count - 1; i >= 0; i--)
            {
                if (logEntry.TimeStamp > _logs[i].TimeStamp)
                {
                    _logs.Insert(i + 1, logEntry);
                    added = true;
                    break;
                }
            }
            if (!added)
            {
                _logs.Insert(0, logEntry);
            }
        }
    }
    finally
    {
        _logsLock.ExitWriteLock();
    }
}

4. 实时订阅机制

Dashboard使用发布-订阅模式实现实时数据更新：

// 订阅管理
private readonly List<Subscription> _resourceSubscriptions = new();
private readonly List<Subscription> _logSubscriptions = new();
private readonly List<Subscription> _metricsSubscriptions = new();
private readonly List<Subscription> _tracesSubscriptions = new();

// 触发订阅更新
private void RaiseSubscriptionChanged(List<Subscription> subscriptions)
{
    lock (_lock)
    {
        foreach (var subscription in subscriptions)
        {
            subscription.TryExecute();
        }
    }
}

5. 暂停管理 (PauseManager)

支持暂停数据采集功能，避免在调试期间数据过载：

public void AddLogs(AddContext context, RepeatedField<ResourceLogs> resourceLogs)
{
    if (_pauseManager.AreStructuredLogsPaused(out _))
    {
        _logger.LogTrace("{Count} incoming structured log(s) ignored because of an active pause.", resourceLogs.Count);
        return;
    }
    // ... 处理日志
}

6. 组件关系图

graph TD
subgraph "Dashboard Web Application"
DWA[DashboardWebApplication] --> OTLP_HTTP[OtlpHttpEndpointsBuilder]
DWA --> OTLP_GRPC[gRPC Services]
DWA --> CONFIG[Configuration]

subgraph "OTLP Services"
OTLP_HTTP --> LS[OtlpLogsService]
OTLP_HTTP --> TS[OtlpTraceService]
OTLP_HTTP --> MS[OtlpMetricsService]

OTLP_GRPC --> GLS[OtlpGrpcLogsService]
OTLP_GRPC --> GTS[OtlpGrpcTraceService]
OTLP_GRPC --> GMS[OtlpGrpcMetricsService]

GLS --> LS
GTS --> TS
GMS --> MS
end

subgraph "Data Layer"
LS --> TR[TelemetryRepository]
TS --> TR
MS --> TR

TR --> PM[PauseManager]
TR --> CB1[CircularBuffer&lt;Logs&gt;]
TR --> CB2[CircularBuffer&lt;Traces&gt;]
TR --> RM[Resource Manager]
TR --> SUB[Subscription System]
end

subgraph "UI Components"
SUB --> LV[Log Viewer]
SUB --> MV[Metrics Viewer]
SUB --> TV[Trace Viewer]
end
end

subgraph "Host Integration"
RP[ResourcePublisher] --> DC[DashboardClient]
DC --> TR
end

style DWA fill:#f9f,stroke:#333,stroke-width:3px
style TR fill:#bbf,stroke:#333,stroke-width:2px
style SUB fill:#bfb,stroke:#333,stroke-width:2px

配置与端点

端点配置 (OtlpOptions)

public sealed class OtlpOptions
{
    public string? PrimaryApiKey { get; set; }          // 主API密钥
    public string? SecondaryApiKey { get; set; }        // 备用API密钥
    public OtlpAuthMode? AuthMode { get; set; }         // 认证模式
    public string? GrpcEndpointUrl { get; set; }        // gRPC端点URL
    public string? HttpEndpointUrl { get; set; }        // HTTP端点URL
    public OtlpCors Cors { get; set; } = new();         // CORS配置
}

CORS配置

public sealed class OtlpCors
{
    public string? AllowedOrigins { get; set; }   // 允许的来源域名
    public string? AllowedHeaders { get; set; }   // 允许的请求头

    [MemberNotNullWhen(true, nameof(AllowedOrigins))]
    public bool IsCorsEnabled => !string.IsNullOrEmpty(AllowedOrigins);
}

应用程序配置

Dashboard在DashboardWebApplication中进行完整的服务配置：

// 注册OTLP服务
builder.Services.AddSingleton<TelemetryRepository>();
builder.Services.AddTransient<OtlpLogsService>();
builder.Services.AddTransient<OtlpTraceService>();
builder.Services.AddTransient<OtlpMetricsService>();

// 配置gRPC
builder.Services.AddGrpc();

// 映射端点
_app.MapHttpOtlpApi(dashboardOptions.Otlp);      // HTTP端点
_app.MapGrpcService<OtlpGrpcMetricsService>();   // gRPC指标服务
_app.MapGrpcService<OtlpGrpcTraceService>();     // gRPC追踪服务
_app.MapGrpcService<OtlpGrpcLogsService>();      // gRPC日志服务

与Aspire宿主的集成

资源发布 (ResourcePublisher)

Dashboard通过ResourcePublisher与Aspire宿主通信，获取应用程序资源信息：

internal sealed class ResourcePublisher
{
    private readonly Dictionary<string, SourceAndResourceSnapshot> _snapshot = [];
    private ImmutableHashSet<Channel<ResourceSnapshotChange>> _outgoingChannels = [];

    // 集成资源变更并广播给订阅者
    internal async ValueTask IntegrateAsync(IResource source, ResourceSnapshot snapshot, ResourceSnapshotChangeType changeType)
    {
        lock (_syncLock)
        {
            switch (changeType)
            {
                case ResourceSnapshotChangeType.Upsert:
                    _snapshot[snapshot.Name] = new SourceAndResourceSnapshot(source, snapshot);
                    break;
                case ResourceSnapshotChangeType.Delete:
                    _snapshot.Remove(snapshot.Name);
                    break;
            }
        }

        // 通知所有订阅者
        foreach (var channel in channels)
        {
            await channel.Writer.WriteAsync(new(changeType, snapshot), cancellationToken);
        }
    }
}

Dashboard客户端 (DashboardClient)

实现与资源服务的gRPC通信：

internal sealed class DashboardClient : IDashboardClient
{
    private readonly Dictionary<string, ResourceViewModel> _resourceByName = new();
    private readonly GrpcChannel? _channel;
    private Aspire.DashboardService.Proto.V1.DashboardService.DashboardServiceClient? _client;

    // 订阅资源变更
    public async IAsyncEnumerable<IReadOnlyList<ResourceViewModelChange>> SubscribeResourcesAsync()
    {
        // 通过gRPC流式接收资源更新
    }
}

数据流转过程

1. 数据接收流程

sequenceDiagram
participant App as 应用程序
participant OTLP as OTLP Exporter
participant HTTP as HTTP端点
participant GRPC as gRPC端点
participant Service as 数据处理服务
participant Repo as TelemetryRepository
participant Storage as 存储层

App->>OTLP: 生成遥测数据
OTLP->>HTTP: HTTP/JSON 请求
OTLP->>GRPC: gRPC 调用

HTTP->>Service: OtlpLogsService.Export()
GRPC->>Service: OtlpGrpcLogsService.Export()

Service->>Repo: AddLogs/AddTraces/AddMetrics
Repo->>Storage: 写入CircularBuffer
Storage-->>Repo: 存储确认
Repo-->>Service: 处理结果
Service-->>HTTP: ExportResponse
Service-->>GRPC: ExportResponse

2. 数据查询流程

sequenceDiagram
participant UI as Dashboard UI
participant Sub as 订阅系统
participant Repo as TelemetryRepository
participant Filter as 数据过滤器
participant Storage as 存储层

UI->>Sub: 订阅数据更新
Sub->>Repo: 注册订阅回调

loop 实时数据查询
UI->>Repo: 查询遥测数据
Repo->>Filter: 应用过滤条件
Filter->>Storage: 读取数据
Storage-->>Filter: 返回数据
Filter-->>Repo: 过滤后数据
Repo-->>UI: 返回结果
end

Note over Storage: 数据变更时
Storage->>Repo: 触发变更事件
Repo->>Sub: 通知订阅者
Sub->>UI: 实时推送更新

3. 实时更新机制

flowchart LR
A[数据变更] --> B[触发订阅]
B --> C[推送到UI组件]
C --> D[前端实时刷新]

subgraph 订阅管理
E[ResourceSubscriptions]
F[LogSubscriptions]
G[MetricsSubscriptions]
H[TracesSubscriptions]
end

B --> E
B --> F
B --> G
B --> H

性能优化特性

1. 内存管理

• 循环缓冲区: 自动清理老数据，避免内存泄漏
• 容量限制: 可配置的数据条数上限
• 分段锁: 使用ReaderWriterLockSlim减少锁争用

2. 数据压缩

• HTTP压缩: 支持响应压缩
• Protocol Buffers: 高效的二进制序列化

3. 异步处理

• 流式处理: 支持大批量数据的流式处理
• 异步订阅: 非阻塞的实时数据推送

总结

Aspire Dashboard采用了模块化的架构设计，通过标准的OpenTelemetry协议接收遥测数据，使用高效的存储机制和实时订阅模式，为分布式应用程序提供了完整的可观测性解决方案。其核心优势包括：

1. 标准化: 完全基于OpenTelemetry标准，确保与各种应用程序的兼容性
2. 高性能: 使用循环缓冲区和异步处理，支持高吞吐量的数据采集
3. 实时性: 基于订阅模式的实时数据更新机制
4. 可配置: 灵活的配置选项，支持不同的部署场景
5. 多协议: 同时支持HTTP和gRPC两种传输协议

该架构为.NET生态系统中的分布式应用程序监控和调试提供了强大的基础设施支持。