ASP.NET Core – Logging & Serilog

前言

以前就写过了 Asp.net core 学习笔记 (Logging 日志), 只是有点乱, 这篇作为整理版.

参考：

docs – Logging in .NET Core and ASP.NET Core

Github – Serilog.AspNetCore

Setting up Serilog in .NET 6

ASP.NET Core Build-in Logging

ASP.NET Core 提供了 Logging 的抽象接口, third party 都会依据抽象来做实现. ASP.NET Core 自己也实现了一套简单的 log, 它只能 log to console. 不能 log to file.

所以绝大部分项目都会搭配一个 third party library, 比如 Serilog.

我们先看看 ASP.NET Core build-in 的 log.

Quick Example for Web

dotnet new webapp -o TestLog

自带的模板首页就有 inject logger 了, 在 OnGet 使用它就可以了.

打开 Visual Studio > View > Output, F5 run 就可以看见了

VS Code

Quick Example for Console Application

安装

dotnet add package Microsoft.Extensions.Logging.Console

main.cs

public class Program
{
  public static async Task Main()
  {
    using var loggerFactory = LoggerFactory.Create(builder =>
    {
      builder
        .AddFilter("Microsoft", LogLevel.Warning)
        .AddFilter("System", LogLevel.Warning)
        .AddFilter("LoggingConsoleApp.Program", LogLevel.Debug)
        .AddConsole();
    });
    ILogger logger = loggerFactory.CreateLogger<Program>();
    logger.LogInformation("Example log message");
  }
}

Log Category

每个 log 都有 category 的概念, 在 appsetting.json 可以为每个 category 设置 min level

比如某 category 在 production 的时候 min level 是 warning, 在 dev 的时候是 info.

像上面的注入方式, 它的 category name 是 namespace + class "TestLog.Pages.IndexModel"

如果想自定义的话可以用 factory 来创建 logger

在 appsetting.json 声明 min level

level 的顺序是 Trace = 0, Debug = 1, Information = 2, Warning = 3, Error = 4, Critical = 5, and None = 6.

Serilog level 的顺序是 Verbose = 0, Debug = 1, Information = 2, Warning = 3, Error = 4, Fatal = 5

None 就是不要 log. Trace 就类似 track 追踪的意思

Serilog 安装

dotnet add package Serilog.AspNetCore

program.cs

var builder = WebApplication.CreateBuilder(args);
builder.Host.UseSerilog();

Log.Logger = new LoggerConfiguration()
            .MinimumLevel.Override("Microsoft", LogEventLevel.Information)
            .Enrich.FromLogContext()
            .WriteTo.Console()
            .CreateLogger();
try
{
    Log.Information("Starting web host");
    app.Run(); // 把 app.Run(); 放进来
    return 0;
}
catch (Exception ex)
{
    Log.Fatal(ex, "Host terminated unexpectedly");
    return 1;
}
finally
{
    Log.CloseAndFlush();
}

Serilog 的 config 不是通过 appsetting 设置的, 如果想用 appsetting 来管理可以另外安装一个 DLL, 下面会介绍.

appsetting 里的 log config 是给 ASP.NET Core build-in log 的哦, Serilog 不会读取它来用.

Formatting Output

参考: Github serilog – Formatting Output

Log.Information 只是填写了 message, 但最终的 log 需要具备其它的资料比如时间, 而这些就有 output template 来完成.

下面这个是 ConsoleLogger 的默认模板

parameters definition:

我们可以通过 config 去调整它.

Log.Logger = new LoggerConfiguration()
    .WriteTo.Console(outputTemplate: "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj}{NewLine}{Exception}")
    .CreateLogger();

serilog-sinks-console 和 serilog-sinks-file 都支持 outputTemplate 设定. 但如果我们看源码会发现它们的实现方式不太一样.

console 源码显示, 它有自己实现一套 ITextFormatter

 

而 file 源码显示, 它用了 Serilog.Formatting.Display 的 MessageTemplateTextFormatter

 

Request Log

Serilog 有一个 build-in 的 request log 取代 ASP.NET Core build-in 的, 因为 ASP.NET Core log 太多了.

先关掉 ASP.NET Core 的

Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
    .MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Warning)

然后

app.UseSerilogRequestLogging(options =>
{
    options.MessageTemplate = "HTTP {RequestMethod} {RequestPath} responded {StatusCode} in {Elapsed:0} ms";
    options.GetLevel = (httpContext, elapsed, ex) => LogEventLevel.Information;
    options.EnrichDiagnosticContext = (diagnosticContext, httpContext) =>
    {
　　　　　// 这里可以加一点料, 加了 template 就可以用 {RequestHost} 这样
        // diagnosticContext.Set("RequestHost", httpContext.Request.Host.Value);
        // diagnosticContext.Set("RequestScheme", httpContext.Request.Scheme);
    };
});

通过这个 middleware 每个请求都会被记入下来. middleware 放在 static file middleware 下面, 不然 static file 也会被 log 就不好了.

想更 customize 一下可以参考这个 Adding Useful Information to ASP.NET Core Web API Serilog Logs

Write to File

安装

dotnet add package Serilog.Sinks.File

设置

Log.Logger = new LoggerConfiguration()
    .WriteTo.File("log.txt", rollingInterval: RollingInterval.Day)

write to file 是马上执行的, 这对 IO 性能不太好.

Async Write to File

所以 Serilog 推出了一个 async 版本. 它会先写入 ram, 等一定量之后才写入 file.

dotnet add package Serilog.Sinks.Async

设置

Log.Logger = new LoggerConfiguration()
 .WriteTo.Async(a => a.File("log.txt", rollingInterval: RollingInterval.Day, buffered: true), bufferSize: 500)

wrap 起来就可以了, buffered = true, buffer size 默认是 10,000 items 那是多少我也不清楚.

然后在 app 结束前一定要释放哦

try
{
    CreateHostBuilder(args).Build().Run();
}
catch (Exception ex)
{
    Log.Fatal(ex, "Host terminated unexpectedly");
}
finally
{
    Log.CloseAndFlush(); // 重要
}

Use appsetting.json as Config

安装

dotnet add package Serilog.Settings.Configuration

设置

Log.Logger = new LoggerConfiguration()
    .ReadFrom.Configuration(builder.Configuration) 

appsetting.json

 "Serilog": {
    "Using": [ "Serilog.Sinks.File" ], // 这个在 dotnet core project 下是多余的，可以拿掉哦
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Warning",
        "Microsoft.AspNetCore": "Warning",
        "System": "Warning"
      }
    },
    "WriteTo": [
      {
        "Name": "File",
        "Args": {
          "path": "log.txt",
          "rollingInterval": "Day",
          "outputTemplate": "[{Timestamp:hh:mm:ss tt} {Level:u3}] {Message:lj} {Properties:j}{NewLine}{Exception}"
        }
      },
      {
        "Name": "Async",
        "Args": {
          "configure": [
            {
              "Name": "File",
              "Args": {
                "path": "log-async.txt",
                "rollingInterval": "Day",
                "outputTemplate": "[{Timestamp:hh:mm:ss tt} {Level:u3}] {Message:lj} {Properties:j}{NewLine}{Exception}",
                "buffered": true
              }
            }
          ]
        }
      }
    ],
    "Enrich": [ "FromLogContext" ]
  }

Send Email on Error

如果程序不经常出错的话, 每天查看 log 是挺浪费时间的, 这时就可以改用 Email 通知. 当出现 error 时, 把 error log 发送到我们电邮.

Serilog 有一个插件可以完成这个事儿. 它叫 serilog-sinks-email

很遗憾的是, 我在试用的时候遇到了 SSL 的问题. 可能是它太久没有更新了吧. 它底层依赖 MailKit

而设定 SSL 的方式是 useSsl

这个是比较古老的 way 了. 在我之前写的教程中, 调用 MailKit 应该是下面这样的

所以只好放弃这个插件了. 不开 SSL 倒是可以正常发送, 代码如下

dotnet add package Serilog.Sinks.Email

program.cs

Log.Logger = new LoggerConfiguration()
    .WriteTo.Email(
            connectionInfo: new EmailConnectionInfo
            {
                EmailSubject = "System Error",
                EnableSsl = false,
                FromEmail = "dada@hotmail.com",
                Port = 587,
                ToEmail = "dada@gmail.com",
                MailServer = "smtp.office365.com",
                NetworkCredentials = new NetworkCredential
                {
                    UserName = "dada@hotmail.com",
                    Password = "dada"
                },
                ServerCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) => true
            },
        restrictedToMinimumLevel: LogEventLevel.Error
    )
    .CreateLogger();

相关 issues:

Stack Overflow – Serilog.Sinks.Email - No Indication of Failure?

Github – Serilog Email sink is not sending emails

Stack Overflow – SslHandshakeException: An error occurred while attempting to establish an SSL or TLS connection

Stack Overflow – Serilog Email sink enableSSL false checks

Github – Serilog Email Sink not working with .NET Core 5

Debug Sink

参考: Github – Debugging and Diagnostics

WriteTo File, Console, Email, Database 都好, by right 是不应该出现 error 的. 因为这个已经是 final 了. 但是在开发阶段任何会有 bug 的可能.

比如上面的 send email 就因为 SSL 报错了. 要 debug 它的话需要开启

Serilog.Debugging.SelfLog.Enable(Console.WriteLine);

这样它就会把 send email 时的错误写入 Console 里. then 用 VS Code DEBUG CONSOLE 就可以看见 Exception 了.

自定义 Sink

既然 serilog-sinks-email 满足不到我们的需求, 那就自己写一个吧.

参考: Developing a sink

首先定义 2 个 class

public class MySink : ILogEventSink
{
    public void Emit(LogEvent logEvent)
    {
        var message = logEvent.RenderMessage();
        var level = logEvent.Level;
        var now = logEvent.Timestamp.UtcDateTime;
        var exception = logEvent.Exception;
    }
}

public static class MySinkExtensions
{
    public static LoggerConfiguration MySink(this LoggerSinkConfiguration loggerConfiguration) => loggerConfiguration.Sink(new MySink());
}

然后调用

Log.Logger = new LoggerConfiguration()
    .WriteTo.MySink()
    .CreateLogger();

当 log 发生时 Emit 就会被调用了. 通过 logEvent 可以拿到 message, level, timespan, exception (如果 throw Exception 的话就有), 等等.

有了这些, 我们可以写入 file, 写入 Database, 发电子邮件等等.

但它有几个点要注意

1. Emit 不支持 async

Github – Asynchronous support throughout Serilog

Github – Changed sinks and loggers to async

C#同步方法中调用异步方法

How to call asynchronous method from synchronous method in C#?

2. 每一次 log 发生都会被调用 (过于频密).

所以, serilog-sinks-email 里面还依赖了 serilog-sinks-periodicbatching, 它的功能是把 log 搜集起来一次触发, 同时也支持了 async 写法.

主要就是替代掉了 ILogEventSink, 然后设定 config

3. 支持 Formatting Output, 可以参考上面的 Console 和 File 是如何支持 Formatting Output 的.

4. Emit 是会被并发调用的

所以参考, ConsoleSink.cs 和 RollingFileSink.cs 的实现. Emit 里面都加了 lock 来确保线程安全.

 

自定义 Sink for Send Email on Error

dotnet add package MailKit
dotnet add package Serilog.AspNetCore
dotnet add package Serilog.Sinks.PeriodicBatching

program.cs

using System.Security.Authentication;
using MailKit.Net.Smtp;
using MimeKit;
using Serilog;
using Serilog.Configuration;
using Serilog.Events;
using Serilog.Formatting;
using Serilog.Formatting.Display;
using Serilog.Sinks.PeriodicBatching;

namespace TestLogMySinkEmail;

class MyEmailSink : IBatchedLogEventSink
{
    private readonly ITextFormatter _textFormatter;
    public MyEmailSink(ITextFormatter textFormatter)
    {
        _textFormatter = textFormatter;
    }

    public async Task EmitBatchAsync(IEnumerable<LogEvent> logEvents)
    {
        var logEvent = logEvents.Single();
        var outputWriter = new StringWriter();
        _textFormatter.Format(logEvent, outputWriter);
        var output = outputWriter.ToString();

        using var client = new SmtpClient();
        await client.ConnectAsync(
            host: "smtp.office365.com",
            port: 587,
            options: MailKit.Security.SecureSocketOptions.StartTls
        );
        await client.AuthenticateAsync("dada@hotmail.com", "dada");
        client.SslProtocols = SslProtocols.Tls12 | SslProtocols.Tls13;

        var message = new MimeMessage();
        message.From.Add(new MailboxAddress("dada", "dada@hotmail.com"));
        message.To.Add(new MailboxAddress("dada87", "dada87@gmail.com"));
        var builder = new BodyBuilder
        {
            HtmlBody = $@"
        <!DOCTYPE html>
        <html lang=""en"" xmlns=""http://www.w3.org/1999/xhtml"" xmlns:o=""urn:schemas-microsoft-com:office:office"">
        <head>
            <meta charset=""UTF-8"">
            <meta name=""viewport"" content=""width=device-width,initial-scale=1"">
            <meta name=""x-apple-disable-message-reformatting"">
            <title></title>
            </head>
        <body style=""color:red"">
            <p>{output}</p>
        </body>
        </html>",
        };
        message.Subject = "Website Error Log";
        message.Body = builder.ToMessageBody();
        await client.SendAsync(message);
        await client.DisconnectAsync(quit: true);
    }

    public Task OnEmptyBatchAsync()
    {
        return Task.CompletedTask;
    }
}

public static class LoggerSinkConfigurationExtensions
{
    public static LoggerConfiguration MyEmailSink(
        this LoggerSinkConfiguration loggerSinkConfiguration,
        LogEventLevel restrictedToMinimumLevel = LogEventLevel.Error,
        string outputTemplate = "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj}{NewLine}{Exception}"
    )
    {
        var formatter = new MessageTemplateTextFormatter(outputTemplate);
        var myEmailSink = new MyEmailSink(formatter);
        var batchingOptions = new PeriodicBatchingSinkOptions
        {
            BatchSizeLimit = 100,
            Period = TimeSpan.FromSeconds(2),
            EagerlyEmitFirstEvent = true,
            QueueLimit = 10000,
        };

        var batchingSink = new PeriodicBatchingSink(myEmailSink, batchingOptions);
        return loggerSinkConfiguration.Sink(batchingSink, restrictedToMinimumLevel);
    }
}

public class Program
{
    public static void Main(string[] args)
    {
        var builder = WebApplication.CreateBuilder(args);
        builder.Host.UseSerilog();

        Log.Logger = new LoggerConfiguration()
            .MinimumLevel.Override("Microsoft", LogEventLevel.Information)
            .Enrich.FromLogContext()
            .WriteTo.Console(outputTemplate: "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj}{NewLine}{Exception}")
            .WriteTo.MyEmailSink(LogEventLevel.Error)
            .CreateLogger();

        builder.Services.AddRazorPages();

        var app = builder.Build();

        if (!app.Environment.IsDevelopment())
        {
            app.UseExceptionHandler("/Error");
            app.UseHsts();
        }

        app.UseHttpsRedirection();
        app.UseStaticFiles();

        app.UseRouting();

        app.UseAuthorization();

        app.MapRazorPages();

        try
        {
            app.Run();
        }
        catch (Exception ex)
        {
            Log.Fatal(ex, "Host terminated unexpectedly");
        }
        finally
        {
            Log.CloseAndFlush();
        }
    }
}

index.cshtml.cs

using Microsoft.AspNetCore.Mvc.RazorPages;

namespace TestLogMySinkEmail.Pages;

public class IndexModel : PageModel
{
    private readonly ILogger<IndexModel> _logger;

    public IndexModel(ILogger<IndexModel> logger)
    {
        _logger = logger;
    }

    public void OnGet()
    {
        throw new Exception("Error");
    }
}