Akka-CQRS（10）- gRPC on SSL/TLS 安全连接

使用gRPC作为云平台和移动前端的连接方式，网络安全应该是必须考虑的一个重点。gRPC是支持ssl/tls安全通讯机制的。用了一个周末来研究具体使用方法，实际上是一个周末的挖坑填坑过程。把这次经历记录下来与各位分享。

gRPC的ssl/tls的原理是在服务端安装安全证书公用certificate和私钥key, 在客户端安装公共证书就可以了，gRPC代码是这样写的：

// Server
SslContext sslContext =  SslContextBuilder.forServer(
      new File("/Users/u/Desktop/api.grpc/src/main/resources/my-public-key-cert.pem"),
      new File("/Users/u/Desktop/api.grpc/src/main/resources/my-private-key.pem"))
      .build();

server = NettyServerBuilder.forPort(port).sslContext(sslContext)
      .addService(GreeterGrpc.bindService(new GreeterImpl())).build()
      .start();

/ Client
SslContext sslContext = SslContextBuilder.forClient().trustManager(new File(
                  "/Users/u/Desktop/api.grpc/src/main/resources/my-public-key-cert.pem")).build();
      channel = NettyChannelBuilder.forAddress(host, port)
            .sslContext(sslContext)
            .build();
      blockingStub = GreeterGrpc.newBlockingStub(channel);

先构建SslContextBuilder，然后在构建NettyServerBuilder和NettyChannelBuilder时加入sslContext。上面的my-public-key-cert.pem,my-private_key.pem是用openssl产生的：

openssl req -x509 -newkey rsa:4096 -keyout my-private-key.pem -out my-public-key-cert.pem -days 365 -nodes -subj '/CN=localhost'

不过使用这个证书和私钥测试时出现了错误：

Jan 27, 2019 4:08:22 PM io.grpc.netty.GrpcSslContexts defaultSslProvider
INFO: netty-tcnative unavailable (this may be normal)
java.lang.ClassNotFoundException: io.netty.internal.tcnative.SSL
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
...
INFO: Jetty ALPN unavailable (this may be normal)
java.lang.ClassNotFoundException: org/eclipse/jetty/alpn/ALPN
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
...

仔细研究了一下github上的gRPC-java说明文件SECURITY.MD，感觉应该是grpc和netty版本问题，特别是下面这几个依赖：

Find the dependency tree (e.g., mvn dependency:tree), and look for versions of:

io.grpc:grpc-netty
io.netty:netty-handler (really, make sure all of io.netty except for netty-tcnative has the same version)
io.netty:netty-tcnative-boringssl-static:jar

...

grpc-netty version	netty-handler version	netty-tcnative-boringssl-static version
1.0.0-1.0.1	4.1.3.Final	1.1.33.Fork19
1.0.2-1.0.3	4.1.6.Final	1.1.33.Fork23
1.1.x-1.3.x	4.1.8.Final	1.1.33.Fork26
1.4.x	4.1.11.Final	2.0.1.Final
1.5.x	4.1.12.Final	2.0.5.Final
1.6.x	4.1.14.Final	2.0.5.Final
1.7.x-1.8.x	4.1.16.Final	2.0.6.Final
1.9.x-1.10.x	4.1.17.Final	2.0.7.Final
1.11.x-1.12.x	4.1.22.Final	2.0.7.Final
1.13.x	4.1.25.Final	2.0.8.Final
1.14.x-1.15.x	4.1.27.Final	2.0.12.Final
1.16.x-1.17.x	4.1.30.Final	2.0.17.Final
1.18.x-1.19.x	4.1.32.Final	2.0.20.Final
1.20.x-1.21x	4.1.34.Final	2.0.22.Final
1.22.x-	4.1.35.Final	2.0.25.Final

解决问题必须先搞清楚这些库的版本。这个可以用sbt插件sbt-dependency-graph。加入project/plugins.sbt：

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.9")
addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.9.2")
addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.3.15")
addSbtPlugin("com.thesamet" % "sbt-protoc" % "0.99.21")
addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.9.2")
libraryDependencies += "com.thesamet.scalapb" %% "compilerplugin" % "0.9.0-M6"

在sbt中执行dependencyTree：

~/scala/intellij/learn-grpc> sbt
[info] Loading settings for project global-plugins from idea.sbt ...
sbt:learn-grpc> dependencyTree
[info]   | | +-com.google.protobuf:protobuf-java:3.7.1
[info]   | | +-io.grpc:grpc-api:1.21.0
[info]   | | +-io.netty:netty-handler:4.1.34.Final
[info]   | | | +-io.netty:netty-buffer:4.1.34.Final
[info]   | | | | +-io.netty:netty-common:4.1.34.Final
...

好像缺失了io.netty:netty-tcnative-boringssl-static:jar，按照对应的gRPC版本在build.sbt里加上：

name := "learn-grpc"

version := "0.1"

scalaVersion := "2.12.8"

scalacOptions += "-Ypartial-unification"

val akkaversion = "2.5.23"

libraryDependencies := Seq(
  "com.typesafe.akka" %% "akka-cluster-metrics" % akkaversion,
  "com.typesafe.akka" %% "akka-cluster-sharding" % akkaversion,
  "com.typesafe.akka" %% "akka-persistence" % akkaversion,
  "com.lightbend.akka" %% "akka-stream-alpakka-cassandra" % "1.0.1",
  "org.mongodb.scala" %% "mongo-scala-driver" % "2.6.0",
  "com.lightbend.akka" %% "akka-stream-alpakka-mongodb" % "1.0.1",
  "com.typesafe.akka" %% "akka-persistence-query" % akkaversion,
  "com.typesafe.akka" %% "akka-persistence-cassandra" % "0.97",
  "com.datastax.cassandra" % "cassandra-driver-core" % "3.6.0",
  "com.datastax.cassandra" % "cassandra-driver-extras" % "3.6.0",
  "ch.qos.logback"  %  "logback-classic"   % "1.2.3",
  "io.monix" %% "monix" % "3.0.0-RC2",
  "org.typelevel" %% "cats-core" % "2.0.0-M1",
  "io.grpc" % "grpc-netty" % scalapb.compiler.Version.grpcJavaVersion,
  "io.netty" % "netty-tcnative-boringssl-static" % "2.0.22.Final",
  "com.thesamet.scalapb" %% "scalapb-runtime" % scalapb.compiler.Version.scalapbVersion % "protobuf",
  "com.thesamet.scalapb" %% "scalapb-runtime-grpc" % scalapb.compiler.Version.scalapbVersion

)

// (optional) If you need scalapb/scalapb.proto or anything from
// google/protobuf/*.proto
//libraryDependencies += "com.thesamet.scalapb" %% "scalapb-runtime" % scalapb.compiler.Version.scalapbVersion % "protobuf"

PB.targets in Compile := Seq(
  scalapb.gen() -> (sourceManaged in Compile).value
)

enablePlugins(JavaAppPackaging)

试了一下启动服务，现在不出错误了（构建sslContext成功了）。不过客户端在使用了证书后仍然无法连接到服务端。没办法，又要再去查资料了。看来现在应该是证书的问题了。先看看是不是因为使用的证书是自签的self-signed-certificate。grpc-java里提供了一些测试用的证书和私钥和说明文档。在测试程序里使用了它们提供的server1.pem,server1.key,ca.pem：

package learn.grpc.server
import io.grpc.{ServerBuilder,ServerServiceDefinition}
import io.grpc.netty.NettyServerBuilder

import java.io._

trait gRPCServer {

  val serverCrtFile = new File("/Users/tiger/certs/server1.pem")
  val serverKeyFile = new File("/Users/tiger/certs/server1.key")

  def runServer(service: ServerServiceDefinition): Unit = {
    val server = NettyServerBuilder
      .forPort(50051)
      .addService(service)
      .useTransportSecurity(serverCrtFile,serverKeyFile)
      .build
      .start
    // make sure our server is stopped when jvm is shut down
    Runtime.getRuntime.addShutdownHook(new Thread() {
      override def run(): Unit = server.shutdown()
    })

    server.awaitTermination()
  }

}

...

package learn.grpc.sum.one2many.client
import java.io.File

import io.grpc.stub.StreamObserver
import demo.services.sum._
import io.grpc.netty.{GrpcSslContexts, NegotiationType, NettyChannelBuilder}

object One2ManyClient {

  def main(args: Array[String]): Unit = {

    val clientCrtFile = new File("/Users/tiger/certs/ca.pem")  // new File(getClass.getClassLoader.getResource("badserver.pem").getPath)

    val sslContextBuilder = GrpcSslContexts.forClient().trustManager(clientCrtFile)

    //build connection channel
    val channel = NettyChannelBuilder
      .forAddress("192.168.0.189",50051)
      .negotiationType(NegotiationType.TLS)
      .sslContext(sslContextBuilder.build())
      .overrideAuthority("foo.test.google.fr")
      .build()

    // get asyn stub
    val client: SumOneToManyGrpc.SumOneToManyStub = SumOneToManyGrpc.stub(channel)
    // prepare stream observer
    val streamObserver = new StreamObserver[SumResponse] {
      override def onError(t: Throwable): Unit = println(s"error: ${t.getMessage}")
      override def onCompleted(): Unit = println("Done incrementing !!!")
      override def onNext(value: SumResponse): Unit = println(s"current value: ${value.currentResult}")
    }
    // call service with stream observer
    client.addOneToMany(SumRequest().withToAdd(6),streamObserver)

    // wait for async execution
    scala.io.StdIn.readLine()
  }

}

连接成功了。判断正确，是证书的问题。再研究一下证书是怎么产生的，尝试按文档指引重新产生这些自签证书：可惜的是好像还有些文件是缺失的，如serial。那么上面的.overrideAuthority("foo.test.google.fr")又是什么意思呢？算了，以后有时间再研究吧。这次起码证明grpc ssl/tls是可以发挥作用的。