consul dev用起來都很正常，但開了加密和校驗，難度翻了好幾倍。

首先看一下consul connect enovy的日志：

2022-10-12 10:38:10.418][45382][info][config] [source/server/configuration_impl.cc:97] loading 1 cluster(s) [2022-10-12 10:38:10.522][45382][info][config] [source/server/configuration_impl.cc:101] loading 0 listener(s) [2022-10-12 10:38:10.522][45382][info][config] [source/server/configuration_impl.cc:113] loading stats configuration [2022-10-12 10:38:10.523][45382][info][runtime] [source/common/runtime/runtime_impl.cc:463] RTDS has finished initialization [2022-10-12 10:38:10.523][45382][info][upstream] [source/common/upstream/cluster_manager_impl.cc:221] cm init: initializing cds [2022-10-12 10:38:10.526][45382][warning][main] [source/server/server.cc:784] there is no configured limit to the number of allowed active connections. Set a limit via the runtime key overload.global_downstream_max_connections [2022-10-12 10:38:10.527][45382][info][main] [source/server/server.cc:905] starting main dispatch loop [2022-10-12 10:38:25.523][45382][warning][config] [source/common/config/grpc_subscription_impl.cc:118] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.cluster.v3.Cluster [2022-10-12 10:38:25.523][45382][info][upstream] [source/common/upstream/cluster_manager_impl.cc:225] cm init: all clusters initialized [2022-10-12 10:38:25.523][45382][info][main] [source/server/server.cc:886] all clusters initialized. initializing init manager [2022-10-12 10:38:40.524][45382][warning][config] [source/common/config/grpc_subscription_impl.cc:118] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.listener.v3.Listener [2022-10-12 10:38:40.524][45382][info][config] [source/server/listener_manager_impl.cc:841] all dependencies initialized. starting workers [2022-10-12 10:39:06.293][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 55s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:39:18.840][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 68s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:39:48.373][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 97s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:39:57.092][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 106s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:40:10.912][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 120s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:40:23.269][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 132s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:40:50.591][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 160s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:41:04.465][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 173s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:41:20.063][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 189s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:41:35.213][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 204s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:41:40.294][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 209s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:41:54.596][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 224s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:42:11.217][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 240s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:42:24.340][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 253s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:42:30.040][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 259s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:42:33.164][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 262s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination [2022-10-12 10:42:58.807][45382][warning][config] [./source/common/config/grpc_stream.h:196] DeltaAggregatedResources gRPC config stream closed since 288s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination

DeltaAggregatedResources gRPC config stream closed since 55s ago: 14, upstream connect error or disconnect/reset before headers. reset reason: connection termination

consul的日志：

2022-10-12T10:38:10.915+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50406": tls: first record does not look like a TLS handshake 2022-10-12T10:38:12.529+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50408": tls: first record does not look like a TLS handshake 2022-10-12T10:38:12.549+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50410": tls: first record does not look like a TLS handshake 2022-10-12T10:38:14.088+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50414": tls: first record does not look like a TLS handshake 2022-10-12T10:38:16.652+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50416": tls: first record does not look like a TLS handshake 2022-10-12T10:38:17.414+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50418": tls: first record does not look like a TLS handshake 2022-10-12T10:38:22.881+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50424": tls: first record does not look like a TLS handshake 2022-10-12T10:38:31.523+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50430": tls: first record does not look like a TLS handshake 2022-10-12T10:38:36.600+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50436": tls: first record does not look like a TLS handshake 2022-10-12T10:38:36.811+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50438": tls: first record does not look like a TLS handshake 2022-10-12T10:38:40.302+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50440": tls: first record does not look like a TLS handshake 2022-10-12T10:38:45.743+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50446": tls: first record does not look like a TLS handshake 2022-10-12T10:39:06.293+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50468": tls: first record does not look like a TLS handshake 2022-10-12T10:39:10.438+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50470": tls: first record does not look like a TLS handshake 2022-10-12T10:39:18.839+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50480": tls: first record does not look like a TLS handshake 2022-10-12T10:39:27.425+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50488": tls: first record does not look like a TLS handshake 2022-10-12T10:39:34.695+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50498": tls: first record does not look like a TLS handshake 2022-10-12T10:39:48.373+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50504": tls: first record does not look like a TLS handshake 2022-10-12T10:39:52.666+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50512": tls: first record does not look like a TLS handshake 2022-10-12T10:39:57.091+0800 [WARN] agent: [core]grpc: Server.Serve failed to complete security handshake from "127.0.0.1:50516": tls: first record does not look like a TLS handshake

首先，consul的配置很重要，一開始我也是這樣想的，后面把研究反向對到envoy身上，為啥envoy集群起不來。

consul：

Consul = 1.13.2

Envoy=1.23.0
ACLs = Enabled
TLS = Enabled

client的配置：記得grpc端口要，默認好像是不開的。

"ca_file": "/etc/ssl/certs/foobar-consul-ca.pem","cert_file": "/etc/consul/client1.dc1.consul.pem","key_file": "/etc/consul/client1.dc1.consul.key","connect": {"enabled": true},"ports": {"grpc": 8502,"https": 8501}, consul connect envoy --sidecar-for foobar -admin-bind localhost:19000

還有添加系統環境

CONSUL_HTTP_SSL=true CONSUL_HTTP_ADDR=127.0.0.1:8501 CONSUL_CACERT=/etc/ssl/certs/consul-ca.pem CONSUL_CLIENT_CERT=/etc/consul/client1.dc1.consul.pem CONSUL_CLIENT_KEY=/etc/consul/client1.dc1.consul.key

這個點，基本都能在網上找到。但其實還差了些。

CONSUL_GRPC_ADDR=https://127.0.0.1:8502

少了這個，因為envoy是要用到grpc的，而且consul的grpc專門就是給envoy 的xdc 用的。

就這樣就可以跑起來的話，就沒下面的什么事情了。

但我情況有點怪，系統環境感覺不生效，直接輸出的都正常，就是grpc有問題。我也想過在connect命令行里面加更多的配置，但把token和ca的參數都帶上，還是不行。

consul connect envoy \-grpc-addr=https://localhost:8502 \-ca-file=/etc/consul.d/ssl.ca.d/ssl.chain.pem \-client-cert=/etc/consul.d/ssl.crt.pem \-client-key=/etc/consul.d/ssl.key.pem \-http-addr=https://localhost:8501 \-tls-server-name=localhost \-token=... \-admin-bind 127.0.0.1:19005 \-envoy-version=1.14.2 \-sidecar-for some-service

看到別人說的一個例子，試了一下，居然正常了。其實本來想著consul來直接帶著一堆參數來輔助調用envoy，但還是要把tls的參數在命令行帶過去。

consul的文檔變得有點快，但看起來越來越清晰，之前看1.13.1的文檔，配置啥的東一塊西一塊，陷阱都有說，但不是在一起！！！

參考：

Envoy Proxy breaks when enabling Consul TLS · Issue #7926 · hashicorp/consul · GitHub

環境參數? ? Commands | Consul | HashiCorp Developer

總結

以上是生活随笔為你收集整理的consul connect envoy 启动acls和tls后，grpc连接异常的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。