EMQX：高性能开源MQTT消息 broker，赋能物联网通信

什么是EMQX？

EMQX是一款基于Erlang/OTP平台开发的开源MQTT消息 broker，专为物联网场景设计。它支持MQTT、MQTT-SN、CoAP、WebSocket等多种协议，能够处理大规模的设备连接和海量消息吞吐，是构建物联网通信平台的理想选择。

部署EMQX

使用 EMQX Operator 安装 EMQX

删除之前的sa

1
eksctl delete iamserviceaccount --name cert-manager --cluster pre-us --namespace cert-manager

安装cert-manager

1
2
3
4
5
6
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm upgrade --install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--set installCRDs=true

安装emqx-operator

1
2
3
4
5
helm repo add emqx https://repos.emqx.io/charts
helm repo update
helm upgrade --install emqx-operator emqx/emqx-operator \
--namespace emqx-operator-system \
--create-namespace

配置emqx的StorageClass

1
2
3
4
5
6
7
8
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
parameters:
  type: gp3

StorageClass设为默认

1
kubectl patch storageclass gp3 -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

安装EMQX并配置TLS

使用的是aws的NLB做TLS负载均衡器，SSL验证放在NLB上解析

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
apiVersion: v1
kind: Namespace
metadata:
  name: emqx
---
apiVersion: apps.emqx.io/v2beta1
kind: EMQX
metadata:
  name: emqx
  namespace: emqx
spec:
  image: emqx:5.7.1
  coreTemplate:
    spec:
      replicas: 3
      ## EMQX custom resources do not support updating this field at runtime   
      volumeClaimTemplates:
        ## More content: https://docs.aws.amazon.com/eks/latest/userguide/storage-classes.html
        ## Please manage the Amazon EBS CSI driver as an Amazon EKS add-on.
        ## For more documentation please refer to: https://docs.aws.amazon.com/zh_cn/eks/latest/userguide/managing-ebs-csi.html
        storageClassName: gp3
        resources:
          requests:
            storage: 500Gi
        accessModes:
          - ReadWriteOnce
  dashboardServiceTemplate:
    metadata:
      ## More content: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/service/annotations/
      annotations:
        ## Specifies whether the NLB is Internet-facing or internal. If not specified, defaults to internal.
        service.beta.kubernetes.io/aws-load-balancer-scheme: internal
        ## Specify the availability zone to which the NLB will route traffic. Specify at least one subnet, either subnetID or subnetName (subnet name label) can be used.
       # service.beta.kubernetes.io/aws-load-balancer-subnets: subnet-xxx1,subnet-xxx2
        service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-west-2:xxxxx:certificate/xxxx
        ## Specifies whether to use TLS for the backend traffic between the load balancer and the kubernetes pods.
        service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
        ## but direct access to K8S service port does not require TLS authentication
        service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "18083"
    spec:
      type: LoadBalancer
      ## More content: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/service/nlb/
      loadBalancerClass: service.k8s.aws/nlb
  listenersServiceTemplate:
    metadata:
      ## More content: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/service/annotations/
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
        service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    spec:
      type: LoadBalancer
      ## More content: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/service/nlb/
      loadBalancerClass: service.k8s.aws/nlb
      ports:
      ##- name: ssl-default
      ##  nodePort: 31173
      ##  port: 8883
      ##  protocol: TCP
      ##  targetPort: 8883
      - name: tcp-default
        port: 8883
        protocol: TCP
        targetPort: 8883
      - name: tcp-default-2
        port: 1883
        protocol: TCP
        targetPort: 1883
      - name: ws-default
        port: 8084
        protocol: TCP
        targetPort: 8084
      #- name: wss-default
      #  nodePort: 30437
      #  port: 8084
      #  protocol: TCP
      #  targetPort: 8084

修改Dashboard的账号密码

进入emqx的容器

1
emqx_ctl admins passwd admin xxxx

升级emqx节点后，驱逐已停止节点

1
2
3
4
5
6
# 查看节点状态
emqx ctl cluster status --json
# 驱逐停止的节点
emqx ctl cluster force-leave emqx@emqx-core-d894447cf-0.emqx-headless.emqx.svc.cluster.local
emqx ctl cluster force-leave emqx@emqx-core-d894447cf-1.emqx-headless.emqx.svc.cluster.local
emqx ctl cluster force-leave emqx@emqx-core-d894447cf-2.emqx-headless.emqx.svc.cluster.local