Kubernetes 部署

1. 概述

Kubernetes (K8s) 是生产环境部署 RabbitMQ 集群的理想平台，提供自动扩缩容、滚动更新、自愈等能力。本章将详细介绍在 Kubernetes 上部署 RabbitMQ 的多种方式，包括手动部署、Helm Chart 部署和 Operator 部署。

2. 系统要求

2.1 Kubernetes 版本要求

组件	最低版本	推荐版本
Kubernetes	1.21+	1.28+
kubectl	1.21+	1.28+
Helm (可选)	3.0+	3.12+

2.2 集群资源要求

部署类型	节点数	CPU	内存
开发环境	1	2 核	4 GB
测试环境	3	4 核	8 GB
生产环境	3+	8 核	16 GB

2.3 检查 Kubernetes 环境

bash

kubectl version --short
kubectl cluster-info
kubectl get nodes
kubectl get storageclass

3. 手动部署

3.1 创建命名空间

yaml

apiVersion: v1
kind: Namespace
metadata:
  name: rabbitmq
  labels:
    app: rabbitmq

bash

kubectl apply -f namespace.yaml

3.2 创建 Secret

yaml

apiVersion: v1
kind: Secret
metadata:
  name: rabbitmq-secret
  namespace: rabbitmq
type: Opaque
stringData:
  username: admin
  password: YourStrongPassword123
  erlang-cookie: RABBITMQ_CLUSTER_COOKIE_SECRET

bash

kubectl apply -f secret.yaml

3.3 创建 ConfigMap

yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: rabbitmq-config
  namespace: rabbitmq
data:
  rabbitmq.conf: |
    listeners.tcp.default = 5672
    management.tcp.port = 15672
    default_user = admin
    default_pass = YourStrongPassword123
    loopback_users = none
    vm_memory_high_watermark.relative = 0.6
    disk_free_limit.absolute = 1GB
    log.file.level = info
    log.console = true
    log.console.level = info
    cluster_formation.peer_discovery_backend = rabbit_peer_discovery_k8s
    cluster_formation.k8s.host = kubernetes.default.svc.cluster.local
    cluster_formation.k8s.address_type = hostname
    cluster_formation.node_cleanup.interval = 10
    cluster_formation.node_cleanup.only_log_warning = true
    cluster_partition_handling = autoheal

  enabled_plugins: |
    [rabbitmq_management,rabbitmq_peer_discovery_k8s].

bash

kubectl apply -f configmap.yaml

3.4 创建 Service

yaml

apiVersion: v1
kind: Service
metadata:
  name: rabbitmq-headless
  namespace: rabbitmq
  labels:
    app: rabbitmq
spec:
  clusterIP: None
  ports:
    - name: amqp
      port: 5672
      targetPort: 5672
    - name: management
      port: 15672
      targetPort: 15672
    - name: epmd
      port: 4369
      targetPort: 4369
    - name: cluster
      port: 25672
      targetPort: 25672
  selector:
    app: rabbitmq
---
apiVersion: v1
kind: Service
metadata:
  name: rabbitmq
  namespace: rabbitmq
  labels:
    app: rabbitmq
spec:
  type: ClusterIP
  ports:
    - name: amqp
      port: 5672
      targetPort: 5672
    - name: management
      port: 15672
      targetPort: 15672
  selector:
    app: rabbitmq

bash

kubectl apply -f service.yaml

3.5 创建 StatefulSet

yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: rabbitmq
  namespace: rabbitmq
  labels:
    app: rabbitmq
spec:
  serviceName: rabbitmq-headless
  replicas: 3
  selector:
    matchLabels:
      app: rabbitmq
  template:
    metadata:
      labels:
        app: rabbitmq
    spec:
      terminationGracePeriodSeconds: 60
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app: rabbitmq
              topologyKey: "kubernetes.io/hostname"
      containers:
        - name: rabbitmq
          image: rabbitmq:3.13-management
          imagePullPolicy: IfNotPresent
          ports:
            - name: amqp
              containerPort: 5672
            - name: management
              containerPort: 15672
            - name: epmd
              containerPort: 4369
            - name: cluster
              containerPort: 25672
          env:
            - name: RABBITMQ_NODENAME
              value: rabbit@$(HOSTNAME).rabbitmq-headless.rabbitmq.svc.cluster.local
            - name: RABBITMQ_ERLANG_COOKIE
              valueFrom:
                secretKeyRef:
                  name: rabbitmq-secret
                  key: erlang-cookie
            - name: RABBITMQ_DEFAULT_USER
              valueFrom:
                secretKeyRef:
                  name: rabbitmq-secret
                  key: username
            - name: RABBITMQ_DEFAULT_PASS
              valueFrom:
                secretKeyRef:
                  name: rabbitmq-secret
                  key: password
          volumeMounts:
            - name: rabbitmq-data
              mountPath: /var/lib/rabbitmq
            - name: rabbitmq-config
              mountPath: /etc/rabbitmq/rabbitmq.conf
              subPath: rabbitmq.conf
            - name: rabbitmq-config
              mountPath: /etc/rabbitmq/enabled_plugins
              subPath: enabled_plugins
          resources:
            requests:
              cpu: "500m"
              memory: "512Mi"
            limits:
              cpu: "2000m"
              memory: "2Gi"
          livenessProbe:
            exec:
              command:
                - rabbitmqctl
                - status
            initialDelaySeconds: 60
            periodSeconds: 30
            timeoutSeconds: 10
            failureThreshold: 3
          readinessProbe:
            exec:
              command:
                - rabbitmqctl
                - status
            initialDelaySeconds: 20
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 3
      volumes:
        - name: rabbitmq-config
          configMap:
            name: rabbitmq-config
  volumeClaimTemplates:
    - metadata:
        name: rabbitmq-data
        namespace: rabbitmq
      spec:
        accessModes:
          - ReadWriteOnce
        storageClassName: standard
        resources:
          requests:
            storage: 10Gi

bash

kubectl apply -f statefulset.yaml

3.6 部署验证

bash

kubectl get pods -n rabbitmq -w
kubectl get statefulset -n rabbitmq
kubectl get svc -n rabbitmq

kubectl logs -n rabbitmq rabbitmq-0 -f
kubectl exec -n rabbitmq rabbitmq-0 -- rabbitmqctl cluster_status

4. Helm Chart 部署

4.1 添加 Helm 仓库

bash

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
helm search repo bitnami/rabbitmq

4.2 创建 values.yaml

yaml

auth:
  username: admin
  password: YourStrongPassword123
  erlangCookie: RABBITMQ_CLUSTER_COOKIE_SECRET

replicaCount: 3

persistence:
  enabled: true
  storageClass: "standard"
  size: 10Gi

resources:
  limits:
    cpu: 2000m
    memory: 2Gi
  requests:
    cpu: 500m
    memory: 512Mi

service:
  type: ClusterIP
  port: 5672
  managementPort: 15672

ingress:
  enabled: false

metrics:
  enabled: true
  serviceMonitor:
    enabled: false

extraConfiguration: |
  vm_memory_high_watermark.relative = 0.6
  disk_free_limit.absolute = 1GB
  log.file.level = info
  log.console = true
  log.console.level = info

podAntiAffinityPreset: hard
nodeAffinityPreset:
  type: ""
  key: ""
  values: []

tolerations: []

nodeSelector: {}

affinity: {}

priorityClassName: ""

topologySpreadConstraints: []

terminationGracePeriodSeconds: 60

livenessProbe:
  enabled: true
  initialDelaySeconds: 60
  periodSeconds: 30
  timeoutSeconds: 10
  failureThreshold: 3

readinessProbe:
  enabled: true
  initialDelaySeconds: 20
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 3

4.3 安装 RabbitMQ

bash

helm install rabbitmq bitnami/rabbitmq \
  -n rabbitmq \
  --create-namespace \
  -f values.yaml

4.4 自定义安装

bash

helm install rabbitmq bitnami/rabbitmq \
  -n rabbitmq \
  --create-namespace \
  --set auth.username=admin \
  --set auth.password=YourStrongPassword123 \
  --set auth.erlangCookie=RABBITMQ_CLUSTER_COOKIE_SECRET \
  --set replicaCount=3 \
  --set persistence.size=20Gi \
  --set resources.limits.memory=4Gi \
  --set service.type=LoadBalancer

4.5 升级和回滚

bash

helm upgrade rabbitmq bitnami/rabbitmq \
  -n rabbitmq \
  -f values.yaml

helm rollback rabbitmq 1 -n rabbitmq

helm history rabbitmq -n rabbitmq

4.6 卸载

bash

helm uninstall rabbitmq -n rabbitmq
kubectl delete pvc -l app.kubernetes.io/name=rabbitmq -n rabbitmq
kubectl delete namespace rabbitmq

5. RabbitMQ Operator 部署

5.1 安装 Operator

bash

kubectl apply -f https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml

kubectl get pods -n rabbitmq-system

5.2 创建 RabbitmqCluster

yaml

apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
  name: rabbitmq-cluster
  namespace: rabbitmq
spec:
  replicas: 3
  image: rabbitmq:3.13-management
  
  resources:
    requests:
      cpu: 500m
      memory: 512Mi
    limits:
      cpu: 2000m
      memory: 2Gi
  
  persistence:
    storageClassName: standard
    storage: 10Gi
  
  rabbitmq:
    additionalConfig: |
      vm_memory_high_watermark.relative = 0.6
      disk_free_limit.absolute = 1GB
      log.file.level = info
      log.console = true
      log.console.level = info
    
    additionalPlugins:
      - rabbitmq_management
      - rabbitmq_prometheus
      - rabbitmq_peer_discovery_k8s
  
  service:
    type: ClusterIP
  
  override:
    service:
      metadata:
        annotations:
          prometheus.io/scrape: "true"
          prometheus.io/port: "15692"
  
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchLabels:
              app.kubernetes.io/name: rabbitmq-cluster
          topologyKey: kubernetes.io/hostname

bash

kubectl apply -f rabbitmq-cluster.yaml

5.3 查看集群状态

bash

kubectl get rabbitmqcluster -n rabbitmq
kubectl describe rabbitmqcluster rabbitmq-cluster -n rabbitmq
kubectl get pods -n rabbitmq -l app.kubernetes.io/name=rabbitmq-cluster

5.4 创建用户

yaml

apiVersion: rabbitmq.com/v1beta1
kind: User
metadata:
  name: rabbitmq-user
  namespace: rabbitmq
spec:
  rabbitmqClusterReference:
    name: rabbitmq-cluster
  username: app-user
  password:
    secretKeyRef:
      name: app-user-password
      key: password
  tags:
    - management
  permissions:
    - vhost: "/"
      configure: ".*"
      write: ".*"
      read: ".*"

5.5 创建队列

yaml

apiVersion: rabbitmq.com/v1beta1
kind: Queue
metadata:
  name: test-queue
  namespace: rabbitmq
spec:
  name: test-queue
  vhost: "/"
  rabbitmqClusterReference:
    name: rabbitmq-cluster
  durable: true
  autoDelete: false

6. 高级配置

6.1 Ingress 配置

yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: rabbitmq-ingress
  namespace: rabbitmq
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - rabbitmq.example.com
      secretName: rabbitmq-tls
  rules:
    - host: rabbitmq.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: rabbitmq
                port:
                  number: 15672

6.2 PodDisruptionBudget

yaml

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: rabbitmq-pdb
  namespace: rabbitmq
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: rabbitmq

6.3 NetworkPolicy

yaml

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: rabbitmq-network-policy
  namespace: rabbitmq
spec:
  podSelector:
    matchLabels:
      app: rabbitmq
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              name: default
        - podSelector:
            matchLabels:
              app: myapp
      ports:
        - protocol: TCP
          port: 5672
        - protocol: TCP
          port: 15672
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: rabbitmq
      ports:
        - protocol: TCP
          port: 4369
        - protocol: TCP
          port: 25672
    - to:
        - namespaceSelector: {}
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53

6.4 HorizontalPodAutoscaler

yaml

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: rabbitmq-hpa
  namespace: rabbitmq
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: StatefulSet
    name: rabbitmq
  minReplicas: 3
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80

6.5 Prometheus 监控

yaml

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: rabbitmq-monitor
  namespace: rabbitmq
  labels:
    release: prometheus
spec:
  selector:
    matchLabels:
      app: rabbitmq
  endpoints:
    - port: management
      path: /metrics
      interval: 30s
      scrapeTimeout: 10s

7. 常见问题与解决方案

7.1 Pod 无法启动

问题表现：

Pod 一直处于 Pending 或 CrashLoopBackOff 状态

排查步骤：

bash

kubectl describe pod rabbitmq-0 -n rabbitmq
kubectl logs rabbitmq-0 -n rabbitmq
kubectl get events -n rabbitmq --sort-by='.lastTimestamp'

解决方案：

bash

# 检查 PVC 状态
kubectl get pvc -n rabbitmq

# 检查 StorageClass
kubectl get storageclass

# 检查资源限制
kubectl describe node <node-name>

7.2 集群节点无法加入

问题表现：

Error: {badrpc,nodedown}

解决方案：

bash

# 检查 Erlang Cookie
kubectl exec -n rabbitmq rabbitmq-0 -- cat /var/lib/rabbitmq/.erlang.cookie
kubectl exec -n rabbitmq rabbitmq-1 -- cat /var/lib/rabbitmq/.erlang.cookie

# 检查 Service 解析
kubectl exec -n rabbitmq rabbitmq-0 -- nslookup rabbitmq-headless

# 重置节点
kubectl exec -n rabbitmq rabbitmq-1 -- rabbitmqctl stop_app
kubectl exec -n rabbitmq rabbitmq-1 -- rabbitmqctl reset
kubectl exec -n rabbitmq rabbitmq-1 -- rabbitmqctl join_cluster rabbit@rabbitmq-0.rabbitmq-headless.rabbitmq.svc.cluster.local
kubectl exec -n rabbitmq rabbitmq-1 -- rabbitmqctl start_app

7.3 PVC 无法绑定

问题表现：

PVC 一直处于 Pending 状态

解决方案：

bash

# 检查 StorageClass
kubectl get storageclass

# 检查是否有默认 StorageClass
kubectl get storageclass -o jsonpath='{.items[?(@.metadata.annotations.storageclass\.kubernetes\.io/is-default-class=="true")].metadata.name}'

# 创建 StorageClass 或指定 storageClassName

7.4 内存告警

问题表现：

memory resource limit alarm set on node

解决方案：

修改 ConfigMap 或 values.yaml：

yaml

extraConfiguration: |
  vm_memory_high_watermark.relative = 0.4
  total_memory_available_override_value = 4GB

7.5 管理界面无法访问

问题表现：

访问 Service 或 Ingress 无响应

解决方案：

bash

# 检查 Service
kubectl get svc -n rabbitmq
kubectl describe svc rabbitmq -n rabbitmq

# 检查端口
kubectl get endpoints -n rabbitmq

# 端口转发测试
kubectl port-forward -n rabbitmq svc/rabbitmq 15672:15672

# 检查 Ingress
kubectl get ingress -n rabbitmq
kubectl describe ingress rabbitmq-ingress -n rabbitmq

8. 验证方法

8.1 检查部署状态

bash

kubectl get all -n rabbitmq
kubectl get statefulset -n rabbitmq
kubectl get pods -n rabbitmq -o wide
kubectl get pvc -n rabbitmq

8.2 检查集群状态

bash

kubectl exec -n rabbitmq rabbitmq-0 -- rabbitmqctl cluster_status
kubectl exec -n rabbitmq rabbitmq-0 -- rabbitmqctl list_nodes
kubectl exec -n rabbitmq rabbitmq-0 -- rabbitmqctl node_health_check

8.3 检查服务连通性

bash

# 端口转发
kubectl port-forward -n rabbitmq svc/rabbitmq 5672:5672 15672:15672

# 测试连接
curl -u admin:YourStrongPassword123 http://localhost:15672/api/overview

8.4 检查日志

bash

kubectl logs -n rabbitmq rabbitmq-0 -f
kubectl logs -n rabbitmq rabbitmq-0 --previous

8.5 进入容器

bash

kubectl exec -n rabbitmq -it rabbitmq-0 -- /bin/bash
kubectl exec -n rabbitmq -it rabbitmq-0 -- rabbitmqctl status

9. 运维命令速查

bash

# 查看 Pod
kubectl get pods -n rabbitmq -w

# 查看日志
kubectl logs -n rabbitmq rabbitmq-0 -f

# 进入容器
kubectl exec -n rabbitmq -it rabbitmq-0 -- /bin/bash

# 执行 rabbitmqctl 命令
kubectl exec -n rabbitmq rabbitmq-0 -- rabbitmqctl status
kubectl exec -n rabbitmq rabbitmq-0 -- rabbitmqctl list_users
kubectl exec -n rabbitmq rabbitmq-0 -- rabbitmqctl list_queues
kubectl exec -n rabbitmq rabbitmq-0 -- rabbitmqctl cluster_status

# 扩缩容
kubectl scale statefulset rabbitmq -n rabbitmq --replicas=5

# 重启 Pod
kubectl rollout restart statefulset rabbitmq -n rabbitmq

# 删除 Pod（StatefulSet 会自动重建）
kubectl delete pod rabbitmq-0 -n rabbitmq

# 端口转发
kubectl port-forward -n rabbitmq svc/rabbitmq 5672:5672 15672:15672

# Helm 操作
helm list -n rabbitmq
helm status rabbitmq -n rabbitmq
helm upgrade rabbitmq bitnami/rabbitmq -n rabbitmq -f values.yaml
helm rollback rabbitmq 1 -n rabbitmq

Kubernetes 部署 ​

1. 概述 ​

2. 系统要求 ​

2.1 Kubernetes 版本要求 ​

2.2 集群资源要求 ​

2.3 检查 Kubernetes 环境 ​

3. 手动部署 ​

3.1 创建命名空间 ​

3.2 创建 Secret ​

3.3 创建 ConfigMap ​

3.4 创建 Service ​

3.5 创建 StatefulSet ​

3.6 部署验证 ​

4. Helm Chart 部署 ​

4.1 添加 Helm 仓库 ​

4.2 创建 values.yaml ​

4.3 安装 RabbitMQ ​

4.4 自定义安装 ​

4.5 升级和回滚 ​

4.6 卸载 ​

5. RabbitMQ Operator 部署 ​

5.1 安装 Operator ​

5.2 创建 RabbitmqCluster ​

5.3 查看集群状态 ​

5.4 创建用户 ​

5.5 创建队列 ​

6. 高级配置 ​

6.1 Ingress 配置 ​

6.2 PodDisruptionBudget ​

6.3 NetworkPolicy ​

6.4 HorizontalPodAutoscaler ​

6.5 Prometheus 监控 ​

7. 常见问题与解决方案 ​

7.1 Pod 无法启动 ​

7.2 集群节点无法加入 ​

7.3 PVC 无法绑定 ​

7.4 内存告警 ​

7.5 管理界面无法访问 ​

8. 验证方法 ​

8.1 检查部署状态 ​

8.2 检查集群状态 ​

8.3 检查服务连通性 ​

8.4 检查日志 ​

8.5 进入容器 ​

9. 运维命令速查 ​

10. 相关链接 ​

Kubernetes 部署

1. 概述

2. 系统要求

2.1 Kubernetes 版本要求

2.2 集群资源要求

2.3 检查 Kubernetes 环境

3. 手动部署

3.1 创建命名空间

3.2 创建 Secret

3.3 创建 ConfigMap

3.4 创建 Service

3.5 创建 StatefulSet

3.6 部署验证

4. Helm Chart 部署

4.1 添加 Helm 仓库

4.2 创建 values.yaml

4.3 安装 RabbitMQ

4.4 自定义安装

4.5 升级和回滚

4.6 卸载

5. RabbitMQ Operator 部署

5.1 安装 Operator

5.2 创建 RabbitmqCluster

5.3 查看集群状态

5.4 创建用户

5.5 创建队列

6. 高级配置

6.1 Ingress 配置

6.2 PodDisruptionBudget

6.3 NetworkPolicy

6.4 HorizontalPodAutoscaler

6.5 Prometheus 监控

7. 常见问题与解决方案

7.1 Pod 无法启动

7.2 集群节点无法加入

7.3 PVC 无法绑定

7.4 内存告警

7.5 管理界面无法访问

8. 验证方法

8.1 检查部署状态

8.2 检查集群状态

8.3 检查服务连通性

8.4 检查日志

8.5 进入容器

9. 运维命令速查

10. 相关链接