# Docker 部署 Keepalived 2.3.4 实现三节点高可用 VIP 本文档详细说明如何在三台机器上通过 Docker 部署 Keepalived 2.3.4,实现 VIP(${VIP_IP})的高可用,并通过仲裁节点防止脑裂问题。 ## 1. 节点信息 | 节点名称 | IP 地址 | 角色 | 基础优先级 | 备注 | | --- | --- | --- | --- | --- | | Node A | ${NODE_A_IP} | 主节点 (Master) | 110 | MySQL 故障优先级扣 50 → 60 | | Node B | ${NODE_B_IP} | 备节点 (Backup) | 100 | MySQL 故障优先级扣 50 → 50 | | Node C | ${NODE_C_IP} | 仲裁节点 (Arbiter) | 40 | 参与选举但不持有 VIP | + **VIP 地址**:${VIP_IP} + **仲裁节点作用**:Node C 仅参与选举流程,防止 Node A 和 Node B 之间发生脑裂,正常场景下不持有 VIP。 ## 2. 构建 Keepalived 镜像(解决 libip4tc 报错) 采用多阶段构建方式,确保镜像最小化且包含所有运行时依赖。 ### 2.1 Dockerfile 内容 ```dockerfile # Stage 1: 构建 Keepalived 二进制文件 FROM alpine:3.23.2 AS builder # 安装构建依赖 RUN apk add --no-cache \ gcc \ make \ libc-dev \ openssl-dev \ libnl3-dev \ ipset-dev \ iptables-dev \ libnfnetlink-dev \ libnftnl-dev \ libmnl-dev \ net-snmp-dev \ libssh2-dev \ pcre2-dev \ autoconf \ automake \ linux-headers \ curl \ tar # 下载并编译 Keepalived 2.3.4 ENV KEEPALIVED_VERSION=2.3.4 RUN curl -sSL https://www.keepalived.org/software/keepalived-${KEEPALIVED_VERSION}.tar.gz -o keepalived.tar.gz && \ tar -zxf keepalived.tar.gz && \ cd keepalived-${KEEPALIVED_VERSION} && \ ./configure --prefix=/usr --sysconfdir=/etc && \ make && \ make install DESTDIR=/install # Stage 2: 构建最终运行镜像 FROM alpine:3.23.2 # 安装运行时依赖 RUN apk add --no-cache \ libnl3 \ ipset \ iptables \ libnftnl \ libmnl \ libnfnetlink \ net-snmp-libs \ libssh2 \ pcre2 \ openssl \ ca-certificates \ tzdata \ libip4tc \ libip6tc # 从构建阶段复制编译好的二进制文件 COPY --from=builder /install / # 创建默认配置目录 RUN mkdir -p /etc/keepalived # 暴露 VRRP 协议端口 (112) EXPOSE 112 # 前台运行 Keepalived(适配 Docker 容器运行模式) # --dont-fork: 不后台运行 # --log-console: 日志输出到控制台(便于 Docker 日志查看) ENTRYPOINT ["keepalived", "--dont-fork", "--log-console"] ``` ### 2.2 构建镜像命令 ```bash docker build -t keepalived:2.3.4 . ``` ## 3. 配置文件部署 ### 3.1 前置准备 在三台机器上统一创建配置目录: ```bash mkdir -p /data/keepalived ``` ### 3.2 Docker Compose 配置(三台机器通用) 创建 `/data/keepalived/docker-compose.yml`: ```yaml services: keepalived: image: keepalived:2.3.4 container_name: keepalived restart: always network_mode: host # 主机网络模式,确保 VRRP 协议正常通信 cap_add: # 添加网络管理权限 - NET_ADMIN - NET_BROADCAST - NET_RAW volumes: - /data/keepalived/keepalived.conf:/etc/keepalived/keepalived.conf:ro # 只读挂载配置文件 - /data/keepalived/check-mysql.sh:/etc/keepalived/check-mysql.sh:ro # 挂载 MySQL 检测脚本(仅 Node A/B 需要) ``` ### 3.3 各节点 Keepalived 配置 #### 3.3.1 Node A(${NODE_A_IP})配置 路径:`/data/keepalived/keepalived.conf` ```nginx global_defs { router_id ${ROUTER_ID_A} vrrp_skip_check_adv_addr script_user root # 指定脚本执行用户 enable_script_security # 启用脚本安全校验,防止执行未授权脚本 } # MySQL 存活检测脚本定义 vrrp_script check_mysql { script "/etc/keepalived/check-mysql.sh" interval 3 # 检测间隔:3 秒 weight -50 # 检测失败时,优先级扣 50 fall 3 # 连续 3 次失败判定为故障 rise 2 # 连续 2 次成功判定为恢复 } vrrp_instance VI_1 { state BACKUP # 所有节点统一设为 BACKUP,靠优先级决定 Master interface ${INTERFACE} # 替换为宿主机实际网卡名(如 ens33/br0 等) virtual_router_id ${VIRTUAL_ROUTER_ID} # 虚拟路由 ID,集群内必须一致(1-255) priority 110 # 基础优先级 preempt_delay 30 # 优先级恢复后,延迟 30 秒抢占 VIP(减少抖动) advert_int 1 # VRRP 通告间隔:1 秒 # 认证配置(集群内必须一致) authentication { auth_type PASS auth_pass ${AUTH_PASS} } # 单播配置(替代组播,适配部分网络环境) unicast_src_ip ${NODE_A_IP} # 本机 IP unicast_peer { # 集群其他节点 IP ${NODE_B_IP} ${NODE_C_IP} } # 虚拟 IP 配置 virtual_ipaddress { ${VIP_IP}/32 # VIP 地址(32 位掩码) } # 绑定检测脚本 track_script { check_mysql } } ``` #### 3.3.2 Node B(${NODE_B_IP})配置 路径:`/data/keepalived/keepalived.conf` ```nginx global_defs { router_id ${ROUTER_ID_B} vrrp_skip_check_adv_addr script_user root enable_script_security } # MySQL 存活检测脚本定义 vrrp_script check_mysql { script "/etc/keepalived/check-mysql.sh" interval 3 weight -50 fall 3 rise 2 } vrrp_instance VI_1 { state BACKUP interface ${INTERFACE} # 替换为宿主机实际网卡名 virtual_router_id ${VIRTUAL_ROUTER_ID} priority 100 # 基础优先级(低于 Node A) advert_int 1 authentication { auth_type PASS auth_pass ${AUTH_PASS} } unicast_src_ip ${NODE_B_IP} unicast_peer { ${NODE_A_IP} ${NODE_C_IP} } virtual_ipaddress { ${VIP_IP}/32 } track_script { check_mysql } } ``` #### 3.3.3 Node C(${NODE_C_IP},仲裁节点)配置 路径:`/data/keepalived/keepalived.conf` ```nginx global_defs { router_id ${ROUTER_ID_C} vrrp_skip_check_adv_addr } vrrp_instance VI_1 { state BACKUP interface ${INTERFACE} # 替换为宿主机实际网卡名 virtual_router_id ${VIRTUAL_ROUTER_ID} priority 40 # 最低优先级(仅参与选举,不持有 VIP) advert_int 1 authentication { auth_type PASS auth_pass ${AUTH_PASS} } unicast_src_ip ${NODE_C_IP} unicast_peer { ${NODE_A_IP} ${NODE_B_IP} } # 仲裁节点不配置 virtual_ipaddress,不持有 VIP } ``` ### 3.4 MySQL 检测脚本(仅 Node A/B 需要) 路径:`/data/keepalived/check-mysql.sh` ```bash #!/bin/sh # 检测逻辑: # 1. timeout 2:限制命令 2 秒内完成,避免脚本阻塞 # 2. nc -z:仅检测端口连通性(无数据传输) # 3. 屏蔽所有输出,仅返回退出码 if timeout 2 nc -z 127.0.0.1 3306 > /dev/null 2>&1; then # 端口连通,返回 0(Keepalived 判定正常) exit 0 else # 端口不通/超时,返回 1(Keepalived 触发优先级扣减) exit 1 fi ``` 添加执行权限: ```bash chmod +x /data/keepalived/check-mysql.sh ``` ## 4. 启动与验证 ### 4.1 启动容器 三台机器执行相同命令: ```bash cd /data/keepalived docker-compose up -d ``` ### 4.2 日志查看 ```bash docker logs -f keepalived ``` > 正常日志会显示 VRRP 选举过程、VIP 绑定/释放、脚本检测结果等信息。 > ### 4.3 VIP 验证 在任意节点执行,查看 VIP 是否绑定: ```bash ip addr show ${INTERFACE} # 替换为实际网卡名 ``` + 正常情况下,VIP(${VIP_IP})会绑定在优先级最高的节点(初始为 Node A)。 + 输出示例:`inet ${VIP_IP}/32 scope global ${INTERFACE}` ## 5. 故障场景分析 详细的故障场景模拟与 VIP 切换逻辑分析,请参阅:[Keepalived 故障场景分析](./fault-analysis.md) ## 6. 核心总结 ### 6.1 优先级规则 + VIP 归属由 Keepalived **有效优先级** 决定:基础优先级 A(110) > B(100) > C(40); + MySQL 故障会让 A/B 优先级扣 50,Keepalived 离线则失去选举资格。 ### 6.2 仲裁节点作用 + Node C 仅参与选举流程,不持有 VIP; + 核心作用是防止 A、B 脑裂,正常/单节点故障场景下不影响 A/B 选举逻辑。 ### 6.3 风险与优化 + **风险**:A/B MySQL 均故障但 Keepalived 正常时,VIP 仍绑定到 A,此时无可用 MySQL;需在业务层增加数据库存活检测,避免连接失效的 VIP。 + **抢占优化**:A 恢复后会在 `preempt_delay`(30 秒)后抢占 VIP,若需避免业务抖动,可在 Node A 配置中添加 `nopreempt` 关闭抢占。 ### 6.4 关键配置注意事项 + 所有节点 `virtual_router_id` 和认证信息必须一致; + 网卡名需替换为宿主机实际名称; + 仲裁节点 C 不配置 `virtual_ipaddress`,不挂载 MySQL 检测脚本。