Initial commit with project setup and basic structure established.
This commit is contained in:
@@ -0,0 +1,325 @@
|
||||
# Docker 部署 Keepalived 2.3.4 实现三节点高可用 VIP
|
||||
本文档详细说明如何在三台机器上通过 Docker 部署 Keepalived 2.3.4，实现 VIP（${VIP_IP}）的高可用，并通过仲裁节点防止脑裂问题。
|
||||
|
||||
## 1. 节点信息
|
||||
| 节点名称 | IP 地址 | 角色 | 基础优先级 | 备注 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Node A | ${NODE_A_IP} | 主节点 (Master) | 110 | MySQL 故障优先级扣 50 → 60 |
|
||||
| Node B | ${NODE_B_IP} | 备节点 (Backup) | 100 | MySQL 故障优先级扣 50 → 50 |
|
||||
| Node C | ${NODE_C_IP} | 仲裁节点 (Arbiter) | 40 | 参与选举但不持有 VIP |
|
||||
|
||||
|
||||
+ **VIP 地址**：${VIP_IP}
|
||||
+ **仲裁节点作用**：Node C 仅参与选举流程，防止 Node A 和 Node B 之间发生脑裂，正常场景下不持有 VIP。
|
||||
|
||||
## 2. 构建 Keepalived 镜像（解决 libip4tc 报错）
|
||||
采用多阶段构建方式，确保镜像最小化且包含所有运行时依赖。
|
||||
|
||||
### 2.1 Dockerfile 内容
|
||||
```dockerfile
|
||||
# Stage 1: 构建 Keepalived 二进制文件
|
||||
FROM alpine:3.23.2 AS builder
|
||||
|
||||
# 安装构建依赖
|
||||
RUN apk add --no-cache \
|
||||
gcc \
|
||||
make \
|
||||
libc-dev \
|
||||
openssl-dev \
|
||||
libnl3-dev \
|
||||
ipset-dev \
|
||||
iptables-dev \
|
||||
libnfnetlink-dev \
|
||||
libnftnl-dev \
|
||||
libmnl-dev \
|
||||
net-snmp-dev \
|
||||
libssh2-dev \
|
||||
pcre2-dev \
|
||||
autoconf \
|
||||
automake \
|
||||
linux-headers \
|
||||
curl \
|
||||
tar
|
||||
|
||||
# 下载并编译 Keepalived 2.3.4
|
||||
ENV KEEPALIVED_VERSION=2.3.4
|
||||
RUN curl -sSL https://www.keepalived.org/software/keepalived-${KEEPALIVED_VERSION}.tar.gz -o keepalived.tar.gz && \
|
||||
tar -zxf keepalived.tar.gz && \
|
||||
cd keepalived-${KEEPALIVED_VERSION} && \
|
||||
./configure --prefix=/usr --sysconfdir=/etc && \
|
||||
make && \
|
||||
make install DESTDIR=/install
|
||||
|
||||
# Stage 2: 构建最终运行镜像
|
||||
FROM alpine:3.23.2
|
||||
|
||||
# 安装运行时依赖
|
||||
RUN apk add --no-cache \
|
||||
libnl3 \
|
||||
ipset \
|
||||
iptables \
|
||||
libnftnl \
|
||||
libmnl \
|
||||
libnfnetlink \
|
||||
net-snmp-libs \
|
||||
libssh2 \
|
||||
pcre2 \
|
||||
openssl \
|
||||
ca-certificates \
|
||||
tzdata \
|
||||
libip4tc \
|
||||
libip6tc
|
||||
|
||||
# 从构建阶段复制编译好的二进制文件
|
||||
COPY --from=builder /install /
|
||||
|
||||
# 创建默认配置目录
|
||||
RUN mkdir -p /etc/keepalived
|
||||
|
||||
# 暴露 VRRP 协议端口 (112)
|
||||
EXPOSE 112
|
||||
|
||||
# 前台运行 Keepalived（适配 Docker 容器运行模式）
|
||||
# --dont-fork: 不后台运行
|
||||
# --log-console: 日志输出到控制台（便于 Docker 日志查看）
|
||||
ENTRYPOINT ["keepalived", "--dont-fork", "--log-console"]
|
||||
```
|
||||
|
||||
### 2.2 构建镜像命令
|
||||
```bash
|
||||
docker build -t keepalived:2.3.4 .
|
||||
```
|
||||
|
||||
## 3. 配置文件部署
|
||||
### 3.1 前置准备
|
||||
在三台机器上统一创建配置目录：
|
||||
|
||||
```bash
|
||||
mkdir -p /data/keepalived
|
||||
```
|
||||
|
||||
### 3.2 Docker Compose 配置（三台机器通用）
|
||||
创建 `/data/keepalived/docker-compose.yml`：
|
||||
|
||||
```yaml
|
||||
services:
|
||||
keepalived:
|
||||
image: keepalived:2.3.4
|
||||
container_name: keepalived
|
||||
restart: always
|
||||
network_mode: host # 主机网络模式，确保 VRRP 协议正常通信
|
||||
cap_add: # 添加网络管理权限
|
||||
- NET_ADMIN
|
||||
- NET_BROADCAST
|
||||
- NET_RAW
|
||||
volumes:
|
||||
- /data/keepalived/keepalived.conf:/etc/keepalived/keepalived.conf:ro # 只读挂载配置文件
|
||||
- /data/keepalived/check-mysql.sh:/etc/keepalived/check-mysql.sh:ro # 挂载 MySQL 检测脚本（仅 Node A/B 需要）
|
||||
```
|
||||
|
||||
### 3.3 各节点 Keepalived 配置
|
||||
#### 3.3.1 Node A（${NODE_A_IP}）配置
|
||||
路径：`/data/keepalived/keepalived.conf`
|
||||
|
||||
```nginx
|
||||
global_defs {
|
||||
router_id ${ROUTER_ID_A}
|
||||
vrrp_skip_check_adv_addr
|
||||
script_user root # 指定脚本执行用户
|
||||
enable_script_security # 启用脚本安全校验，防止执行未授权脚本
|
||||
}
|
||||
|
||||
# MySQL 存活检测脚本定义
|
||||
vrrp_script check_mysql {
|
||||
script "/etc/keepalived/check-mysql.sh"
|
||||
interval 3 # 检测间隔：3 秒
|
||||
weight -50 # 检测失败时，优先级扣 50
|
||||
fall 3 # 连续 3 次失败判定为故障
|
||||
rise 2 # 连续 2 次成功判定为恢复
|
||||
}
|
||||
|
||||
vrrp_instance VI_1 {
|
||||
state BACKUP # 所有节点统一设为 BACKUP，靠优先级决定 Master
|
||||
interface ${INTERFACE} # 替换为宿主机实际网卡名（如 ens33/br0 等）
|
||||
virtual_router_id ${VIRTUAL_ROUTER_ID} # 虚拟路由 ID，集群内必须一致（1-255）
|
||||
priority 110 # 基础优先级
|
||||
preempt_delay 30 # 优先级恢复后，延迟 30 秒抢占 VIP（减少抖动）
|
||||
advert_int 1 # VRRP 通告间隔：1 秒
|
||||
|
||||
# 认证配置（集群内必须一致）
|
||||
authentication {
|
||||
auth_type PASS
|
||||
auth_pass ${AUTH_PASS}
|
||||
}
|
||||
|
||||
# 单播配置（替代组播，适配部分网络环境）
|
||||
unicast_src_ip ${NODE_A_IP} # 本机 IP
|
||||
unicast_peer { # 集群其他节点 IP
|
||||
${NODE_B_IP}
|
||||
${NODE_C_IP}
|
||||
}
|
||||
|
||||
# 虚拟 IP 配置
|
||||
virtual_ipaddress {
|
||||
${VIP_IP}/32 # VIP 地址（32 位掩码）
|
||||
}
|
||||
|
||||
# 绑定检测脚本
|
||||
track_script {
|
||||
check_mysql
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 3.3.2 Node B（${NODE_B_IP}）配置
|
||||
路径：`/data/keepalived/keepalived.conf`
|
||||
|
||||
```nginx
|
||||
global_defs {
|
||||
router_id ${ROUTER_ID_B}
|
||||
vrrp_skip_check_adv_addr
|
||||
script_user root
|
||||
enable_script_security
|
||||
}
|
||||
|
||||
# MySQL 存活检测脚本定义
|
||||
vrrp_script check_mysql {
|
||||
script "/etc/keepalived/check-mysql.sh"
|
||||
interval 3
|
||||
weight -50
|
||||
fall 3
|
||||
rise 2
|
||||
}
|
||||
|
||||
vrrp_instance VI_1 {
|
||||
state BACKUP
|
||||
interface ${INTERFACE} # 替换为宿主机实际网卡名
|
||||
virtual_router_id ${VIRTUAL_ROUTER_ID}
|
||||
priority 100 # 基础优先级（低于 Node A）
|
||||
advert_int 1
|
||||
|
||||
authentication {
|
||||
auth_type PASS
|
||||
auth_pass ${AUTH_PASS}
|
||||
}
|
||||
|
||||
unicast_src_ip ${NODE_B_IP}
|
||||
unicast_peer {
|
||||
${NODE_A_IP}
|
||||
${NODE_C_IP}
|
||||
}
|
||||
|
||||
virtual_ipaddress {
|
||||
${VIP_IP}/32
|
||||
}
|
||||
|
||||
track_script {
|
||||
check_mysql
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 3.3.3 Node C（${NODE_C_IP}，仲裁节点）配置
|
||||
路径：`/data/keepalived/keepalived.conf`
|
||||
|
||||
```nginx
|
||||
global_defs {
|
||||
router_id ${ROUTER_ID_C}
|
||||
vrrp_skip_check_adv_addr
|
||||
}
|
||||
|
||||
vrrp_instance VI_1 {
|
||||
state BACKUP
|
||||
interface ${INTERFACE} # 替换为宿主机实际网卡名
|
||||
virtual_router_id ${VIRTUAL_ROUTER_ID}
|
||||
priority 40 # 最低优先级（仅参与选举，不持有 VIP）
|
||||
advert_int 1
|
||||
|
||||
authentication {
|
||||
auth_type PASS
|
||||
auth_pass ${AUTH_PASS}
|
||||
}
|
||||
|
||||
unicast_src_ip ${NODE_C_IP}
|
||||
unicast_peer {
|
||||
${NODE_A_IP}
|
||||
${NODE_B_IP}
|
||||
}
|
||||
|
||||
# 仲裁节点不配置 virtual_ipaddress，不持有 VIP
|
||||
}
|
||||
```
|
||||
|
||||
### 3.4 MySQL 检测脚本（仅 Node A/B 需要）
|
||||
路径：`/data/keepalived/check-mysql.sh`
|
||||
|
||||
```bash
|
||||
#!/bin/sh
|
||||
# 检测逻辑：
|
||||
# 1. timeout 2：限制命令 2 秒内完成，避免脚本阻塞
|
||||
# 2. nc -z：仅检测端口连通性（无数据传输）
|
||||
# 3. 屏蔽所有输出，仅返回退出码
|
||||
|
||||
if timeout 2 nc -z 127.0.0.1 3306 > /dev/null 2>&1; then
|
||||
# 端口连通，返回 0（Keepalived 判定正常）
|
||||
exit 0
|
||||
else
|
||||
# 端口不通/超时，返回 1（Keepalived 触发优先级扣减）
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
添加执行权限：
|
||||
|
||||
```bash
|
||||
chmod +x /data/keepalived/check-mysql.sh
|
||||
```
|
||||
|
||||
## 4. 启动与验证
|
||||
### 4.1 启动容器
|
||||
三台机器执行相同命令：
|
||||
|
||||
```bash
|
||||
cd /data/keepalived
|
||||
docker-compose up -d
|
||||
```
|
||||
|
||||
### 4.2 日志查看
|
||||
```bash
|
||||
docker logs -f keepalived
|
||||
```
|
||||
|
||||
> 正常日志会显示 VRRP 选举过程、VIP 绑定/释放、脚本检测结果等信息。
|
||||
>
|
||||
|
||||
### 4.3 VIP 验证
|
||||
在任意节点执行，查看 VIP 是否绑定：
|
||||
|
||||
```bash
|
||||
ip addr show ${INTERFACE} # 替换为实际网卡名
|
||||
```
|
||||
|
||||
+ 正常情况下，VIP（${VIP_IP}）会绑定在优先级最高的节点（初始为 Node A）。
|
||||
+ 输出示例：`inet ${VIP_IP}/32 scope global ${INTERFACE}`
|
||||
|
||||
## 5. 故障场景分析
|
||||
详细的故障场景模拟与 VIP 切换逻辑分析，请参阅：[Keepalived 故障场景分析](./fault-analysis.md)
|
||||
|
||||
## 6. 核心总结
|
||||
### 6.1 优先级规则
|
||||
+ VIP 归属由 Keepalived **有效优先级** 决定：基础优先级 A(110) > B(100) > C(40)；
|
||||
+ MySQL 故障会让 A/B 优先级扣 50，Keepalived 离线则失去选举资格。
|
||||
|
||||
### 6.2 仲裁节点作用
|
||||
+ Node C 仅参与选举流程，不持有 VIP；
|
||||
+ 核心作用是防止 A、B 脑裂，正常/单节点故障场景下不影响 A/B 选举逻辑。
|
||||
|
||||
### 6.3 风险与优化
|
||||
+ **风险**：A/B MySQL 均故障但 Keepalived 正常时，VIP 仍绑定到 A，此时无可用 MySQL；需在业务层增加数据库存活检测，避免连接失效的 VIP。
|
||||
+ **抢占优化**：A 恢复后会在 `preempt_delay`（30 秒）后抢占 VIP，若需避免业务抖动，可在 Node A 配置中添加 `nopreempt` 关闭抢占。
|
||||
|
||||
### 6.4 关键配置注意事项
|
||||
+ 所有节点 `virtual_router_id` 和认证信息必须一致；
|
||||
+ 网卡名需替换为宿主机实际名称；
|
||||
+ 仲裁节点 C 不配置 `virtual_ipaddress`，不挂载 MySQL 检测脚本。
|
||||
|
||||
Reference in New Issue
Block a user