标签归档:MULTICAST

Keepalived单网络主备单主配置模式(实战)

前言:

以下阐述在Keepalived中配置最简单的主备模式,后面我将一直讲述从简单的单网络单主主备模式,到双网络双主双同步的故障漂移模式。

关于Keepalived介绍,这里就不再叙述,可参阅前文:

keepalived 源代码编译安装及配置文件讲解》。

架构示意图:

多播IP是:224.0.0.111。
NAT网络可根据自己的实际情况配置。

                        +------+
			|Client|
			+------+
                           /\
		       +--------+
                       |Internet|
		       +--------+
                           /\
		       +--------+
                       |NAT 网络|
		       +--------+
                           /\
		+-----------------------+
                | 公网VIP1:172.16.8.100 |
		+-----------------------+
                   /                \
+-----------------------+      +-----------------------+
| KA+Lvs/Nginx/HAProxy  |      | KA+Lvs/Nginx/HAProxy  |
| VIP1:Master    (eth1) |多播IP| VIP1:BACKUP    (eth1) |
| IP1:172.16.8.10(eth1) |      | IP1:172.16.8.11(eth1) |
+-----------------------+      +-----------------------+
                   \                /
                +-----------------------+
                | 公网VIP1:172.16.8.100 |
		+-----------------------+
		           \/
		        +------+
			|资源池|
			+------+

环境:

MASTER:172.16.8.10
BACKUP:172.16.8.11
VIP:172.16.8.100
OS:CentOS Linux release 7.4.1708 (Core)

先决条件:

  • 时间同步。
  • 设置SELinux和防火墙。
  • 互相之间/etc/hosts文件添加对方主机名(可选)。
  • 确认接口支持多播(组播)新网卡默认支持。

keepalived 源代码编译安装及配置文件讲解》文中已完成以上必备条件。

1.单网络主备配置文件

MASTER 配置文件:

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka67
   vrrp_mcast_group4 224.0.0.111
}
vrrp_instance VG_1 {
    state MASTER
    interface eth0
    virtual_router_id 103
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"          
}

BACKUP 配置文件:

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka68@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka68
   vrrp_mcast_group4 224.0.0.111
}
vrrp_instance VG_1 {
    state BACKUP
    interface eth0
    virtual_router_id 103
    priority 95
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"             
}

2.通用脚本

以下内容是notfiy.sh通用检测脚本:

$ cat /usr/local/keepalived/etc/keepalived/notify.sh
#!/bin/bash
contact='root@localhost'
                
notify() {
    local mailsubject="$(hostname) to be $1, vip floating"
    local mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
    echo "$mailbody" | mail -s "$mailsubject" $contact
}
                
case $1 in
master)
    notify master   
    ;;
backup)
    notify backup   
    ;;
fault)
    notify fault    
    ;;
*)
    echo "Usage: $(basename $0) {master|backup|fault}"
    exit 1
    ;;
esac

3.主备测试

测试MASTER

启动keepalived之前,查看网卡信息:

[root@ka67 keepalived]# ip a
...
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
   link/ether 00:15:5d:ae:02:78 brd ff:ff:ff:ff:ff:ff
   inet 172.16.8.10/24 brd 172.16.8.255 scope global eth0
      valid_lft forever preferred_lft forever
   inet6 fe80::436e:b837:43b:797c/64 scope link
      valid_lft forever preferred_lft forever

启动keepalived后,再次查看网卡信息:

[root@ka67 keepalived]# ip a
...
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
   link/ether 00:15:5d:ae:02:78 brd ff:ff:ff:ff:ff:ff
   inet 172.16.8.10/24 brd 172.16.8.255 scope global eth0
      valid_lft forever preferred_lft forever
   inet 172.16.8.100/32 scope global eth0
      valid_lft forever preferred_lft forever
   inet6 fe80::436e:b837:43b:797c/64 scope link
      valid_lft forever preferred_lft forever

已经成功添加VIP 172.16.8.100。

测试MASTER

启动keepalived:

[root@ka68 keepalived]# systemctl start keepalived

现在停止 MASTER,看会不会漂移到BACKUP:

[root@ka67 keepalived]# systemctl stop keepalived

查看BACKUP运行日志:

[root@ka68 keepalived]# cat /cat /var/log/messages
...
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) Transition to MASTER STATE
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) Entering MASTER STATE
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) setting protocol VIPs.
Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
...

已经成功漂移到BACKUP 主机。

再次启动MASTER:

[root@ka67 keepalived]# systemctl start keepalived

查看BACKUP Keepalived服务状态:

[root@ka68 keepalived]# systemctl status keepalived
keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
   Active: active (running) since Tue 2018-03-02 22:13:14 EST; 15min ago
  Process: 1448 ExecStart=/usr/local/keepalived/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 1449 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─1449 /usr/local/keepalived/sbin/keepalived -D
           ├─1450 /usr/local/keepalived/sbin/keepalived -D
           └─1451 /usr/local/keepalived/sbin/keepalived -D

Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) Sending/queueing gratuitous ARPs on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) Entering BACKUP STATE
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) removing protocol VIPs.
Keepalived_vrrp[1451]: Opening script file /usr/local/keepalived/etc/keepalived/notify.sh

以上状态表明,当MASTER恢复服务后,BACKUP的Keepalived会自动漂移到MASTER上。因为MASTER的权重值比BACKUP高。以上是BACKUP的漂移到MASTER的状态。

keepalived 源代码编译安装及配置文件讲解

何为Keepalived?

Keepalived是一个基于vrrp协议来实现的服务器高可用解决方案,可以利用其实现避免IP单点故障,类似的工具还有heartbeat、corosync。不过其不会单独出现,而是搭配着 LVS、Nginx、HAproxy,一起协同工作达到高可用的目的。

何为VRRP协议?

VRRP全称Vritual Router Redundancy Protocol,虚拟路由冗余协议。通过把几台提供路由功能的设备组成一个虚拟路由设备,使用一定的机制保证虚拟路由的高可用,从而达到保持业务的连续性与可靠性。

在配置组成的一个虚拟路由器中,有MASTER和BACKUP之分。MASTER是主节点,在一个虚拟路由器中,只能有一个MASTER,但可以有多个BACKUP,BACKUP是备用节点,也就是当master挂掉之后,BACKUP接管MASTER节点的所有资源,当有多个BACKUP节点时,根据其priority(优先级)的值的大小,来选举谁作为MASTER的替代者。当BACKUP节点的优先级值相同时,根据其IP地址的大小,来决定。

先决条件:

  • 节点之间的时间必须同步。
  • 确保FirewalldSELinux不会成为阻碍。
  • 各节点用于集群服务的网络接口必须支持MULTICAST(多播)通信。采用D类地址(224-239)。多播地址建议手动定义,因为若有多个集群服务都使用默认的,虽有认证机制,但仍会互发信息,可能会影响性能,更会产生无用日志信息。

1.时间同步

请参阅《Centos 7 Chrony 设置服务器集群系统时间同步》。

2.Firewalld防火墙配置

$ firewall-cmd --direct --permanent --add-rule ipv4 filter INPUT 0 --in-interface eth0 --destination 224.0.0.111 --protocol vrrp -j ACCEPT
$ firewall-cmd --direct --permanent --add-rule ipv4 filter OUTPUT 0 --out-interface eth0 --destination 224.0.0.111 --protocol vrrp -j ACCEPT
$ firewall-cmd --reload

3.开启路由转发

$ echo "net.ipv4.ip_forward=1" >>/etc/sysctl.conf
$ echo 1 > /proc/sys/net/ipv4/ip_forward

4.安装keepalived

$ cd /tmp
$ wget //www.keepalived.org/software/keepalived-1.3.9.tar.gz
$ tar xvf keepalived-1.3.9.tar.gz
$ cd keepalived-1.3.9
$ ./configure --prefix=/usr/local/keepalived
$ make && make install
$ ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin
$ mkdir /etc/keepalived/
$ ln -s /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

在安装过程中可能出现以下错误:

可能报错1:

checking for gcc... no
checking for cc... no
checking for cl.exe... no
configure: error: in `/tmp/keepalived-1.3.9':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details

解决方案:

$ yum install gcc -y

可能报错2:

configure: error:
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files.            !!!

解决方案:

$ yum install openssl-devel -y

可能报错3:

*** WARNING - this build will not support IPVS with IPv6. Please install libnl/libnl-3 dev libraries to support IPv6 with IPVS.

解决方案:

$ yum install libnl-devel

可能报错4:

checking libnfnetlink/libnfnetlink.h usability... no
checking libnfnetlink/libnfnetlink.h presence... no
checking for libnfnetlink/libnfnetlink.h... no
configure: error: libnfnetlink headers missing

解决方案:

$ yum install libnfnetlink-devel -y

安装总结:

一个最小化安装的Centos 7系统,在源代码编译安装keepalived时,首先安装以下依赖包,再编译安装keepalived就不会出现以上错误了。

安装依赖包:

$ yum install gcc openssl-devel libnl-devel libnfnetlink-devel ipvsadm -y

5.keepalived配置文件讲解

# 全局配置,包含两个子配置块:全局定义和静态地址和路由。

! Configuration File for keepalived
global_defs {               # 全局定义
   notification_email {     # 通知邮件相关设置
    acassen@firewall.loc    # 邮件发送目标地址
   }
   notification_email_from rwl@renwole.com  # 发件人
   smtp_server 127.0.0.1    # 使用本机邮件服务
   smtp_connect_timeout 30  # 设置连接smtp server的超时时间
   router_id LVS_DEVEL      # 标识当前节点唯一,节点间不能相同
   # 检查vrrp报文中的所有地址比较耗时
   # 设置此标志的意思是如果接收的到报文和上一个报文来至同一个路由器,则不执行检查。默认是跳过检查
   vrrp_skip_check_adv_addr
   vrrp_strict              # 严格执行VRRP协议规范,此模式不支持节点单播
   # 小数类型,单位秒。
   # 在一个网卡上每组gratuitous arp消息之间的延迟时间。
   # 默认为0,一个发送的消息=n组 arp报文
   vrrp_garp_interval 0
   # 小数类型,单位秒
   # 在一个网卡上每组na消息之间的延迟时间,默认为0
   vrrp_gna_interval 0
}

# 此区域是VRRP配置,含两个子配置块:vrrp_sync_group/vrrp_instance,主要对外提供服务VIP区域及其相关属性

vrrp_instance VI_1 {     # VRRP实例
    state MASTER         # 只能有一个是MASTER,余下的都应该为BACKUP;
    interface eth1	 # 对外提供网络的接口
    virtual_router_id 65 # 虚拟路由id标识,数字,必须和backup里相同
    priority 100         # 优先级,数字必须比backup大
    advert_int 1         # 组播信息发送间隔,两个节点设置必须一样,秒
    authentication {	 # 设置验证信息,两个节点必须一致(明文)
        auth_type PASS
        auth_pass 1111
    }
	# 虚拟地址,即Floating IP
    virtual_ipaddress {  # 可简写为单个地址,系统会默认计算掩码和设备
	
        172.16.28.65
	# 172.16.28.65/24
        # 172.16.28.65/24 dev eth1
    }
	# 定义通知脚本(另添加)
        notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
        notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
        notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}

# 此区域是LVS配置。若用Keepalived+LVS,需要这段配置,若用其他,例如:Keepalived+Nginx,则无需配置。
# LVS包含两个子配置块:virtual_server_group/virtual_server
# virtual_server:虚拟服务器。每个虚拟服务器里面包含多个真实服务器real_server。

virtual_server 172.16.28.65 80 {  # 虚拟IP 监听80端口
    delay_loop 6		  # 健康检查时间间隔,秒
    lb_algo rr		          # 负载调度算法,常见使用wlc或rr
    lb_kind NAT		          # LVS负载转发规则,DR,NAT,TUN等
    persistence_timeout 50        # 会话保持时间,秒
    protocol TCP		  # 转发协议一般有tcp和udp两种

    real_server 172.16.28.65 80 { # 配置真实服务器的地址与端口
        weight 1		  # 权重
        SSL_GET {
            url {
              path /		  # 健康检查的页面
              digest ff20ad2481f97b1754ef3e12ecd3a9cc # 计算出的MD5值
            }
            url {
              path /mrtg/
              digest 9b3a0c85a887a256d6939da88aabd8cd
            }
            connect_timeout 3     # 连接超时时间,秒
            nb_get_retry 3	  # 失败重试次数,超过后移除
            delay_before_retry 3  # 失败重试间隔,秒
        }
    }
}
...

6.启动keepalived

在正确配置完keepalived.conf后,就可以启动keepalived,并且加入开机自启动服务。

$ systemctl start keepalived
$ systemctl enable keepalived

keepalived源码编译安装完毕。

说明:此文章主要介绍何为Keepalived及如何安装keepalived与注解keepalived的配置文件作用,无具体实验或生产配置。

欲了解生产环境中的keepalived各种模式配置,请参阅以下文章:

Keepalived单网络主备单主配置模式(实战)
Keepalived单网络双活双主配置模式
Keepalived双网络(内外网)故障非同步漂移主备单主模式
Keepalived双网络(内外网)故障同步漂移主备单主模式
Keepalived双网络(内外网)故障非同步漂移双活双主模式
Keepalived双网络(内外网)故障同步漂移双活双主模式