月度归档:2018年03月

Keepalived双网络(内外网)故障非同步漂移双活双主模式

前言:

在生产环境中,公网与内网都是独立分开的,所以称之为双网络。下面配置将要实现内网和公网故障时不必同步漂移,例如:Keepalived+LVS-DRKeepalived+NginxKeepalived+HAProxy 这些都无需同步漂移的。另外Keepalived+LVS-NAT则需要同步漂移。

1.示意图:

  • 多播IP是:224.0.0.111。
  • 一台机器的VIP内外网互为主备。
                        +------+
			|Client|
			+------+
                           /\
		       +--------+   
                       |Internet|
		       +--------+
                           /\
		       +--------+  
                       |NAT 网络|
		       +--------+
                           /\
                +----------------------+
                | 内网VIP1:10.16.8.100 |
		| 内网VIP2:10.16.8.101 |
                +----------------------+
                   /                \
+-----------------------+      +-----------------------+
|KA+Lvs-DR/Nginx/HAProxy|      |KA+Lvs-DR/Nginx/HAProxy|
|内网VIP1:Master (eth1) |      |内网VIP1:BACKUP (eth1) |
|内网VIP2:BACKUP (eth1) |      |内网VIP2:Master (eth1) |
|内网:10.16.8.10 (eth1) |<---->|内网:10.16.8.11 (eth1) |
|-----------------------|多播IP|-----------------------|
|公网VIP1:Master (eth2) |<---->|公网VIP1:BACKUP (eth2) |
|公网VIP2:BACKUP (eth2) |      |公网VIP2:Master (eth2) |
|公网:172.16.8.10(eth2) |      |公网:172.16.8.11(eth2) |                 
+-----------------------+      +-----------------------+
                   \                /
	        +-----------------------+	 
                | 公网VIP1:172.16.8.100 |
		| 公网VIP2:172.16.8.101 |
	        +-----------------------+
		           \/
			+------+
			|资源池|
			+------+

2.ka67配置文件

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka67
   vrrp_mcast_group4 224.0.0.111
}
vrrp_instance External_1 {
    state MASTER
    interface eth1
    virtual_router_id 171
    priority 100
    advert_int 1    
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        10.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}
vrrp_instance External_2 {
    state BACKUP
    interface eth1
    virtual_router_id 172
    priority 95
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole1
    }
    virtual_ipaddress {
        10.16.8.101
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}
vrrp_instance Internal_1 {
    state MASTER
    interface eth2
    virtual_router_id 191
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole2
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}
vrrp_instance Internal_2 {
    state BACKUP
    interface eth2
    virtual_router_id 192
    priority 95
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole3
    }
    virtual_ipaddress {
        172.16.8.101
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}

3.ka68配置文件

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka68
   vrrp_mcast_group4 224.0.0.111
}
vrrp_instance External_1 {
    state BACKUP
    interface eth1
    virtual_router_id 171
    priority 100
    advert_int 1    
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        10.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}
vrrp_instance External_2 {
    state MASTER
    interface eth1
    virtual_router_id 172
    priority 95
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole1
    }
    virtual_ipaddress {
        10.16.8.101
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}
vrrp_instance Internal_1 {
    state BACKUP
    interface eth2
    virtual_router_id 191
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole2
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}
vrrp_instance Internal_2 {
    state MASTER
    interface eth2
    virtual_router_id 192
    priority 95
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole3
    }
    virtual_ipaddress {
        172.16.8.101
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}

Keepalived双网络(内外网)故障同步漂移主备单主模式

前言:

在生产环境当中,内网与公网都是分开的,要实现内网和公网同步漂移,比如:Keepalived+LVS-NAT模式,那么就需要设置vrrp_sync_group同步组,不同的是下面配置只是主备模式,而不是主主模式。

1.示意图:

  • 多播IP是:224.0.0.111。
  • Master内网和公网VIP属同组。
  • BACKUP内网与公网VIP属同组。
                        +------+
			|Client|
			+------+
                           /\
		       +--------+   
                       |Internet|
		       +--------+
                           /\
		       +--------+  
                       |NAT 网络|
		       +--------+
                           /\
                +---------------------+
                | 内网VIP:10.16.8.100 |
                +---------------------+
                  /                \
+-----------------------+      +-----------------------+
|KA+Lvs/Nginx/HAProxy   |      |KA+Lvs/Nginx/HAProxy   |
|内网VIP:Master  (eth1) |      |内网VIP:BACKUP  (eth1) |
|内网:10.16.8.10 (eth1) |<---->|内网:10.16.8.11 (eth1) |
|-----------------------|多播IP|-----------------------|
|公网VIP:Master  (eth2) |<---->|公网VIP:BACKUP  (eth2) |
|公网:172.16.8.10(eth2) |      |公网:172.16.8.11(eth2) |                  
+-----------------------+      +-----------------------+
                   \                /
		+----------------------+	 
                | 公网VIP:172.16.8.100 |
		+----------------------+
		           \/
			+------+
			|资源池|
		        +------+

2.ka67配置文件

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka67
   vrrp_mcast_group4 224.0.0.111
}
vrrp_sync_group VG_1 {
    group {
        External_1
        Internal_1
    }
}
vrrp_instance External_1 {
    state MASTER
    interface eth1
    virtual_router_id 171
    priority 100
    advert_int 1    
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        10.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}
vrrp_instance Internal_1 {
    state MASTER
    interface eth2
    virtual_router_id 191
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole1
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}

3.ka68配置文件

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka68
   vrrp_mcast_group4 224.0.0.111
}
vrrp_sync_group VG_1 {
    group {
        External_1
        Internal_1
    }
}
vrrp_instance External_1 {
    state BACKUP
    interface eth1
    virtual_router_id 171
    priority 100
    advert_int 1    
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        10.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}
vrrp_instance Internal_1 {
    state BACKUP
    interface eth2
    virtual_router_id 191
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole1
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}

Keepalived双网络(内外网)故障非同步漂移主备单主模式

前言:

在生产环境当中,内网与公网都是独立分开的,因此内网和公网不用同步漂移,例如:Keepalived+LVS-DR、Keepalived+Nginx、Keepalived+HAProxy 都无需同步漂移。

注:Keepalived+LVS-NAT模式除外。

1.示意图:

多播IP是:224.0.0.111。

                        +------+
			|Client|
			+------+
                           /\
		       +--------+   
                       |Internet|
		       +--------+
                           /\
		       +--------+  
                       |NAT 网络|
		       +--------+
                           /\
                +---------------------+
                | 内网VIP:10.16.8.100 |
                +---------------------+
                  /                \
+-----------------------+      +-----------------------+
|KA+Lvs/Nginx/HAProxy   |      |KA+Lvs/Nginx/HAProxy   |
|内网VIP:Master  (eth1) |      |内网VIP:BACKUP  (eth1) |
|内网:10.16.8.10 (eth1) |<---->|内网:10.16.8.11 (eth1) |
|-----------------------|多播IP|-----------------------| 
|公网VIP:Master  (eth2) |<---->|公网VIP:BACKUP  (eth2) |
|公网:172.16.8.10(eth2) |      |公网:172.16.8.11(eth2) |                  
+-----------------------+      +-----------------------+
                   \                /
	        +----------------------+	 
                | 公网VIP:172.16.8.100 |
	        +----------------------+
		           \/
			+------+
			|资源池|
		        +------+

2.ka67配置文件

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka67
   vrrp_mcast_group4 224.0.0.111
}
vrrp_sync_group VG_1 {
    group {
        External_1
        Internal_1
    }
}
vrrp_instance External_1 {
    state MASTER
    interface eth1
    virtual_router_id 171
    priority 100
    advert_int 1    
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        10.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}
vrrp_instance Internal_1 {
    state MASTER
    interface eth2
    virtual_router_id 191
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole1
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}

3.ka68配置文件

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka68
   vrrp_mcast_group4 224.0.0.111
}
vrrp_instance External_1 {
    state BACKUP
    interface eth1
    virtual_router_id 171
    priority 100
    advert_int 1    
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        10.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}
vrrp_instance Internal_1 {
    state BACKUP
    interface eth2
    virtual_router_id 191
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole1
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}

Keepalived单网络双活双主配置模式

前言:

一般这种模式不需要相对复杂的配置,相对单网络单主模式,多了一个主可用模式。主要实现单网络双主故障漂移模式。

1.架构示意图:

多播IP是:224.0.0.111。
NAT网络可根据自己的实际情况配置。

                        +------+
			|Client|
			+------+
                           /\
		       +--------+   
                       |Internet|
		       +--------+
                           /\
		       +--------+  
                       |NAT 网络|
		       +--------+
                           /\
	        +-----------------------+	 
                | 公网VIP1:172.16.8.100 |
		| 公网VIP2:172.16.8.101 |
		+-----------------------+
                   /                \
+-----------------------+      +-----------------------+
| KA+Lvs/Nginx/HAProxy  |      | KA+Lvs/Nginx/HAProxy  |
|                       |<---->|                       |
| VIP1:Master    (eth1) |多播IP| VIP1:BACKUP    (eth1) |
| VIP2:BACKUP    (eth1) |<---->| VIP2:Master    (eth1) |
| IP1:172.16.8.10(eth1) |      | IP1:172.16.8.11(eth1) |
+-----------------------+      +-----------------------+
                   \                /
	        +-----------------------+	 
                | 公网VIP1:172.16.8.100 |
		| 公网VIP2:172.16.8.101 |
		+-----------------------+
			   \/
			+------+
			|资源池|
			+------+

2.ka67配置文件:

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka67
   vrrp_mcast_group4 224.0.0.111
}
vrrp_instance VG_1 {
    state MASTER
    interface eth0
    virtual_router_id 191
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"          
}
vrrp_instance VG_2 {
    state BACKUP
    interface eth0
    virtual_router_id 192
    priority 95
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole1
    }
    virtual_ipaddress {
        172.16.8.101
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"          
}

3.ka68配置文件:

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka68
   vrrp_mcast_group4 224.0.0.111
}
vrrp_instance VG_1 {
    state BACKUP
    interface eth0
    virtual_router_id 191
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"  
}
vrrp_instance VG_2 {
    state MASTER
    interface eth0
    virtual_router_id 192
    priority 95
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole1
    }
    virtual_ipaddress {
        172.16.8.101
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}

Keepalived单网络主备单主配置模式(实战)

前言:

以下阐述在Keepalived中配置最简单的主备模式,后面我将一直讲述从简单的单网络单主主备模式,到双网络双主双同步的故障漂移模式。

关于Keepalived介绍,这里就不再叙述,可参阅前文:

keepalived 源代码编译安装及配置文件讲解》。

架构示意图:

多播IP是:224.0.0.111。
NAT网络可根据自己的实际情况配置。

                        +------+
			|Client|
			+------+
                           /\
		       +--------+
                       |Internet|
		       +--------+
                           /\
		       +--------+
                       |NAT 网络|
		       +--------+
                           /\
		+-----------------------+
                | 公网VIP1:172.16.8.100 |
		+-----------------------+
                   /                \
+-----------------------+      +-----------------------+
| KA+Lvs/Nginx/HAProxy  |      | KA+Lvs/Nginx/HAProxy  |
| VIP1:Master    (eth1) |多播IP| VIP1:BACKUP    (eth1) |
| IP1:172.16.8.10(eth1) |      | IP1:172.16.8.11(eth1) |
+-----------------------+      +-----------------------+
                   \                /
                +-----------------------+
                | 公网VIP1:172.16.8.100 |
		+-----------------------+
		           \/
		        +------+
			|资源池|
			+------+

环境:

MASTER:172.16.8.10
BACKUP:172.16.8.11
VIP:172.16.8.100
OS:CentOS Linux release 7.4.1708 (Core)

先决条件:

  • 时间同步。
  • 设置SELinux和防火墙。
  • 互相之间/etc/hosts文件添加对方主机名(可选)。
  • 确认接口支持多播(组播)新网卡默认支持。

keepalived 源代码编译安装及配置文件讲解》文中已完成以上必备条件。

1.单网络主备配置文件

MASTER 配置文件:

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka67
   vrrp_mcast_group4 224.0.0.111
}
vrrp_instance VG_1 {
    state MASTER
    interface eth0
    virtual_router_id 103
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"          
}

BACKUP 配置文件:

global_defs {
   notification_email {
     root@localhost
   }
   notification_email_from ka68@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka68
   vrrp_mcast_group4 224.0.0.111
}
vrrp_instance VG_1 {
    state BACKUP
    interface eth0
    virtual_router_id 103
    priority 95
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass renwole0
    }
    virtual_ipaddress {
        172.16.8.100
    }
    notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
    notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
    notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"             
}

2.通用脚本

以下内容是notfiy.sh通用检测脚本:

$ cat /usr/local/keepalived/etc/keepalived/notify.sh
#!/bin/bash
contact='root@localhost'
                
notify() {
    local mailsubject="$(hostname) to be $1, vip floating"
    local mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
    echo "$mailbody" | mail -s "$mailsubject" $contact
}
                
case $1 in
master)
    notify master   
    ;;
backup)
    notify backup   
    ;;
fault)
    notify fault    
    ;;
*)
    echo "Usage: $(basename $0) {master|backup|fault}"
    exit 1
    ;;
esac

3.主备测试

测试MASTER

启动keepalived之前,查看网卡信息:

[root@ka67 keepalived]# ip a
...
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
   link/ether 00:15:5d:ae:02:78 brd ff:ff:ff:ff:ff:ff
   inet 172.16.8.10/24 brd 172.16.8.255 scope global eth0
      valid_lft forever preferred_lft forever
   inet6 fe80::436e:b837:43b:797c/64 scope link
      valid_lft forever preferred_lft forever

启动keepalived后,再次查看网卡信息:

[root@ka67 keepalived]# ip a
...
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
   link/ether 00:15:5d:ae:02:78 brd ff:ff:ff:ff:ff:ff
   inet 172.16.8.10/24 brd 172.16.8.255 scope global eth0
      valid_lft forever preferred_lft forever
   inet 172.16.8.100/32 scope global eth0
      valid_lft forever preferred_lft forever
   inet6 fe80::436e:b837:43b:797c/64 scope link
      valid_lft forever preferred_lft forever

已经成功添加VIP 172.16.8.100。

测试MASTER

启动keepalived:

[root@ka68 keepalived]# systemctl start keepalived

现在停止 MASTER,看会不会漂移到BACKUP:

[root@ka67 keepalived]# systemctl stop keepalived

查看BACKUP运行日志:

[root@ka68 keepalived]# cat /cat /var/log/messages
...
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) Transition to MASTER STATE
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) Entering MASTER STATE
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) setting protocol VIPs.
Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
...

已经成功漂移到BACKUP 主机。

再次启动MASTER:

[root@ka67 keepalived]# systemctl start keepalived

查看BACKUP Keepalived服务状态:

[root@ka68 keepalived]# systemctl status keepalived
keepalived.service - LVS and VRRP High Availability Monitor
   Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
   Active: active (running) since Tue 2018-03-02 22:13:14 EST; 15min ago
  Process: 1448 ExecStart=/usr/local/keepalived/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 1449 (keepalived)
   CGroup: /system.slice/keepalived.service
           ├─1449 /usr/local/keepalived/sbin/keepalived -D
           ├─1450 /usr/local/keepalived/sbin/keepalived -D
           └─1451 /usr/local/keepalived/sbin/keepalived -D

Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) Sending/queueing gratuitous ARPs on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: Sending gratuitous ARP on eth0 for 172.16.8.100
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) Entering BACKUP STATE
Keepalived_vrrp[1451]: VRRP_Instance(VG_1) removing protocol VIPs.
Keepalived_vrrp[1451]: Opening script file /usr/local/keepalived/etc/keepalived/notify.sh

以上状态表明,当MASTER恢复服务后,BACKUP的Keepalived会自动漂移到MASTER上。因为MASTER的权重值比BACKUP高。以上是BACKUP的漂移到MASTER的状态。

keepalived 源代码编译安装及配置文件讲解

何为Keepalived?

Keepalived是一个基于vrrp协议来实现的服务器高可用解决方案,可以利用其实现避免IP单点故障,类似的工具还有heartbeat、corosync。不过其不会单独出现,而是搭配着 LVS、Nginx、HAproxy,一起协同工作达到高可用的目的。

何为VRRP协议?

VRRP全称Vritual Router Redundancy Protocol,虚拟路由冗余协议。通过把几台提供路由功能的设备组成一个虚拟路由设备,使用一定的机制保证虚拟路由的高可用,从而达到保持业务的连续性与可靠性。

在配置组成的一个虚拟路由器中,有MASTER和BACKUP之分。MASTER是主节点,在一个虚拟路由器中,只能有一个MASTER,但可以有多个BACKUP,BACKUP是备用节点,也就是当master挂掉之后,BACKUP接管MASTER节点的所有资源,当有多个BACKUP节点时,根据其priority(优先级)的值的大小,来选举谁作为MASTER的替代者。当BACKUP节点的优先级值相同时,根据其IP地址的大小,来决定。

先决条件:

  • 节点之间的时间必须同步。
  • 确保FirewalldSELinux不会成为阻碍。
  • 各节点用于集群服务的网络接口必须支持MULTICAST(多播)通信。采用D类地址(224-239)。多播地址建议手动定义,因为若有多个集群服务都使用默认的,虽有认证机制,但仍会互发信息,可能会影响性能,更会产生无用日志信息。

1.时间同步

请参阅《Centos 7 Chrony 设置服务器集群系统时间同步》。

2.Firewalld防火墙配置

$ firewall-cmd --direct --permanent --add-rule ipv4 filter INPUT 0 --in-interface eth0 --destination 224.0.0.111 --protocol vrrp -j ACCEPT
$ firewall-cmd --direct --permanent --add-rule ipv4 filter OUTPUT 0 --out-interface eth0 --destination 224.0.0.111 --protocol vrrp -j ACCEPT
$ firewall-cmd --reload

3.开启路由转发

$ echo "net.ipv4.ip_forward=1" >>/etc/sysctl.conf
$ echo 1 > /proc/sys/net/ipv4/ip_forward

4.安装keepalived

$ cd /tmp
$ wget //www.keepalived.org/software/keepalived-1.3.9.tar.gz
$ tar xvf keepalived-1.3.9.tar.gz
$ cd keepalived-1.3.9
$ ./configure --prefix=/usr/local/keepalived
$ make && make install
$ ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin
$ mkdir /etc/keepalived/
$ ln -s /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

在安装过程中可能出现以下错误:

可能报错1:

checking for gcc... no
checking for cc... no
checking for cl.exe... no
configure: error: in `/tmp/keepalived-1.3.9':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details

解决方案:

$ yum install gcc -y

可能报错2:

configure: error:
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files.            !!!

解决方案:

$ yum install openssl-devel -y

可能报错3:

*** WARNING - this build will not support IPVS with IPv6. Please install libnl/libnl-3 dev libraries to support IPv6 with IPVS.

解决方案:

$ yum install libnl-devel

可能报错4:

checking libnfnetlink/libnfnetlink.h usability... no
checking libnfnetlink/libnfnetlink.h presence... no
checking for libnfnetlink/libnfnetlink.h... no
configure: error: libnfnetlink headers missing

解决方案:

$ yum install libnfnetlink-devel -y

安装总结:

一个最小化安装的Centos 7系统,在源代码编译安装keepalived时,首先安装以下依赖包,再编译安装keepalived就不会出现以上错误了。

安装依赖包:

$ yum install gcc openssl-devel libnl-devel libnfnetlink-devel ipvsadm -y

5.keepalived配置文件讲解

# 全局配置,包含两个子配置块:全局定义和静态地址和路由。

! Configuration File for keepalived
global_defs {               # 全局定义
   notification_email {     # 通知邮件相关设置
    acassen@firewall.loc    # 邮件发送目标地址
   }
   notification_email_from rwl@renwole.com  # 发件人
   smtp_server 127.0.0.1    # 使用本机邮件服务
   smtp_connect_timeout 30  # 设置连接smtp server的超时时间
   router_id LVS_DEVEL      # 标识当前节点唯一,节点间不能相同
   # 检查vrrp报文中的所有地址比较耗时
   # 设置此标志的意思是如果接收的到报文和上一个报文来至同一个路由器,则不执行检查。默认是跳过检查
   vrrp_skip_check_adv_addr
   vrrp_strict              # 严格执行VRRP协议规范,此模式不支持节点单播
   # 小数类型,单位秒。
   # 在一个网卡上每组gratuitous arp消息之间的延迟时间。
   # 默认为0,一个发送的消息=n组 arp报文
   vrrp_garp_interval 0
   # 小数类型,单位秒
   # 在一个网卡上每组na消息之间的延迟时间,默认为0
   vrrp_gna_interval 0
}

# 此区域是VRRP配置,含两个子配置块:vrrp_sync_group/vrrp_instance,主要对外提供服务VIP区域及其相关属性

vrrp_instance VI_1 {     # VRRP实例
    state MASTER         # 只能有一个是MASTER,余下的都应该为BACKUP;
    interface eth1	 # 对外提供网络的接口
    virtual_router_id 65 # 虚拟路由id标识,数字,必须和backup里相同
    priority 100         # 优先级,数字必须比backup大
    advert_int 1         # 组播信息发送间隔,两个节点设置必须一样,秒
    authentication {	 # 设置验证信息,两个节点必须一致(明文)
        auth_type PASS
        auth_pass 1111
    }
	# 虚拟地址,即Floating IP
    virtual_ipaddress {  # 可简写为单个地址,系统会默认计算掩码和设备
	
        172.16.28.65
	# 172.16.28.65/24
        # 172.16.28.65/24 dev eth1
    }
	# 定义通知脚本(另添加)
        notify_master "/usr/local/keepalived/etc/keepalived/notify.sh master"
        notify_backup "/usr/local/keepalived/etc/keepalived/notify.sh backup"
        notify_fault "/usr/local/keepalived/etc/keepalived/notify.sh fault"
}

# 此区域是LVS配置。若用Keepalived+LVS,需要这段配置,若用其他,例如:Keepalived+Nginx,则无需配置。
# LVS包含两个子配置块:virtual_server_group/virtual_server
# virtual_server:虚拟服务器。每个虚拟服务器里面包含多个真实服务器real_server。

virtual_server 172.16.28.65 80 {  # 虚拟IP 监听80端口
    delay_loop 6		  # 健康检查时间间隔,秒
    lb_algo rr		          # 负载调度算法,常见使用wlc或rr
    lb_kind NAT		          # LVS负载转发规则,DR,NAT,TUN等
    persistence_timeout 50        # 会话保持时间,秒
    protocol TCP		  # 转发协议一般有tcp和udp两种

    real_server 172.16.28.65 80 { # 配置真实服务器的地址与端口
        weight 1		  # 权重
        SSL_GET {
            url {
              path /		  # 健康检查的页面
              digest ff20ad2481f97b1754ef3e12ecd3a9cc # 计算出的MD5值
            }
            url {
              path /mrtg/
              digest 9b3a0c85a887a256d6939da88aabd8cd
            }
            connect_timeout 3     # 连接超时时间,秒
            nb_get_retry 3	  # 失败重试次数,超过后移除
            delay_before_retry 3  # 失败重试间隔,秒
        }
    }
}
...

6.启动keepalived

在正确配置完keepalived.conf后,就可以启动keepalived,并且加入开机自启动服务。

$ systemctl start keepalived
$ systemctl enable keepalived

keepalived源码编译安装完毕。

说明:此文章主要介绍何为Keepalived及如何安装keepalived与注解keepalived的配置文件作用,无具体实验或生产配置。

欲了解生产环境中的keepalived各种模式配置,请参阅以下文章:

Keepalived单网络主备单主配置模式(实战)
Keepalived单网络双活双主配置模式
Keepalived双网络(内外网)故障非同步漂移主备单主模式
Keepalived双网络(内外网)故障同步漂移主备单主模式
Keepalived双网络(内外网)故障非同步漂移双活双主模式
Keepalived双网络(内外网)故障同步漂移双活双主模式

Centos 7 Chrony 设置服务器集群系统时间同步

何为Chrony?

Chrony是一个开源的自由软件,像CentOS 7或基于RHEL 7操作系统,已经是默认服务,默认配置文件在 /etc/chrony.conf 它能保持系统时间与时间服务器(NTP)同步,让时间始终保持同步。相对于NTP时间同步软件,占据很大优势。其用法也很简单。

Chrony有两个核心组件,分别是:

chronyd:是守护进程,主要用于调整内核中运行的系统时间和时间服务器同步。它确定计算机增减时间的比率,并对此进行调整补偿。

chronyc:提供一个用户界面,用于监控性能并进行多样化的配置。它可以在chronyd实例控制的计算机上工作,也可以在一台不同的远程计算机上工作。

OS环境:

10.28.204.65 客户端
10.28.204.66 服务端

CentOS Linux release 7.4.1708 (Core)

情况说明:两台机器都是内网,将204.66作为NTP时间服务器,204.65到此机器上同步时间。

1.安装Chrony

系统默认已经安装,如未安装,请执行以下命令安装:

$ yum install chrony -y

2.启动并加入开机自启动

$ systemctl enable chronyd.service
$ systemctl restart chronyd.service
$ systemctl status chronyd.service

3.Firewalld设置

$ firewall-cmd --add-service=ntp --permanent
$ firewall-cmd --reload

因NTP使用123/UDP端口协议,所以允许NTP服务即可。

4.配置Chrony

以下是系统默认配置文件,我对此加以说明:

$ cat /etc/chrony.conf
# 使用pool.ntp.org项目中的公共服务器。以server开,理论上你想添加多少时间服务器都可以。
# Please consider joining the pool (//www.pool.ntp.org/join.html).
server 0.centos.pool.ntp.org iburst
server 1.centos.pool.ntp.org iburst
server 2.centos.pool.ntp.org iburst
server 3.centos.pool.ntp.org iburst

# 根据实际时间计算出服务器增减时间的比率,然后记录到一个文件中,在系统重启后为系统做出最佳时间补偿调整。
driftfile /var/lib/chrony/drift

# chronyd根据需求减慢或加速时间调整,
# 在某些情况下系统时钟可能漂移过快,导致时间调整用时过长。
# 该指令强制chronyd调整时期,大于某个阀值时步进调整系统时钟。
# 只有在因chronyd启动时间超过指定的限制时(可使用负值来禁用限制)没有更多时钟更新时才生效。
makestep 1.0 3

# 将启用一个内核模式,在该模式中,系统时间每11分钟会拷贝到实时时钟(RTC)。
rtcsync

# Enable hardware timestamping on all interfaces that support it.
# 通过使用hwtimestamp指令启用硬件时间戳
#hwtimestamp eth0
#hwtimestamp eth1
#hwtimestamp *

# Increase the minimum number of selectable sources required to adjust
# the system clock.
#minsources 2

# 指定一台主机、子网,或者网络以允许或拒绝NTP连接到扮演时钟服务器的机器
#allow 192.168.0.0/16
#deny 192.168/16

# Serve time even if not synchronized to a time source.
local stratum 10

# 指定包含NTP验证密钥的文件。
#keyfile /etc/chrony.keys

# 指定日志文件的目录。
logdir /var/log/chrony

# Select which information is logged.
#log measurements statistics tracking

5.设置时区

查看当前系统时区:

$ timedatectl
      Local time: Fri 2018-2-29 13:31:04 CST
  Universal time: Fri 2018-2-29 05:31:04 UTC
        RTC time: Fri 2018-2-29 08:17:20
       Time zone: Asia/Shanghai (CST, +0800)
     NTP enabled: yes
NTP synchronized: yes
 RTC in local TZ: no
      DST active: n/a

如果你当前的时区不正确,请按照以下操作设置。

查看所有可用的时区:

$ timedatectl list-timezones

筛选式查看在亚洲S开的上海可用时区:

$ timedatectl list-timezones |  grep  -E "Asia/S.*"

Asia/Sakhalin
Asia/Samarkand
Asia/Seoul
Asia/Shanghai
Asia/Singapore
Asia/Srednekolymsk

设置当前系统为Asia/Shanghai上海时区:

$ timedatectl set-timezone Asia/Shanghai

设置完时区后,强制同步下系统时钟:

$ chronyc -a makestep
200 OK

6.服务器集群之间的系统时间同步

在生产环境中,其网络都是内网结构,那么内网如何保证服务器之间的时间同步呢?其实这个问题很简单,只需要搭建一台内网时间服务器,然后让所有计算机都到服务端(10.28.204.66)去同步时间即可。

具体操作:在服务端注释以下内容:

#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst

并添加以下内容:(表示与本机同步时间)

server 10.28.204.66 iburst

这样我们需求的一台内网时间服务器已经配置完毕。

同样在客户端注释掉其他server,并在客户端(10.28.204.65)添加以下:

server 10.28.204.66 iburst

到此已经完成系统时间的同步。如有多台机器,操作也是如此。

7.常用命令

查看时间同步源:

$ chronyc sources -v

查看时间同步源状态:

$ chronyc sourcestats -v

设置硬件时间

硬件时间默认为UTC:

$ timedatectl set-local-rtc 1

启用NTP时间同步:

$ timedatectl set-ntp yes

校准时间服务器:

$ chronyc tracking

最后需要注意的是,配置完/etc/chrony.conf后,需重启chrony服务,否则可能会不生效。

mount: unknown filesystem type ‘LVM2_member’

解决硬盘挂载报错:

mount: unknown filesystem type 'LVM2_member'
mount: you must specify the filesystem type

情况说明:

客户托管的服务器到机房,系统是Linux,无法启动,只能做系统恢复。询问后得知:

  • 数据盘有数据。无法全格。
  • 客户是小白,对Linux一点不懂。(既不懂如何管理)?

我的解决思路(保险起见):

上一块500G新硬盘,做上linux系统。将先前那块磁盘挂上去,之后拷贝数据。

下面开始。

查看磁盘情况:

[root@renwole-com ~]# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a9411

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          26      204800   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2              26        6553    52428800   83  Linux
/dev/sda3            6553        7075     4194304   82  Linux swap / Solaris
/dev/sda4            7075       60802   431557656    5  Extended
/dev/sda5            7076       60802   431556608   83  Linux

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xa154a154

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          64      512000   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sdb2              64       60802   487873536   8e  Linux LVM

Disk /dev/mapper/VolGroup-lv_root: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/VolGroup-lv_home: 441.8 GB, 441765068800 bytes
255 heads, 63 sectors/track, 53708 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
...

说明:红色部分是以前系统坏的老盘。

安装 lvm2:

[root@renwole-com ~]# yum install lvm2 -y

查看物理卷:

[root@renwole-com ~]# pvs
PV         VG       Fmt  Attr PSize   PFree
/dev/sdb2  VolGroup lvm2 a--u 465.27g    0

查看卷组:

[root@renwole-com ~]# lvdisplay
--- Logical volume ---
LV Path                /dev/VolGroup/lv_root
LV Name                lv_root
VG Name                VolGroup
LV UUID                3Y5UYD-x0H9-F25h-foUz-rm0O-cgzl-pE8FbF
LV Write Access        read/write
LV Creation host, time localhost.localdomain, 2015-07-20 17:29:35 +0800
LV Status              available
# open                 0
LV Size                50.00 GiB
Current LE             12800
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:0

--- Logical volume ---
LV Path                /dev/VolGroup/lv_home
LV Name                lv_home
VG Name                VolGroup
LV UUID                Eu7UKx-LPkM-RDQ6-ACym-KOOW-t2RC-oVq0Zn
LV Write Access        read/write
LV Creation host, time localhost.localdomain, 2015-07-20 17:29:56 +0800
LV Status              available
# open                 1
LV Size                411.43 GiB
Current LE             105325
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:1

--- Logical volume ---
LV Path                /dev/VolGroup/lv_swap
LV Name                lv_swap
VG Name                VolGroup
LV UUID                PBkdrg-lhsO-vkde-RvC0-ryUh-p0pf-aMnBds
LV Write Access        read/write
LV Creation host, time localhost.localdomain, 2015-07-20 17:32:40 +0800
LV Status              available
# open                 0
LV Size                3.84 GiB
Current LE             984
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     256
Block device           253:2

激活卷组:

[root@renwole-com ~]# vgchange -ay /dev/VolGroup
3 logical volume(s) in volume group "VolGroup" now active

挂载硬盘:

[root@renwole-com ~]# mount /dev/VolGroup/lv_home /apps/

/apps是我新建的目录。

查看挂载情况:

[root@renwole-com ~]# df -h
Filesystem                    Size  Used Avail Use% Mounted on
/dev/sda2                      50G  2.1G   45G   5% /
tmpfs                         3.9G     0  3.9G   0% /dev/shm
/dev/sda1                     194M   34M  151M  19% /boot
/dev/sda5                     406G  199M  385G   1% /home
/dev/mapper/VolGroup-lv_home  405G  199M  385G   1% /apps

进入apps目录一看,哇;数据还在,可喜可贺呀。

另外:如果挂载报错:

mount: you must specify the filesystem type

可以使用以下方法查看磁盘格式类型,然后加参数 -t 指定格式挂载即可。

[root@renwole-com ~]# file -s /dev/sda1
/dev/sda1: Linux rev 1.0 ext4 filesystem data (needs journal recovery) (extents) (huge files)

重要说明:

未找到合适的解决方案前,切勿对硬盘删分区,否则会损坏数据。
即便挂载成功,切勿使用mv或rm命令,请使用cp命令。
特别注意:在Linux下,若使用软件恢复数据,几乎是不现实的。