Nginx + Keepalived 高可用集群部署

負(fù)載均衡技術(shù)對于一個網(wǎng)站尤其是大型網(wǎng)站的web服務(wù)器集群來說是至關(guān)重要的！做好負(fù)載均衡架構(gòu)，可以實(shí)現(xiàn)故障轉(zhuǎn)移和高可用環(huán)境，避免單點(diǎn)故障，保證網(wǎng)站健康持續(xù)運(yùn)行。在使用 Nginx 做反向代理或者負(fù)載均衡的時候，都是以 Nginx 為入口，如果 Nginx 宕機(jī)了，那么所有的服務(wù)都無法正常提供，影響非常嚴(yán)重。

為了避免負(fù)載均衡服務(wù)器宕機(jī)造成嚴(yán)重影響，就需要建立一個備份機(jī)。主服務(wù)器和備份機(jī)上都運(yùn)行高可用（High Availability）監(jiān)控程序，通過傳送諸如“I am alive”這樣的信息來監(jiān)控對方的運(yùn)行狀況。當(dāng)備份機(jī)不能在一定的時間內(nèi)收到這樣的信息時，它就接管主服務(wù)器的服務(wù)IP并繼續(xù)提供負(fù)載均衡服務(wù)；當(dāng)備份管理器又從主管理器收到“I am alive”這樣的信息時，它就釋放服務(wù)IP地址，這樣的主服務(wù)器就開始再次提供負(fù)載均衡服務(wù)。

高可用（High Availability）是分布式系統(tǒng)架構(gòu)設(shè)計(jì)中必須考慮的因素之一，它通常是指，通過設(shè)計(jì)減少系統(tǒng)不能提供服務(wù)的時間。如果一個系統(tǒng)能夠一直提供服務(wù)，那么這個可用性則是百分之百，但是我們不能保證一個系統(tǒng)能永遠(yuǎn)不出問題，所以我們只能通過設(shè)計(jì)來盡可能的去減少由于系統(tǒng)的故障所造成的影響。

由于業(yè)務(wù)擴(kuò)展，網(wǎng)站的訪問量不斷加大，負(fù)載越來越高。現(xiàn)需要在web前端放置nginx負(fù)載均衡,同時結(jié)合keepalived對前端nginx實(shí)現(xiàn)HA高可用。

前文分享了《Linux下Nginx基礎(chǔ)應(yīng)用》，《Linux下實(shí)現(xiàn)高可用軟件-Keepalived基礎(chǔ)知識梳理》；今天簡單分享Nginx + Keepalived 高可用集群部署。

主從集群架構(gòu)圖

環(huán)境說明

hostname	ip	說明
Client-01	172.16.70.171	客戶端測試機(jī)
KeepMaster	172.16.70.181	keepalived 主服務(wù)器 (nginx 主負(fù)載均衡器)
KeepBackup	172.16.70.182	keepalived 備服務(wù)器 (nginx 備負(fù)載均衡器)
VIP	172.16.70.183	vrrp HA 虛擬地址，可有多個IP
Web1	172.16.70.191	后端web主服務(wù)器1 (nginx 站點(diǎn))
Web2	172.16.70.192	后端web備服務(wù)器2 (nginx 站點(diǎn))

準(zhǔn)備環(huán)境

# 本次所有部署服務(wù)器都配置
# cat /etc/redhat-release 
CentOS Linux release 7.9.2009 (Core)
# uname -r
3.10.0-1160.83.1.el7.x86_64

# systemctl stop firewalld
# sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/sysconfig/selinux
# setenforce 0
# ntpdate 0.centos.pool.ntp.org
# yum install net-tools vim wget curl -y

搭建后端web服務(wù)器

主備一樣操作

# 這里以 web-01 為例
[root@web-01 ~] # wget https://nginx.org/packages/centos/7/x86_64/RPMS/nginx-1.20.2-1.el7.ngx.x86_64.rpm
[root@web-01 ~] # rpm -ivh nginx-1.20.2-1.el7.ngx.x86_64.rpm
[root@web-01 ~] # nginx -v
nginx version: nginx /1 .20.2

[root@web-01 ~] # #echo "`hostname` `ifconfig ens33 |sed -n 's#.*inet \(.*\)netmask.*#\1#p'`" > /usr/share/nginx/html/index.html
[root@web-01 ~] # cat /usr/share/nginx/html/index.html
web-01 172.16.70.191 

[root@web-01 ~] # nginx -t
nginx: the configuration  file /etc/nginx/nginx .conf syntax is ok
nginx: configuration  file /etc/nginx/nginx .conf  test is successful

[root@web-01 ~] # systemctl start nginx
[root@web-01 ~] # systemctl enable nginx
[root@web-01 ~] # netstat -ntupl | grep nginx
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      10687 /nginx : master 
[root@web-01 ~] # ps -ef | grep nginx
root      10687      1  0 16:36 ?        00:00:00 nginx: master process  /usr/sbin/nginx -c  /etc/nginx/nginx .conf
nginx     10688  10687  0 16:36 ?        00:00:00 nginx: worker process
nginx     10689  10687  0 16:36 ?        00:00:00 nginx: worker process
nginx     10690  10687  0 16:36 ?        00:00:00 nginx: worker process
nginx     10691  10687  0 16:36 ?        00:00:00 nginx: worker process
root      10761  10586  0 16:45 pts /1 00:00:00  grep --color=auto nginx

# Client-01 測試訪問
[root@Client-01 ~] # curl 172.16.70.191
web-01 172.16.70.191

瀏覽器測試訪問 http://ip/

Keep服務(wù)器上部署nginx負(fù)載均衡器

主備一樣操作

# 這里以 KeepMaster 為例
# 安裝部署nginx
[root@KeepMaster ~] # wget https://nginx.org/packages/centos/7/x86_64/RPMS/nginx-1.18.0-2.el7.ngx.x86_64.rpm
[root@KeepMaster ~] # rpm -vih nginx-1.18.0-2.el7.ngx.x86_64.rpm
[root@KeepMaster ~] # nginx -v
nginx version: nginx /1 .18.0

# 新建
[root@KeepMaster ~] # cat /etc/nginx/conf.d/web.conf
upstream web {
    server 172.16.70.191:80 weight=1 max_fails=3 fail_timeout=20s;
    server 172.16.70.192:80 weight=2 max_fails=3 fail_timeout=20s;
}
    # weight(權(quán)重)和訪問比率成正比，默認(rèn)值為1
    # max_fails 為允許失敗的次數(shù)，默認(rèn)值為1
    # fail_timeout 當(dāng)max_fails次失敗后，暫停將請求分發(fā)到該后端服務(wù)器的時間

server {
  listen 80;
  server_name www.zhangwencheng.org;
 
  location / {
    proxy_pass http: //web ;
    proxy_set_header HOST $http_host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  }
}

[root@KeepMaster ~] # nginx -t
nginx: the configuration  file /etc/nginx/nginx .conf syntax is ok
nginx: configuration  file /etc/nginx/nginx .conf  test is successful

[root@KeepMaster ~] # systemctl start nginx
[root@KeepMaster ~] # systemctl enable nginx

[root@KeepMaster ~] # ps -ef | grep nginx
root       1677      1  0 17:28 ?        00:00:00 nginx: master process  /usr/sbin/nginx -c  /etc/nginx/nginx .conf
nginx      1678   1677  0 17:28 ?        00:00:00 nginx: worker process
root       1708   1444  0 17:33 pts /0 00:00:00  grep --color=auto nginx
[root@KeepMaster ~] # netstat -tnpl | grep nginx
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1677 /nginx : master

測試機(jī)Client-01驗(yàn)證負(fù)載均衡

Keep主備服務(wù)器上的nginx負(fù)載均衡

# 在測試機(jī)上添加host解析, KeepMaster/KeepBackup主機(jī)IP
[root@Client-01 ~] # cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.70.181   www.zhangwencheng.org
172.16.70.182   www.zhangwencheng.org

# 測試時候輪流關(guān)閉Keep節(jié)點(diǎn)，關(guān)閉后還是能夠訪問并看到輪循效果即表示nginx負(fù)載均衡器集群搭建成功。
[root@Client-01 ~] # curl www.zhangwencheng.org
web-01 172.16.70.191  
[root@Client-01 ~] # curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~] # curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~] # curl www.zhangwencheng.org
web-01 172.16.70.191

Keep服務(wù)器上部署keepalived

主備一樣操作

[root@KeepMaster ~] # yum install -y openssl openssl-devel libnl libnl-devel gcc
[root@KeepMaster ~] # mkdir /data/apps/keepalived -p
[root@KeepMaster ~] # wget --no-check-certificate  http://www.keepalived.org/software/keepalived-2.2.4.tar.gz
[root@KeepMaster ~] # tar -xf keepalived-2.2.4.tar.gz
[root@KeepMaster ~] # cd keepalived-2.2.4
[root@KeepMaster keepalived-2.2.4] # ls
aclocal.m4  autogen.sh   build-aux    ChangeLog  configure.ac  COPYING  INSTALL     keepalived.spec. in m4           Makefile. in snap  tools
AUTHOR      bin_install  build_setup  configure  CONTRIBUTORS  doc      keepalived  lib                 Makefile.am  README.md    TODO

[root@KeepMaster keepalived-2.2.4] # ./configure --prefix=/data/apps/keepalived
....
....
# 最后編譯正常輸出如下
Keepalived configuration
------------------------
Keepalived version       : 2.2.4
Compiler                 :  gcc gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Preprocessor flags       : -D_GNU_SOURCE
Compiler flags           : -g -g -O2 -Wextra -Wunused -Wstrict-prototypes -Wabi -Wbad- function -cast -Wcast-align -Wcast-qual -Wdisabled-optimization -Wdouble-promotion \
-Wfloat-equal -Wframe-larger-than=5120 -Winit-self -Winline -Winvalid-pch -Wjump-misses-init -Wlogical- op -Wmissing-declarations -Wmissing-field-initializers -Wmissing-include- dirs \
-Wmissing-prototypes -Wnested-externs -Wold-style-definition -Woverlength- strings -Wpointer-arith -Wredundant-decls -Wshadow -Wstack-protector -Wstrict-overflow=4 -Wsuggest-attribute= format \
-Wsuggest-attribute=noreturn -Wsuggest-attribute=pure -Wsync-nand -Wtrampolines -Wundef -Wuninitialized -Wunknown-pragmas -Wunsafe-loop-optimizations -Wunsuffixed-float-constants -Wvariadic-macros \
-Wwrite- strings -fPIE -Wformat -Werror= format -security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord- gcc -switches -O2
Linker flags             : -pie -Wl,-z,relro -Wl,-z,now
Extra Lib                : -lm -lcrypto -lssl -lnl
Use IPVS Framework       : Yes
IPVS use libnl           : Yes
IPVS syncd attributes    : No
IPVS 64 bit stats        : No
HTTP_GET regex support   : No
fwmark socket support    : Yes
Use VRRP Framework       : Yes
Use VRRP VMAC            : Yes
Use VRRP authentication  : Yes
With track_process       : Yes
With linkbeat            : Yes
Use BFD Framework        : No
SNMP vrrp support        : No
SNMP checker support     : No
SNMP RFCv2 support       : No
SNMP RFCv3 support       : No
DBUS support             : No
Use JSON output          : No
libnl version            : 1
Use IPv4 devconf         : No
Use iptables             : No
Use nftables             : No
init  type : systemd
systemd notify           : No
Strict config checks     : No
Build documentation      : No
Default runtime options  : -D

[root@KeepMaster keepalived-2.2.4] # make -j 4 && make install
[root@KeepMaster keepalived-2.2.4] # ls
aclocal.m4  bin          build_setup  config.status  CONTRIBUTORS  INSTALL          keepalived.spec. in Makefile     README     TODO
AUTHOR      bin_install  ChangeLog    configure      COPYING       keepalived       lib                 Makefile.am  README.md  tools
autogen.sh  build-aux    config.log   configure.ac   doc           keepalived.spec  m4                  Makefile. in snap

[root@KeepMaster keepalived-2.2.4] # cp keepalived/keepalived /usr/local/sbin/ -a
[root@KeepMaster keepalived-2.2.4] # keepalived -v
Keepalived v2.2.4 (08 /21 ,2021)

Copyright(C) 2001-2021 Alexandre Cassen, <acassen@gmail.com>

Built with kernel headers  for Linux 3.10.0
Running on Linux 3.10.0-1160.83.1.el7.x86_64  #1 SMP Wed Jan 25 16:41:43 UTC 2023
Distro: CentOS Linux 7 (Core)

configure options: --prefix= /data/apps/keepalived

Config options:  LVS VRRP VRRP_AUTH VRRP_VMAC OLD_CHKSUM_COMPAT INIT=systemd

System options:  VSYSLOG LIBNL1 RTA_ENCAP RTA_EXPIRES RTA_PREF FRA_SUPPRESS_PREFIXLEN FRA_TUN_ID RTAX_CC_ALGO RTAX_QUICKACK RTA_VIA IFA_FLAGS \
NET_LINUX_IF_H_COLLISION LIBIPTC_LINUX_NET_IF_H_COLLISION LIBIPVS_NETLINK IFLA_LINK_NETNSID GLOB_BRACE GLOB_ALTDIRFUNC INET6_ADDR_GEN_MODE SO_MARK

[root@KeepMaster keepalived-2.2.4] # cd /data/apps/keepalived/
[root@KeepMaster keepalived] # ls
bin  etc  sbin  share

[root@KeepMaster keepalived] # mv etc/keepalived/keepalived.conf etc/keepalived/keepalived.conf_bak
[root@KeepMaster keepalived] # vim etc/keepalived/keepalived.conf
! Configuration File  for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_script chk_nginx {     
    script  "/data/apps/keepalived/chk_nginx.sh"
    interval 2                  
    weight -5                                   
}

vrrp_instance VI_1 {
    state MASTER　　　　 # 備服務(wù)器這為 BACKUP
    interface ens33
    virtual_router_id 51
    priority 110　　　　 # 備服務(wù)器這小于110
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       172.16.70.183 
    }
track_script {
   chk_nginx
    }
}

優(yōu)先級不會不斷的提高或者降低，最終優(yōu)先級的范圍是在[1,254]，不會出現(xiàn)優(yōu)先級小于等于0或者優(yōu)先級大于等于255的情況。
在MASTER節(jié)點(diǎn)的vrrp_instance中配置nopreempt，當(dāng)它異常恢復(fù)后，即使它prio更高也不會搶占，這樣可以避免正常情況下做無謂的切換。

nginx檢測腳本

編寫腳本來判斷本機(jī)nginx是否正常，如果發(fā)現(xiàn)NginX不正常，自重啟nginx。等待2秒再次校驗(yàn)，仍然失敗則不再嘗試，關(guān)閉keepalived，讓其他主機(jī)此時會接管VIP。
此腳本必須在keepalived服務(wù)運(yùn)行的前提下才有效！如果在keepalived服務(wù)先關(guān)閉的情況下，那么nginx服務(wù)關(guān)閉后就不能實(shí)現(xiàn)自啟動了。

[root@KeepMaster keepalived] # cat chk_nginx.sh 
#!/bin/bash
chk=$( ps -C nginx --no-heading| wc -l)
if [  "${chk}" =  "0" ];  then
    systemctl start nginx
    sleep 2
    chk=$( ps -C nginx --no-heading| wc -l)
    if [  "${chk}" =  "0" ];  then
        systemctl stop keepalived
    fi
fi

[root@KeepMaster keepalived] # chmod +x chk_nginx.sh

測試機(jī)Client-01驗(yàn)證VIP

Keep服務(wù)器上的VIP

[root@Client-01 ~] # cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.16.70.183   www.zhangwencheng.org

[root@Client-01 ~] # curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~] # curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~] # curl www.zhangwencheng.org
web-01 172.16.70.191  
[root@Client-01 ~] # curl www.zhangwencheng.org
web-02 172.16.70.192

故障轉(zhuǎn)移測試

#手動關(guān)閉Master機(jī)器上的nginx服務(wù)，最多2秒鐘后就會自啟動
[root@KeepMaster ~] # systemctl stop nginx
[root@KeepMaster ~] # ps -ef | egrep 'nginx|keepalived'
root      57266      1  0 16:21 ?        00:00:00  /data/apps/keepalived/sbin/keepalived -f  /data/apps/keepalived/etc/keepalived/keepalived .conf -D
root      57267  57266  0 16:21 ?        00:00:01  /data/apps/keepalived/sbin/keepalived -f  /data/apps/keepalived/etc/keepalived/keepalived .conf -D
root      60019      1  0 16:42 ?        00:00:00 nginx: master process  /usr/sbin/nginx -c  /etc/nginx/nginx .conf
nginx     60020  60019  0 16:42 ?        00:00:00 nginx: worker process
root      60027   1444  0 16:42 pts /0 00:00:00  grep -E --color=auto nginx|keepalived
[root@KeepMaster ~] # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link /loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1 /8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1 /128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link /ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
    inet 172.16.70.181 /24 brd 172.16.70.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 172.16.70.183 /32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::7726:d409:2cf4:babd /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::833:43b:7d2:6e4c /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::c2be:590b:1ae6:42e3 /64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

#手動關(guān)閉Master機(jī)器上的keepalived服務(wù)，已經(jīng)發(fā)現(xiàn)沒VIP了
[root@KeepMaster ~] # systemctl stop keepalived
[root@KeepMaster ~] # ps -ef | egrep 'nginx|keepalived'
root      60019      1  0 16:42 ?        00:00:00 nginx: master process  /usr/sbin/nginx -c  /etc/nginx/nginx .conf
nginx     60020  60019  0 16:42 ?        00:00:00 nginx: worker process
root      60348   1444  0 16:45 pts /0 00:00:00  grep -E --color=auto nginx|keepalived
[root@KeepMaster ~] # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link /loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1 /8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1 /128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link /ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
    inet 172.16.70.181 /24 brd 172.16.70.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::7726:d409:2cf4:babd /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::833:43b:7d2:6e4c /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::c2be:590b:1ae6:42e3 /64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

[root@KeepMaster keepalived] # tail /var/log/messages
Mar  7 17:01:01 Keepalived-01 systemd: Started Session 27 of user root.
Mar  7 17:01:28 Keepalived-01 Keepalived[60703]: Stopping
Mar  7 17:01:28 Keepalived-01 systemd: Stopping LVS and VRRP High Availability Monitor...
Mar  7 17:01:28 Keepalived-01 Keepalived_vrrp[60704]: (VI_1) sent 0 priority
Mar  7 17:01:28 Keepalived-01 Keepalived_vrrp[60704]: (VI_1) removing VIPs.
Mar  7 17:01:28 Keepalived-01 NetworkManager[570]: <info>  [1678179688.0176] policy:  set - hostname : current  hostname was changed outside NetworkManager:  'KeepMaster'
Mar  7 17:01:29 Keepalived-01 Keepalived_vrrp[60704]: Stopped - used (self /children ) 0.005506 /0 .797936 user  time , 0.172766 /0 .818969 system  time
Mar  7 17:01:29 Keepalived-01 Keepalived[60703]: CPU usage (self /children ) user: 0.000000 /0 .803442 system: 0.001394 /0 .994146
Mar  7 17:01:29 Keepalived-01 Keepalived[60703]: Stopped Keepalived v2.2.4 (08 /21 ,2021)
Mar  7 17:01:29 Keepalived-01 systemd: Stopped LVS and VRRP High Availability Monitor

KeepBackup查看，已經(jīng)接管VIP

[root@KeepBackup keepalived] # ps -ef | egrep 'nginx|keepalived'
root      65036      1  0 17:01 ?        00:00:00  /data/apps/keepalived/sbin/keepalived -f  /data/apps/keepalived/etc/keepalived/keepalived .conf -D
root      65037  65036  0 17:01 ?        00:00:00  /data/apps/keepalived/sbin/keepalived -f  /data/apps/keepalived/etc/keepalived/keepalived .conf -D
root      65067      1  0 17:01 ?        00:00:00 nginx: master process  /usr/sbin/nginx -c  /etc/nginx/nginx .conf
nginx     65068  65067  0 17:01 ?        00:00:00 nginx: worker process
root      65122   1514  0 17:01 pts /0 00:00:00  grep -E --color=auto nginx|keepalived

[root@KeepBackup keepalived] # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link /loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1 /8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1 /128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link /ether 00:0c:29:c4:54:23 brd ff:ff:ff:ff:ff:ff
    inet 172.16.70.182 /24 brd 172.16.70.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 172.16.70.183 /32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::7726:d409:2cf4:babd /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::833:43b:7d2:6e4c /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::c2be:590b:1ae6:42e3 /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
[root@KeepBackup keepalived] # tail /var/log/messages 
Mar  7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33  for 172.16.70.183
Mar  7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33  for 172.16.70.183
Mar  7 17:01:28 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33  for 172.16.70.183
Mar  7 17:01:28 Keepalived-02 NetworkManager[573]: <info>  [1678179688.8137] policy:  set - hostname : current  hostname was changed outside NetworkManager:  'KeepBackup'
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: (VI_1) Sending /queueing gratuitous ARPs on ens33  for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33  for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33  for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33  for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33  for 172.16.70.183
Mar  7 17:01:33 Keepalived-02 Keepalived_vrrp[65037]: Sending gratuitous ARP on ens33  for 172.16.70.183

雙機(jī)高可用一般是通過虛擬IP（飄移IP）方法來實(shí)現(xiàn)的，基于Linux/Unix的IP別名技術(shù)，目前分為兩種：

雙機(jī)主從模式：即前端使用兩臺服務(wù)器，一臺主服務(wù)器和一臺熱備服務(wù)器，正常情況下，主服務(wù)器綁定一個公網(wǎng)虛擬IP，提供負(fù)載均衡服務(wù)，熱備服務(wù)器處于空閑狀態(tài)；當(dāng)主服務(wù)器發(fā)生故障時，熱備服務(wù)器接管主服務(wù)器的公網(wǎng)虛擬IP，提供負(fù)載均衡服務(wù)；但是熱備服務(wù)器在主機(jī)器不出現(xiàn)故障的時候，永遠(yuǎn)處于浪費(fèi)狀態(tài)，對于服務(wù)器不多的網(wǎng)站，該方案不經(jīng)濟(jì)實(shí)惠。
雙機(jī)主主模式：即前端使用兩臺負(fù)載均衡服務(wù)器，互為主備，且都處于活動狀態(tài)，同時各自綁定一個公網(wǎng)虛擬IP，提供負(fù)載均衡服務(wù)；當(dāng)其中一臺發(fā)生故障時，另一臺接管發(fā)生故障服務(wù)器的公網(wǎng)虛擬IP（這時由非故障機(jī)器一臺負(fù)擔(dān)所有的請求）。這種方案，經(jīng)濟(jì)實(shí)惠，非常適合于當(dāng)前架構(gòu)環(huán)境。

主主模式集群架構(gòu)圖

當(dāng)了解主備模式后，雙主模式就容易配置多了。只需要在每臺keepalived配置文件，加上一個vrrp_instance命名vrrp_instance VI_2即可，更改幾個參數(shù)，設(shè)置另一個VIP：172.16.70.184

KeepMaster：state BACKUP ，priority 100， virtual_router_id 52
KeepBackup：state MASTER ，priority 110， virtual_router_id 52

KeepMaster上的keepalived.conf

[root@KeepMaster keepalived] # cat etc/keepalived/keepalived.conf
! Configuration File  for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_script chk_nginx {     
    script  "/data/apps/keepalived/chk_nginx.sh"
    interval 2                  
    weight -5                   
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 110
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       172.16.70.183 
    }
track_script {
   chk_nginx
    }
}

vrrp_instance VI_2 {
    state BACKUP
    interface ens33
    virtual_router_id 52
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       172.16.70.184
    }
track_script {
   chk_nginx
    }
}

[root@KeepMaster keepalived] # systemctl restart keepalived

KeepBackup上的keepalived.conf

[root@KeepBackup keepalived] # cat etc/keepalived/keepalived.conf
! Configuration File  for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_script chk_nginx {     
    script  "/data/apps/keepalived/chk_nginx.sh"
    interval 2                  
    weight -5                   
}

vrrp_instance VI_1 {
    state BAKCUP
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       172.16.70.183 
    }
track_script {
   chk_nginx
    }
}

vrrp_instance VI_2 {
    state MASTER
    interface ens33
    virtual_router_id 52
    priority 110
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
       172.16.70.184
    }
track_script {
   chk_nginx
    }
}

[root@KeepBackup keepalived] # systemctl restart keepalived

測試機(jī)Client-01驗(yàn)證VIP2

Keep服務(wù)器上的VIP2

# 注釋VIP1解析，此時僅測試VIP2；測試成功后再取消VIP1注釋
[root@Client-01 ~] # cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
#172.16.70.183  www.zhangwencheng.org
172.16.70.184   www.zhangwencheng.org

[root@Client-01 ~] # curl www.zhangwencheng.org
web-01 172.16.70.191  
[root@Client-01 ~] # curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~] # curl www.zhangwencheng.org
web-02 172.16.70.192  
[root@Client-01 ~] # curl www.zhangwencheng.org
web-01 172.16.70.191

查看VIP情況

[root@KeepMaster ~] # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link /loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1 /8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1 /128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link /ether 00:0c:29:a1:82:4e brd ff:ff:ff:ff:ff:ff
    inet 172.16.70.181 /24 brd 172.16.70.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 172.16.70.183 /32 scope global ens33　　　　 # VIP1
       valid_lft forever preferred_lft forever
    inet6 fe80::7726:d409:2cf4:babd /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::833:43b:7d2:6e4c /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::c2be:590b:1ae6:42e3 /64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

[root@KeepBackup ~] # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link /loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1 /8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1 /128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link /ether 00:0c:29:c4:54:23 brd ff:ff:ff:ff:ff:ff
    inet 172.16.70.182 /24 brd 172.16.70.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 172.16.70.184 /32 scope global ens33　　　　 # VIP2
       valid_lft forever preferred_lft forever
    inet6 fe80::7726:d409:2cf4:babd /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::833:43b:7d2:6e4c /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever
    inet6 fe80::c2be:590b:1ae6:42e3 /64 scope link tentative noprefixroute dadfailed 
       valid_lft forever preferred_lft forever

故障轉(zhuǎn)移測試類同，不再贅述。

posted @ 2023-03-09 09:02 講文張字閱讀(1507) 評論(1) 收藏舉報

刷新頁面返回頂部

返回頂部