LVS (Linux Virtual Server)구축 : ldirectord + heartbeat
L4 스위치의 부하 분산 기능을 리눅스 서버를 통해 진행하고자 한다.
서버의 구성은 아래와 같이 하면 될 것 같다.
| VIP : 121.254.175.70LVS1 : 121.254.175.77LVS2 : 121.254.175.78WEB1 : 121.254.175.75WEB2 : 121.254.175.74 |
임의 설정한 호스트네임 정보를 각 LVS 서버에 등록을 해야 한다.
| /etc/hosts /etc/sysconfig/network /proc/sys/kernel/hostname [Hostname]LVS Active: lvsactive.gunnm.xyzLVS Standby: lvsstandby.gunnm.xyz |
1. LVS Active 서버에서의 설정 : 121.254.175.77
| # echo ‘lvsactive.gunnm.xyz’ > /proc/sys/kernel/hostname |
| # vi /etc/sysconfig/network NETWORKING=yesHOSTNAME=lvsactive.gunnm.xyz |
| # vi /etc/hosts 121.254.175.77 lvsactive.gunnm.xyz121.254.175.78 lvsstandby.gunnm.xyz |
| # vi /etc/sysctl.conf net.ipv4.ip_forward = 1 # 하단에 추가net.ipv4.vs.secure_tcp=3net.ipv4.vs.drop_packet=0net.ipv4.vs.drop_entry=0net.ipv4.vs.am_droprate=10net.ipv4.vs.amemthresh=1024 # sysctl -p |
| heartbeat + ldirectord 설치 # rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm# yum install heartbeat* -y # yum -y install ipvsadm# yum -y install perl-IO-Socket-INET6# yum -y install perl-Email-Send# yum -y install perl-MailTools# yum -y install perl-Net-SSLeay# yum -y install perl-libwww-perl# yum install ldirect* 빨간색은 실패하는 설치 패키지이다. 아래의 링크를 통해 ldirectord 파일 다운로드 및 설치를 진행해야 한다. – 32bit# wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/i686/ldirectord-3.9.5-3.1.i686.rpm – 64bit# wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/x86_64/ldirectord-3.9.5-3.1.x86_64.rpm # rpm -ivh ldirectord-3.9.5-3.1.x86_64.rpm |
| HA 설정 (Standby 의 상태 체크)/usr/share/doc/heartbeat-3.0.4/ha.cf 를 복사해와도 되지만 그냥 새로 만들고 아래 내용을 입력한다. # vi /etc/ha.d/ha.cf debugfile /var/log/ha-debug.loglogfile /var/log/ha.logkeepalive 2deadtime 10udpport 694ucast eth0 121.254.175.78node lvsactive.gunnm.xyznode lvsstandby.gunnm.xyzping 121.254.175.65auto_failback onrespawn hacluster /usr/lib64/heartbeat/ipfailapiauth ipfail gid=haclient uid=hacluster |
| 가상IP 를 설정한다. # vi /etc/ha.d/haresources lvsactive.gunnm.xyz \ldirectord::ldirectord.cf \IPaddr2::121.254.175.70/32/eth0 |
| authkey 를 생성한다. # vi /etc/ha.d/authkeys auth 22 sha1 gunnmlvs root 만 읽을수 있도록 권한을 설정한다. # chmod 600 /etc/ha.d/authkeys -rw——-. 1 root root 23 2018-08-08 13:07 /etc/ha.d/authkeys |
| 분배 설정을 진행한다.# vi /etc/ha.d/ldirectord.cf checktimeout=10checkinterval=10autoreload=nologfile=”/var/log/ldirectord.log”quiescent=no# Web Server LVSvirtual=121.254.175.70:80real=121.254.175.75:80 gate 10real=121.254.175.74:80 gate 10service=httprequest=”gunnm.lvs”receive=”gunnm”scheduler=wlcpersistent=3600protocol=tcpchecktype=negotiate |
| 세션 동기화를 위한 방화벽 룰셋 추가 # iptables -t nat -A OUTPUT -d 224.0.0.81 -p udp –dport 8848 -j DNAT –to 121.254.175.78:8848# iptables -t nat -A PREROUTING -s 121.254.175.78 -p udp –dport 8848 -j DNAT –to 224.0.0.81:8848# iptables -t mangle -A PREROUTING -p tcp -s 0.0.0.0/0 -d 121.254.175.70/32 –dport 80 -j MARK –set-mark 1# iptables -t mangle -A PREROUTING -p tcp -s 0.0.0.0/0 -d 121.254.175.70/32 –dport 443 -j MARK –set-mark 1 # service iptables save # service iptables restart |
2. LVS Standby 서버에서의 설정 : 121.254.175.78
| # echo ‘lvsstandby.gunnm.xyz’ > /proc/sys/kernel/hostname |
| # vi /etc/sysconfig/network NETWORKING=yesHOSTNAME=lvsstandby.gunnm.xyz |
| # vi /etc/hosts 121.254.175.77 lvsactive.gunnm.xyz121.254.175.78 lvsstandby.gunnm.xyz |
| # vi /etc/sysctl.conf net.ipv4.ip_forward = 1 # 하단에 추가net.ipv4.vs.secure_tcp=3net.ipv4.vs.drop_packet=0net.ipv4.vs.drop_entry=0net.ipv4.vs.am_droprate=10net.ipv4.vs.amemthresh=1024 # sysctl -p |
| heartbeat + ldirectord 설치 # rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm# yum install heartbeat* -y # yum -y install ipvsadm# yum -y install perl-IO-Socket-INET6# yum -y install perl-Email-Send# yum -y install perl-MailTools# yum -y install perl-Net-SSLeay# yum -y install perl-libwww-perl# yum install ldirect* 빨간색은 실패하는 설치 패키지이다. 아래의 링크를 통해 ldirectord 파일 다운로드 및 설치를 진행해야 한다. – 32bit# wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/i686/ldirectord-3.9.5-3.1.i686.rpm – 64bit# wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/x86_64/ldirectord-3.9.5-3.1.x86_64.rpm # rpm -ivh ldirectord-3.9.5-3.1.x86_64.rpm |
| HA 설정 (Active 의 상태 체크)/usr/share/doc/heartbeat-3.0.4/ha.cf 를 복사해와도 되지만 그냥 새로 만들고 아래 내용을 입력한다. # vi /etc/ha.d/ha.cf debugfile /var/log/ha-debug.loglogfile /var/log/ha.logkeepalive 2deadtime 10udpport 694ucast eth0 121.254.175.77node lvsactive.gunnm.xyznode lvsstandby.gunnm.xyzping 121.254.175.65auto_failback onrespawn hacluster /usr/lib64/heartbeat/ipfailapiauth ipfail gid=haclient uid=hacluster |
| 가상IP 를 설정한다. # vi /etc/ha.d/haresources lvsstandby.gunnm.xyz \ldirectord::ldirectord.cf \IPaddr2::121.254.175.70/32/eth0 |
| authkey 를 생성한다. # vi /etc/ha.d/authkeys auth 22 sha1 gunnmlvs root 만 읽을수 있도록 권한을 설정한다. # chmod 600 /etc/ha.d/authkeys -rw——-. 1 root root 23 2018-08-08 14:12 /etc/ha.d/authkeys |
| 분배 설정을 진행한다.# vi /etc/ha.d/ldirectord.cf checktimeout=10checkinterval=10autoreload=nologfile=”/var/log/ldirectord.log”quiescent=no# Web Server LVSvirtual=121.254.175.70:80real=121.254.175.74:80 gate 10real=121.254.175.75:80 gate 10service=httprequest=”gunnm.lvs”receive=”gunnm”scheduler=wlcpersistent=3600protocol=tcpchecktype=negotiate |
| 세션 동기화를 위한 방화벽 룰셋 추가 # iptables -t nat -A OUTPUT -d 224.0.0.81 -p udp –dport 8848 -j DNAT –to 121.254.175.77:8848# iptables -t nat -A PREROUTING -s 121.254.175.77 -p udp –dport 8848 -j DNAT –to 224.0.0.81:8848# iptables -t mangle -A PREROUTING -p tcp -s 0.0.0.0/0 -d 121.254.175.70/32 –dport 80 -j MARK –set-mark 1# iptables -t mangle -A PREROUTING -p tcp -s 0.0.0.0/0 -d 121.254.175.70/32 –dport 443 -j MARK –set-mark 1 # service iptables save # service iptables restart |
3. LVS Active 및 Standby 서버에서 Heartbeat 구동
| # service heartbeat start Starting High-Availability services: ERROR: Cannot locate resource script \ldirectord/usr/share/heartbeat/ResourceManager: line 209: ldirectord.cf: command not foundDone. 서비스 구동이 실패한다! 실패 원인을 확인해 본다. /etc/ha.d/ldirectord.cf 의 구문 띄어쓰기에 문제가 있었음이 확인됐다. (별 말도 안되는 걸로 오류가 남) checktimeout=10checkinterval=10autoreload=nologfile=”/var/log/ldirectord.log”quiescent=no# Web Server LVSvirtual=121.254.175.70:80 real=121.254.175.75:80 gate 10 real=121.254.175.74:80 gate 10 service=http request=”gunnm.lvs” receive=”gunnm” scheduler=wlc persistent=3600 protocol=tcp checktype=negotiate 이렇게 수정하고 나니깐 정상적으로 작동 된다. LVS Active 에서 # ip addr을 통해서 아이피 두 개가 활성화 된 것을 확인 할 수 있다. 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 5e:df:dd:9f:3b:e0 brd ff:ff:ff:ff:ff:ff inet 121.254.175.77/28 brd 121.254.175.79 scope global eth0 inet 121.254.175.70/32 brd 121.254.175.79 scope global eth0 inet6 fe80::5cdf:ddff:fe9f:3be0/64 scope link valid_lft forever preferred_lft forever LVS Standby 에서는 아래와 같이 확인 된다. # ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 2a:8b:f9:ad:f0:30 brd ff:ff:ff:ff:ff:ff inet 121.254.175.78/28 brd 121.254.175.79 scope global eth0 inet6 fe80::288b:f9ff:fead:f030/64 scope link valid_lft forever preferred_lft forever |
4. 강제 failover 발생시 구동 상태 확인
| LVS Active 에서 heartbeat 를 중지한다. # service heartbeat stopStopping High-Availability services: Done. # ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 5e:df:dd:9f:3b:e0 brd ff:ff:ff:ff:ff:ff inet 121.254.175.77/28 brd 121.254.175.79 scope global eth0 inet6 fe80::5cdf:ddff:fe9f:3be0/64 scope link valid_lft forever preferred_lft forever # vi /var/log/ha.logResourceManager(default)[10507]: 2018/08/08_16:33:14 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stopAug 08 16:33:15 lvsactive.gunnm.xyz heartbeat: [10494]: info: All HA resources relinquished.Aug 08 16:33:16 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing /usr/lib64/heartbeat/ipfail process group 9816 with signal 15Aug 08 16:33:17 lvsactive.gunnm.xyz heartbeat: [9790]: WARN: 1 lost packet(s) for [lvsstandby.gunnm.xyz] [542:544]Aug 08 16:33:17 lvsactive.gunnm.xyz heartbeat: [9790]: info: No pkts missing from lvsstandby.gunnm.xyz!Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing HBFIFO process 9793 with signal 15Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing HBWRITE process 9794 with signal 15Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing HBREAD process 9795 with signal 15Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing HBWRITE process 9796 with signal 15Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing HBREAD process 9797 with signal 15Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: Core process 9797 exited. 5 remainingAug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: Core process 9794 exited. 4 remainingAug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: Core process 9795 exited. 3 remainingAug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: Core process 9793 exited. 2 remainingAug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: Core process 9796 exited. 1 remainingAug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: lvsactive.gunnm.xyz Heartbeat shutdown complete. LVS Standby 에서 상태 확인 한다. # ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 2a:8b:f9:ad:f0:30 brd ff:ff:ff:ff:ff:ff inet 121.254.175.78/28 brd 121.254.175.79 scope global eth0 inet 121.254.175.70/32 brd 121.254.175.79 scope global eth0 inet6 fe80::288b:f9ff:fead:f030/64 scope link valid_lft forever preferred_lft forever # vi /var/log/ha.logAug 08 16:33:16 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Received shutdown notice from ‘lvsactive.gunnm.xyz’.Aug 08 16:33:16 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Resources being acquired from lvsactive.gunnm.xyz.Aug 08 16:33:16 lvsstandby.gunnm.xyz heartbeat: [3108]: info: acquire all HA resources (standby).ResourceManager(default)[3135]: 2018/08/08_16:33:16 info: Acquiring resource group: lvsstandby.gunnm.xyz ldirectord::ldirectord.cf IPaddr2::121.254.175.70/32/eth0Aug 08 16:33:16 lvsstandby.gunnm.xyz heartbeat: [3109]: info: Local Resource acquisition completed.ResourceManager(default)[3135]: 2018/08/08_16:33:16 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start/usr/lib/ocf/resource.d//heartbeat/IPaddr2(IPaddr2_121.254.175.70)[3224]: 2018/08/08_16:33:17 INFO: Resource is stoppedResourceManager(default)[3135]: 2018/08/08_16:33:17 info: Running /etc/ha.d/resource.d/IPaddr2 121.254.175.70/32/eth0 startIPaddr2(IPaddr2_121.254.175.70)[3356]: 2018/08/08_16:33:17 INFO: Adding inet address 121.254.175.70/32 with broadcast address 121.254.175.79 to device eth0IPaddr2(IPaddr2_121.254.175.70)[3356]: 2018/08/08_16:33:17 INFO: Bringing device eth0 up Active 다운시 자동으로 Standby 가 역할 전담하는 것을 확인했다. 다시 Active 활성화시의 상태를 확인한다. Active log Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10859]: info: Pacemaker support: falseAug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10859]: WARN: Logging daemon is disabled –enabling logging daemon is recommendedAug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10859]: info: **************************Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10859]: info: Configuration validated. Starting heartbeat 3.0.4Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: heartbeat: version 3.0.4Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: Heartbeat generation: 1533705889Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: write socket priority set to IPTOS_LOWDELAY on eth0Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: bound send socket to device: eth0Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: set SO_REUSEPORT(w)Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: bound receive socket to device: eth0Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: set SO_REUSEPORT(w)Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: started on port 694 interface eth0 to 121.254.175.78Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ping heartbeat started.Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: G_main_add_TriggerHandler: Added signal manual handlerAug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: G_main_add_TriggerHandler: Added signal manual handlerAug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: G_main_add_SignalHandler: Added signal handler for signal 17Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: Local status now set to: ‘up’Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: Link 121.254.175.65:121.254.175.65 up.Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: Status update for node 121.254.175.65: status pingAug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10860]: info: Link lvsstandby.gunnm.xyz:eth0 up.Aug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10860]: info: Status update for node lvsstandby.gunnm.xyz: status activeharc(default)[10869]: 2018/08/08_16:37:56 info: Running /etc/ha.d//rc.d/status statusAug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10860]: info: Comm_now_up(): updating status to activeAug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10860]: info: Local status now set to: ‘active’Aug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10860]: info: Starting child client “/usr/lib64/heartbeat/ipfail” (496,495)Aug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10886]: info: Starting “/usr/lib64/heartbeat/ipfail” as uid 496 gid 495 (pid 10886)Aug 08 16:37:57 lvsactive.gunnm.xyz heartbeat: [10860]: info: remote resource transition completed.Aug 08 16:37:57 lvsactive.gunnm.xyz heartbeat: [10860]: info: remote resource transition completed.Aug 08 16:37:57 lvsactive.gunnm.xyz heartbeat: [10860]: info: Local Resource acquisition completed. (none)Aug 08 16:37:57 lvsactive.gunnm.xyz heartbeat: [10860]: info: lvsstandby.gunnm.xyz wants to go standby [foreign]Aug 08 16:37:58 lvsactive.gunnm.xyz heartbeat: [10860]: info: standby: acquire [foreign] resources from lvsstandby.gunnm.xyzAug 08 16:37:58 lvsactive.gunnm.xyz heartbeat: [10890]: info: acquire local HA resources (standby).ResourceManager(default)[10903]: 2018/08/08_16:37:58 info: Acquiring resource group: lvsactive.gunnm.xyz ldirectord::ldirectord.cf IPaddr2::121.254.175.70/32/eth0ResourceManager(default)[10903]: 2018/08/08_16:37:58 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start/usr/lib/ocf/resource.d//heartbeat/IPaddr2(IPaddr2_121.254.175.70)[10967]: 2018/08/08_16:37:59 INFO: Resource is stoppedResourceManager(default)[10903]: 2018/08/08_16:37:59 info: Running /etc/ha.d/resource.d/IPaddr2 121.254.175.70/32/eth0 startIPaddr2(IPaddr2_121.254.175.70)[11098]: 2018/08/08_16:37:59 INFO: Adding inet address 121.254.175.70/32 with broadcast address 121.254.175.79 to device eth0IPaddr2(IPaddr2_121.254.175.70)[11098]: 2018/08/08_16:37:59 INFO: Bringing device eth0 upIPaddr2(IPaddr2_121.254.175.70)[11098]: 2018/08/08_16:37:59 INFO: /usr/libexec/heartbeat/send_arp -i 200 -r 5 -p /var/run/resource-agents/send_arp-121.254.175.70 eth0 121.254.175.70 auto not_used not_used/usr/lib/ocf/resource.d//heartbeat/IPaddr2(IPaddr2_121.254.175.70)[11072]: 2018/08/08_16:37:59 INFO: SuccessAug 08 16:37:59 lvsactive.gunnm.xyz heartbeat: [10890]: info: local HA resource acquisition completed (standby).Aug 08 16:37:59 lvsactive.gunnm.xyz heartbeat: [10860]: info: Standby resource acquisition done [foreign].Aug 08 16:37:59 lvsactive.gunnm.xyz heartbeat: [10860]: info: Initial resource acquisition complete (auto_failback)Aug 08 16:38:00 lvsactive.gunnm.xyz heartbeat: [10860]: info: remote resource transition completed.Aug 08 16:38:06 lvsactive.gunnm.xyz ipfail: [10886]: info: Telling other node that we have more visible ping nodes.Aug 08 16:38:12 lvsactive.gunnm.xyz heartbeat: [10860]: info: lvsstandby.gunnm.xyz wants to go standby [all]Aug 08 16:38:13 lvsactive.gunnm.xyz heartbeat: [10860]: info: standby: acquire [all] resources from lvsstandby.gunnm.xyzAug 08 16:38:13 lvsactive.gunnm.xyz heartbeat: [11183]: info: acquire all HA resources (standby).ResourceManager(default)[11196]: 2018/08/08_16:38:13 info: Acquiring resource group: lvsactive.gunnm.xyz ldirectord::ldirectord.cf IPaddr2::121.254.175.70/32/eth0ResourceManager(default)[11196]: 2018/08/08_16:38:14 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start/usr/lib/ocf/resource.d//heartbeat/IPaddr2(IPaddr2_121.254.175.70)[11254]: 2018/08/08_16:38:14 INFO: Running OKAug 08 16:38:14 lvsactive.gunnm.xyz heartbeat: [11183]: info: all HA resource acquisition completed (standby).Aug 08 16:38:14 lvsactive.gunnm.xyz heartbeat: [10860]: info: Standby resource acquisition done [all].Aug 08 16:38:15 lvsactive.gunnm.xyz heartbeat: [10860]: info: remote resource transition completed. Standby log Aug 08 16:37:55 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Heartbeat restart on node lvsactive.gunnm.xyzAug 08 16:37:55 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Link lvsactive.gunnm.xyz:eth0 up.Aug 08 16:37:55 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Status update for node lvsactive.gunnm.xyz: status initAug 08 16:37:55 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Status update for node lvsactive.gunnm.xyz: status upharc(default)[3761]: 2018/08/08_16:37:55 info: Running /etc/ha.d//rc.d/status statusharc(default)[3777]: 2018/08/08_16:37:55 info: Running /etc/ha.d//rc.d/status statusAug 08 16:37:57 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Status update for node lvsactive.gunnm.xyz: status activeharc(default)[3793]: 2018/08/08_16:37:57 info: Running /etc/ha.d//rc.d/status statusAug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [1415]: info: remote resource transition completed.Aug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [1415]: info: lvsstandby.gunnm.xyz wants to go standby [foreign]Aug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [1415]: info: standby: lvsactive.gunnm.xyz can take our foreign resourcesAug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [3809]: info: give up foreign HA resources (standby).Aug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [3809]: info: foreign HA resource release completed (standby).Aug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Local standby process completed [foreign].Aug 08 16:38:00 lvsstandby.gunnm.xyz heartbeat: [1415]: WARN: 1 lost packet(s) for [lvsactive.gunnm.xyz] [12:14]Aug 08 16:38:00 lvsstandby.gunnm.xyz heartbeat: [1415]: info: remote resource transition completed.Aug 08 16:38:00 lvsstandby.gunnm.xyz heartbeat: [1415]: info: No pkts missing from lvsactive.gunnm.xyz!Aug 08 16:38:00 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Other node completed standby takeover of foreign resources.Aug 08 16:38:12 lvsstandby.gunnm.xyz heartbeat: [1415]: info: lvsstandby.gunnm.xyz wants to go standby [all]Aug 08 16:38:13 lvsstandby.gunnm.xyz heartbeat: [1415]: info: standby: lvsactive.gunnm.xyz can take our all resourcesAug 08 16:38:13 lvsstandby.gunnm.xyz heartbeat: [3849]: info: give up all HA resources (standby).ResourceManager(default)[3862]: 2018/08/08_16:38:13 info: Releasing resource group: lvsstandby.gunnm.xyz ldirectord::ldirectord.cf IPaddr2::121.254.175.70/32/eth0ResourceManager(default)[3862]: 2018/08/08_16:38:13 info: Running /etc/ha.d/resource.d/IPaddr2 121.254.175.70/32/eth0 stopIPaddr2(IPaddr2_121.254.175.70)[3923]: 2018/08/08_16:38:13 INFO: IP status = ok, IP_CIP=/usr/lib/ocf/resource.d//heartbeat/IPaddr2(IPaddr2_121.254.175.70)[3897]: 2018/08/08_16:38:13 INFO: SuccessResourceManager(default)[3862]: 2018/08/08_16:38:13 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stopAug 08 16:38:13 lvsstandby.gunnm.xyz heartbeat: [3849]: info: all HA resource release completed (standby).Aug 08 16:38:13 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Local standby process completed [all].Aug 08 16:38:15 lvsstandby.gunnm.xyz heartbeat: [1415]: WARN: 1 lost packet(s) for [lvsactive.gunnm.xyz] [26:28]Aug 08 16:38:15 lvsstandby.gunnm.xyz heartbeat: [1415]: info: remote resource transition completed.Aug 08 16:38:15 lvsstandby.gunnm.xyz heartbeat: [1415]: info: No pkts missing from lvsactive.gunnm.xyz!Aug 08 16:38:15 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Other node completed standby takeover of all resources. Active 와 Standby 가 알아서 역할을 잘 설정한다. |
5. 실시간 모니터링 및 ldirectord 설정 변경시 적용 파일 생성
| # vi /root/lvs.sh #!/bin/shwatch -n1 -t “echo ; ip addr ; echo ; ipvsadm -Ln ; ipvsadm -Ln –rate ; ipvsadm -Lnc” # vi /root/ldirect.sh (/etc/ha.d/ldirectord.cf 파일 내용 수정 후, reload 하는 파일) #!/bin/sh/etc/ha.d/resource.d/ldirectord /etc/ha.d/ldirectord.cf reload Active 에서# sh lvs.sh 입력시 접속 세션 확인 가능 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 5e:df:dd:9f:3b:e0 brd ff:ff:ff:ff:ff:ff inet 121.254.175.77/28 brd 121.254.175.79 scope global eth0 inet 121.254.175.70/32 brd 121.254.175.79 scope global eth0 inet6 fe80::5cdf:ddff:fe9f:3be0/64 scope link valid_lft forever preferred_lft forever IP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 121.254.175.70:80 wlc persistent 3600 -> 121.254.175.74:80 Route 10 0 1 -> 121.254.175.75:80 Route 10 5 2IP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port CPS InPPS OutPPS InBPS OutBPS -> RemoteAddress:PortTCP 121.254.175.70:80 0 1 0 85 0 -> 121.254.175.74:80 0 0 0 38 0 -> 121.254.175.75:80 0 0 0 47 0IPVS connection entriespro expire state source virtual destinationTCP 48:09 NONE 92.31.1.6:0 121.254.175.70:80 121.254.175.75:80TCP 14:37 ESTABLISHED 222.231.27.246:6060 121.254.175.70:80 121.254.175.75:80TCP 05:11 ESTABLISHED 222.231.27.246:6005 121.254.175.70:80 121.254.175.75:80TCP 14:36 ESTABLISHED 222.231.27.246:6064 121.254.175.70:80 121.254.175.75:80TCP 01:08 FIN_WAIT 222.231.27.246:6061 121.254.175.70:80 121.254.175.75:80TCP 01:36 FIN_WAIT 222.231.27.246:6062 121.254.175.70:80 121.254.175.75:80TCP 05:11 ESTABLISHED 222.231.27.246:6020 121.254.175.70:80 121.254.175.75:80TCP 00:00 CLOSE 118.69.170.35:60980 121.254.175.70:80 121.254.175.74:80TCP 14:47 ESTABLISHED 121.254.171.226:52493 121.254.175.70:80 121.254.175.75:80TCP 59:49 NONE 118.69.170.35:0 121.254.175.70:80 121.254.175.74:80TCP 59:17 NONE 121.254.171.226:0 121.254.175.70:80 121.254.175.75:80TCP 59:36 NONE 222.231.27.246:0 121.254.175.70:80 121.254.175.75:80 Standby 에서는 아무것도 연결이 되어 있지 않다. 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 2a:8b:f9:ad:f0:30 brd ff:ff:ff:ff:ff:ff inet 121.254.175.78/28 brd 121.254.175.79 scope global eth0 inet 121.254.175.70/32 brd 121.254.175.79 scope global eth0 inet6 fe80::288b:f9ff:fead:f030/64 scope link valid_lft forever preferred_lft forever IP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 121.254.175.70:80 wlc persistent 3600 -> 121.254.175.75:80 Route 10 3 0IP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port CPS InPPS OutPPS InBPS OutBPS -> RemoteAddress:PortTCP 121.254.175.70:80 0 0 0 0 0 -> 121.254.175.75:80 0 0 0 0 0IPVS connection entriespro expire state source virtual destinationTCP 11:27 ESTABLISHED 222.231.27.246:5992 121.254.175.70:80 121.254.175.75:80TCP 56:27 NONE 222.231.27.246:0 121.254.175.70:80 121.254.175.75:80TCP 11:27 ESTABLISHED 222.231.27.246:5991 121.254.175.70:80 121.254.175.75:80TCP 11:27 ESTABLISHED 222.231.27.246:5993 121.254.175.70:80 121.254.175.75:80 |
이번 테스트는 동일 아이피 대역의 서버들끼로 진행을 했으니깐 간단하게 설정이 되었으나, 대역이 다른 서버들간의 구성시에는 라우트 추가가 진행되어야 한다.
출처: https://gunnm.tistory.com/category/리눅스 공부 [독까의 이야기]
댓글 없음:
참고: 블로그의 회원만 댓글을 작성할 수 있습니다.