MySQL復制能夠保證數(shù)據(jù)的冗余的同時可以做讀寫分離來分擔系統(tǒng)壓力,如果是主主復制還可以很好的避免主節(jié)點的單點故障。但是MySQL主主復制存在一些問題無法滿足我們的實際需要:未提供統(tǒng)一訪問入口來實現(xiàn)負載均衡,如果其中master宕掉的話需要手動切換到另外一個master,而不能自動進行切換。
這篇文章下面要介紹如何通過LVS+Keepalived的方式來是實現(xiàn)MySQL的高可用性,同時解決以上問題。
Keepalived是一個基于VRRP(虛擬路由冗余協(xié)議)可用來實現(xiàn)服務高可用性的軟件方案,避免出現(xiàn)單點故障。Keepalived一般用來實現(xiàn)輕量級高可用性,且不需要共享存儲,一般用于兩個節(jié)點之間,常見有LVS+Keepalived、Nginx+Keepalived組合。
LVS(Linux Virtual Server)是一個高可用性虛擬的服務器集群系統(tǒng)。本項目在1998年5月由章文嵩博士成立,是中國國內(nèi)最早出現(xiàn)的自由軟件項目之一。
LVS主要用于多服務器的負載均衡,作用于網(wǎng)絡層。LVS構(gòu)建的服務器集群系統(tǒng)中,前端的負載均衡層被稱為Director Server;后端提供服務的服務器組層被稱為Real Server。通過下圖可以大致了解LVS的基礎(chǔ)架構(gòu)。
LVS有三種工作模式,分別是DR(Direct Routing 直接路由)、TUN(Tunneling IP隧道)、NAT(Network Address Translation 網(wǎng)絡地址轉(zhuǎn)換)。其中TUN模式能夠支持更多的Real Server,但需要所有服務器支持IP隧道協(xié)議;DR也可以支持相當?shù)腞eal Server,但需要保證Director Server虛擬網(wǎng)卡與物理網(wǎng)卡在同一網(wǎng)段;NAT擴展性有限,無法支持更多的Real Server,因為所有的請求包和應答包都需要Director Server進行解析再生,影響效率。 同時,LVS負載均衡有10中調(diào)度算法,分別是rr、wrr、lc、wlc、lblc、lblcr、dh、sh、sed、nq
詳細的LVS說明請參見 傳送門
本文中將利用LVS實現(xiàn)MySQL的讀寫負載均衡,Keepalived避免節(jié)點出現(xiàn)單點故障。
LVS1:192.168.1.2
LVS2:192.168.1.11
MySQL Server1:192.168.1.5
MySQL Server2:192.168.1.6
VIP:192.168.1.100
OS: CentOS 6.4
Keepalive安裝
需要安裝以下軟件包
# yum install -y kernel-devel openssl openssl-devel
解壓keepalived到/usr/local/并進入目錄執(zhí)行配置編譯
# ./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-431.5.1.el6.x86_64/Keepalived configuration------------------------Keepalived version : 1.2.13Compiler : gccCompiler flags : -g -O2Extra Lib : -lssl -lcrypto -lcrypt Use IPVS Framework : YesIPVS sync daemon support : YesIPVS use libnl : Nofwmark socket support : YesUse VRRP Framework : YesUse VRRP VMAC : YesSNMP support : NoSHA1 support : NoUse Debug flags : No# make && make install
默認情況下keepalived啟動時會去/etc/keepalived目錄下找配置文件,將需要的配置文件拷貝到指定位置
# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/# chkconfig mysqld on# chkconfig keepalived on
LVS安裝
需要安裝以下軟件包
# yum install -y libnl* popt*
查看是否加載lvs模塊
# modprobe -l |grep ipvs
解壓安裝
# ln -s /usr/src/kernels/2.6.32-431.5.1.el6.x86_64/ /usr/src/linux# tar -zxvf ipvsadm-1.26.tar.gz# make && make install
LVS安裝完成,查看當前LVS集群
# ipvsadm -L -nIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags-> RemoteAddress:Port Forward Weight ActiveConn InActConn
搭建MySQL主主復制
這里不再贅述,請參考MySQL復制
配置Keepalived
下面是LVS1節(jié)點(Keepalived主節(jié)點)上的Keepalived配置,LVS2類似
# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalivedglobal_defs { router_id LVS1}vrrp_instance VI_1 { state MASTER #指定instance初始狀態(tài),實際根據(jù)優(yōu)先級決定.backup節(jié)點不一樣 interface eth0 #虛擬IP所在網(wǎng) virtual_router_id 51 #VRID,相同VRID為一個組,決定多播MAC地址 priority 100 #優(yōu)先級,另一臺改為90.backup節(jié)點不一樣 advert_int 1 #檢查間隔 authentication { auth_type PASS #認證方式,可以是pass或ha auth_pass 1111 #認證密碼 } virtual_ipaddress { 192.168.1.100 #VIP }}virtual_server 192.168.1.100 3306 { delay_loop 6 #服務輪詢的時間間隔 lb_algo wrr #加權(quán)輪詢調(diào)度,LVS調(diào)度算法 rr|wrr|lc|wlc|lblc|sh|sh lb_kind DR #LVS集群模式 NAT|DR|TUN,其中DR模式要求負載均衡器網(wǎng)卡必須有一塊與物理網(wǎng)卡在同一個網(wǎng)段 #nat_mask 255.255.255.0 persistence_timeout 50 #會話保持時間 protocol TCP #健康檢查協(xié)議 ## Real Server設置,3306就是MySQL連接端口 real_server 192.168.1.5 3306 { weight 3 ##權(quán)重 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 3306 } } real_server 192.168.1.6 3306 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 3306 } }}
配置LVS
編寫LVS啟動腳本/etc/init.d/realserver
#!/bin/shVIP=192.168.1.100. /etc/rc.d/init.d/functionscase "$1" in# 禁用本地的ARP請求、綁定本地回環(huán)地址start) /sbin/ifconfig lo down /sbin/ifconfig lo up echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce /sbin/sysctl -p >/dev/null 2>&1 /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 up #在回環(huán)地址上綁定VIP,設定掩碼,與Direct Server(自身)上的IP保持通信 /sbin/route add -host $VIP dev lo:0 echo "LVS-DR real server starts successfully.\n" ;;stop) /sbin/ifconfig lo:0 down /sbin/route del $VIP >/dev/null 2>&1 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announceecho "LVS-DR real server stopped.\n" ;;status) isLoOn=`/sbin/ifconfig lo:0 | grep "$VIP"` isRoOn=`/bin/netstat -rn | grep "$VIP"` if [ "$isLoON" == "" -a "$isRoOn" == "" ]; then echo "LVS-DR real server has run yet." else echo "LVS-DR real server is running." fi exit 3 ;;*) echo "Usage: $0 {start|stop|status}" exit 1esacexit 0
將lvs腳本加入開機自啟動
# chmod +x /etc/init.d/realserver# echo "/etc/init.d/realserver" >> /etc/rc.d/rc.local
分別啟動LVS和keepalived
# service realserver start# service keepalived start
注意此時網(wǎng)卡的變化,可以看到虛擬網(wǎng)卡已經(jīng)分配到了realserver上。
此時查看LVS集群狀態(tài),可以看到集群下有兩個Real Server,調(diào)度算法,權(quán)重等信息。ActiveConn代表當前Real Server的活躍連接數(shù)
# ipvsadm -lnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 192.168.1.100:3306 wrr persistent 50 -> 192.168.1.5:3306 Route 3 4 1 -> 192.168.1.6:3306 Route 3 0 2
此時LVS+Keepalived+MySQL主主復制已經(jīng)搭建完成。
功能性驗證
關(guān)閉MySQL Server2
# service mysqld stop
在LVS1查看/var/log/messages中關(guān)于keepalived日志,LVS1檢測到了MySQL Server2宕機,同時LVS集群自動剔除了故障節(jié)點
Sep 9 13:50:53 192.168.1.2 Keepalived_healthcheckers[18797]: TCP connection to [192.168.1.6]:3306 failed !!!Sep 9 13:50:53 192.168.1.2 Keepalived_healthcheckers[18797]: Removing service [192.168.1.6]:3306 from VS [192.168.1.100]:3306
從新啟動MySQL Server2后自動將故障節(jié)點自動加入LVS集群
Sep 9 13:51:41 192.168.1.2 Keepalived_healthcheckers[18797]: TCP connection to [192.168.1.6]:3306 success.Sep 9 13:51:41 192.168.1.2 Keepalived_healthcheckers[18797]: Adding service [192.168.1.6]:3306 to VS [192.168.1.100]:3306
關(guān)閉LVS1上的Keepalived(模擬宕機操作),查看LVS1上的日志,可以看到Keepalived移出了LVS1上的VIP
Sep 9 14:01:27 192.168.1.2 Keepalived[18796]: Stopping Keepalived v1.2.13 (09/09,2014)Sep 9 14:01:27 192.168.1.2 Keepalived_healthcheckers[18797]: Removing service [192.168.1.5]:3306 from VS [192.168.1.100]:3306Sep 9 14:01:27 192.168.1.2 Keepalived_healthcheckers[18797]: Removing service [192.168.1.6]:3306 from VS [192.168.1.100]:3306Sep 9 14:01:27 192.168.1.2 Keepalived_vrrp[18799]: VRRP_Instance(VI_1) sending 0 prioritySep 9 14:01:27 192.168.1.2 Keepalived_vrrp[18799]: VRRP_Instance(VI_1) removing protocol VIPs.
同時查看LVS2上日志,可以看到LVS2成為了Master,并接管了VIP
Sep 9 14:11:24 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Transition to MASTER STATESep 9 14:11:25 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Entering MASTER STATESep 9 14:11:25 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) setting protocol VIPs.Sep 9 14:11:25 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.100Sep 9 14:11:25 192.168.1.11 Keepalived_healthcheckers[7456]: Netlink reflector reports IP 192.168.1.100 addedSep 9 14:11:25 192.168.1.11 avahi-daemon[1407]: Registering new address record for 192.168.1.100 on eth0.IPv4.Sep 9 14:11:30 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.100
在LVS2上查看LVS集群狀態(tài),一切正常。
# ipvsadm -lnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 192.168.1.100:3306 wrr persistent 50 -> 192.168.1.5:3306 Route 3 2 0 -> 192.168.1.6:3306 Route 3 1 0