消息隊(duì)列是非?；A(chǔ)的關(guān)鍵服務(wù)，為保證公司隊(duì)列服務(wù)的高可用及負(fù)載均衡，現(xiàn)通過如下方式實(shí)現(xiàn): RabbitMQ Cluster + Queue HA + Haproxy + Keepalived
3臺rabbitMQ服務(wù)器構(gòu)建broker集群，允許2臺服務(wù)器故障而服務(wù)不受影響，
在此基礎(chǔ)上，通過queue mirror實(shí)現(xiàn)隊(duì)列的高可用，在本例中鏡像到所有服務(wù)器，即1個(gè)master，2個(gè)slave；
為保證客戶端訪問入口地址的唯一性，通過haproxy做4層代理來提供mq服務(wù)，通過簡單的輪詢方式來進(jìn)行負(fù)載均衡，設(shè)置健康檢查來屏蔽故障節(jié)點(diǎn)對客戶端的影響，
并通過2臺haproxy做keepalived實(shí)現(xiàn)客戶端訪問入口的高可用。
rabbitmq隊(duì)列基本概念參考：
http://baike.baidu.com/link?url=ySoVSgecyl7dcLNqyjvwXVW-nNTSA7tIHmhwTHx37hL_H4wnYa70VCqmOZ59AaSEz2DYyfUiSMnQV2tHKD7OQK
參考官方文檔：
http://www.rabbitmq.com/clustering.html
http://www.rabbitmq.com/ha.html
http://www.rabbitmq.com/man/rabbitmqctl.1.man.html
http://www.rabbitmq.com/production-checklist.html
一、基礎(chǔ)知識：
rabbitmq集群：RabbitMQ broker 集群是多個(gè)erlang節(jié)點(diǎn)的邏輯組，每個(gè)節(jié)點(diǎn)運(yùn)行rabbitmq應(yīng)用，他們之間共享用戶、虛擬主機(jī)、隊(duì)列、exchange、綁定和運(yùn)行時(shí)參數(shù)。
集群之間復(fù)制什么信息：除了message queue（存在一個(gè)節(jié)點(diǎn)，從其他節(jié)點(diǎn)都可見、訪問該隊(duì)列，要實(shí)現(xiàn)queue的復(fù)制就需要做queue的HA）之外，任何一個(gè)rabbitmq broker上的所有操作的data和state都會在所有的節(jié)點(diǎn)之間進(jìn)行復(fù)制。
集群運(yùn)行的前提：
1、集群所有節(jié)點(diǎn)必須運(yùn)行相同的erlang及rabbitmq版本
2、hostname解析，節(jié)點(diǎn)之間通過域名相互通信，本文為3個(gè)node的集群，采用配置hosts的形式。
端口及用途
5672 客戶端連接用途
15672 web管理接口
25672 集群通信用途
集群的搭建方式：
1、通過rabbitmqctl手工配置 （本文采用此方式）
2、通過配置文件聲明
3、通過rabbitmq-autocluster插件聲明
4、通過rabbitmq-clusterer插件聲明
集群故障處理機(jī)制：
1、rabbitmq broker集群允許個(gè)體節(jié)點(diǎn)down機(jī)，
2、對應(yīng)集群的的網(wǎng)絡(luò)分區(qū)問題（ network partitions）
集群推薦用于LAN環(huán)境，不適用WAN環(huán)境；
要通過WAN連接broker，Shovel or Federation插件是最佳的解決方案。
Shovel or Federation不同于集群。
RabbitMQ clustering has several modes of dealing with network partitions, primarily consistency oriented. Clustering is meant to be used across LAN. It is not recommended to run clusters that span WAN. The Shovel or Federation plugins are better solutions for connecting brokers across a WAN. Note that Shovel and Federation are not equivalent to clustering.
節(jié)點(diǎn)運(yùn)行模式：
為保證數(shù)據(jù)持久性，目前所有node節(jié)點(diǎn)跑在disk模式，如果今后壓力大，需要提高性能，考慮采用ram模式
集群節(jié)點(diǎn)之間是如何相互認(rèn)證的：
通過Erlang Cookie，相當(dāng)于共享秘鑰的概念，長度任意，只要所有節(jié)點(diǎn)都一致即可。
rabbitmq server在啟動的時(shí)候，erlang VM會自動創(chuàng)建一個(gè)隨機(jī)的cookie文件。
cookie文件的位置： /var/lib/rabbitmq/.erlang.cookie 或者/root/.erlang.cookie
我們的放在：/root/.erlang.cookie
為保證cookie的完全一致，采用從一個(gè)節(jié)點(diǎn)copy的方式。
二、RabbitMQ集群部署過程
首先安裝單機(jī)版rabbitMQ，參考同事寫的文檔：部署erlang環(huán)境和rabbitmq的文檔
下面開始集群配置過程：
1、設(shè)置hosts解析,所有節(jié)點(diǎn)配置相同
[root@xx_rabbitMQ135 ~]# tail -n4 /etc/hosts
###rabbitmq 集群通信用途，所有節(jié)點(diǎn)配置一致 laijingli 20160220
192.168.100.135 xx_rabbitMQ135
192.168.100.136 xx_rabbitMQ136
192.168.100.137 xx_rabbitMQ137
2、設(shè)置節(jié)點(diǎn)間認(rèn)證的cookie
[root@xx_rabbitMQ135 ~]# scp /root/.erlang.cookie 192.168.100.136:~
[root@xx_rabbitMQ135 ~]# scp /root/.erlang.cookie 192.168.100.137:~
3、分別啟動獨(dú)立的單機(jī)版rabbitmq broker節(jié)點(diǎn)：
[root@xx_rabbitMQ135 ~]# rabbitmq-server -detached
[root@xx_rabbitMQ136 ~]# rabbitmq-server -detached
[root@xx_rabbitMQ137 ~]# rabbitmq-server -detached
這樣就在每個(gè)節(jié)點(diǎn)上創(chuàng)建了獨(dú)立的RabbitMQ brokers，
查看broker的狀態(tài)：
[root@xx_rabbitMQ135 ~]# rabbitmqctl status
Status of node rabbit@xx_rabbitMQ135 ...
[{pid,116968},
{running_applications,
[{rabbitmq_shovel_management,"Shovel Status","3.6.0"},
{rabbitmq_management,"RabbitMQ Management Console","3.6.0"},
查看broker的集群狀態(tài)：
[root@xx_rabbitMQ135 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@xx_rabbitMQ135 ...
[{nodes,[{disc,[rabbit@xx_rabbitMQ135]}]},
{running_nodes,[rabbit@xx_rabbitMQ135]},
{cluster_name,<<"rabbit@xx_rabbitMQ135">>},
{partitions,[]}]
4、創(chuàng)建broker集群：
為了把集群中的3歌節(jié)點(diǎn)聯(lián)系起來，我們把136和137分別加入到135的集群
先在136上stop rabbitmq，然后加到135的集群（join cluster會隱式的重置該節(jié)點(diǎn)，并刪除該節(jié)點(diǎn)上所有的資源和數(shù)據(jù)），然后查看集群狀態(tài)里有了2個(gè)node。
[root@xx_rabbitMQ136 ~]# rabbitmqctl stop_app
Stopping node rabbit@xx_rabbitMQ136 ...
[root@xx_rabbitMQ136 ~]# rabbitmqctl join_cluster rabbit@xx_rabbitMQ135
Clustering node rabbit@xx_rabbitMQ136 with rabbit@xx_rabbitMQ135 ...
[root@xx_rabbitMQ136 ~]# rabbitmqctl start_app
[root@xx_rabbitMQ136 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@xx_rabbitMQ136 ...
[{nodes,[{disc,[rabbit@xx_rabbitMQ135,rabbit@xx_rabbitMQ136]}]}]
137同理，加入集群時(shí)選擇135或者136哪個(gè)node都沒有影響。
[root@xx_rabbitMQ135 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@xx_rabbitMQ135 ...
[{nodes,[{disc,[rabbit@xx_rabbitMQ135,rabbit@xx_rabbitMQ136,
rabbit@xx_rabbitMQ137]}]},
{running_nodes,[rabbit@xx_rabbitMQ136,rabbit@xx_rabbitMQ137,
rabbit@xx_rabbitMQ135]},
{cluster_name,<<"rabbit@xx_rabbitMQ135">>},
{partitions,[]}]
修改集群的名字為xx_rabbitMQ_cluster(默認(rèn)是第一個(gè)node的名字)：
[root@xx_rabbitMQ135 ~]# rabbitmqctl set_cluster_name xx_rabbitMQ_cluster
[root@xx_rabbitMQ135 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@xx_rabbitMQ135 ...
[{nodes,[{disc,[rabbit@xx_rabbitMQ135,rabbit@xx_rabbitMQ136,
rabbit@xx_rabbitMQ137]}]},
{running_nodes,[rabbit@xx_rabbitMQ135,rabbit@xx_rabbitMQ136,
rabbit@xx_rabbitMQ137]},
{cluster_name,<<"xx_rabbitMQ_cluster">>},
{partitions,[]}]
5、重啟集群：
通過rabbitmqctl stop、rabbitmq-server -detached來重啟集群，觀察集群的運(yùn)行狀態(tài)變化
重要信息：
(1)、當(dāng)整個(gè)集群down掉時(shí)，最后一個(gè)down機(jī)的節(jié)點(diǎn)必須第一個(gè)啟動到在線狀態(tài)，如果不是這樣，節(jié)點(diǎn)會等待30s等最后的磁盤節(jié)點(diǎn)恢復(fù)狀態(tài)，然后失敗。
如果最后下線的節(jié)點(diǎn)不能上線，可以通過forget_cluster_node 指令來踢出集群。
When the entire cluster is brought down, the last node to go down must be the first node to be brought online. If this doesn't happen, the nodes will wait 30 seconds for the last disc node to come back online, and fail afterwards.
If the last node to go offline cannot be brought back up, it can be removed from the cluster using the forget_cluster_node command - consult the rabbitmqctl manpage for more information.
(2)、如果所有的節(jié)點(diǎn)不受控制的同時(shí)宕機(jī)，比如掉電，會進(jìn)入所有的節(jié)點(diǎn)都會認(rèn)為其他節(jié)點(diǎn)比自己宕機(jī)的要晚，即自己先宕機(jī)，這種情況下可以使用force_boot指令來啟動一個(gè)節(jié)點(diǎn)。
If all cluster nodes stop in a simultaneous and uncontrolled manner (for example with a power cut) you can be left with a situation in which all nodes think that some other node stopped after them. In this case you can use the force_boot command on one node to make it bootable again - consult the rabbitmqctl manpage for more information.
6、打破集群：
當(dāng)一個(gè)節(jié)點(diǎn)不屬于這個(gè)集群的時(shí)候，我們需要顯式的踢出，可以通過本地或者遠(yuǎn)程的方式
[root@xx_rabbitMQ137 ~]# rabbitmqctl stop_app
Stopping node rabbit@xx_rabbitMQ137 ...
[root@xx_rabbitMQ137 ~]# rabbitmqctl reset
Resetting node rabbit@xx_rabbitMQ137 ...
[root@xx_rabbitMQ137 ~]# rabbitmqctl start_app
Starting node rabbit@xx_rabbitMQ137 ...
[root@xx_rabbitMQ137 ~]# rabbitmqctl cluster_status
Cluster status of node rabbit@xx_rabbitMQ137 ...
[{nodes,[{disc,[rabbit@xx_rabbitMQ137]}]},
{running_nodes,[rabbit@xx_rabbitMQ137]},
{cluster_name,<<"rabbit@xx_rabbitMQ137">>},
{partitions,[]}]
7、客戶端連接集群測試
通過web管理頁面進(jìn)行創(chuàng)建隊(duì)列、發(fā)布消息、創(chuàng)建用戶、創(chuàng)建policy等
http://192.168.100.137:15672/
或者通過rabbitmqadmin命令行來測試
[root@xx_rabbitMQ136 ~]# wget http://192.168.100.136:15672/cli/rabbitmqadmin
[root@xx_rabbitMQ136 ~]# chmod +x rabbitmqadmin
[root@xx_rabbitMQ136 ~]# mv rabbitmqadmin /usr/local/rabbitmq_server-3.6.0/sbin/
Declare an exchange
$ rabbitmqadmin declare exchange name=my-new-exchange type=fanout
exchange declared
Declare a queue, with optional parameters
$ rabbitmqadmin declare queue name=my-new-queue durable=false
queue declared
Publish a message
$ rabbitmqadmin publish exchange=my-new-exchange routing_key=test payload="hello, world"
Message published
And get it back
$ rabbitmqadmin get queue=test requeue=false
+-------------+----------+---------------+--------------+------------------+-------------+
| routing_key | exchange | message_count | payload | payload_encoding | redelivered |
+-------------+----------+---------------+--------------+------------------+-------------+
| test | | 0 | hello, world | string | False |
+-------------+----------+---------------+--------------+------------------+-------------+
測試后發(fā)現(xiàn)問題問題：
[root@xx_rabbitMQ135 ~]# rabbitmqctl stop_app
[root@xx_rabbitMQ135 ~]# rabbitmqctl stop
在stop_app或者stop掉broker之后在135節(jié)點(diǎn)的上隊(duì)列已經(jīng)不可用了，重啟135的app或broker之后，雖然集群工作正常，但135上隊(duì)列中消息會被清空（queue還是存在的）
對于生產(chǎn)環(huán)境而已，這肯定是不可接受的，如果不能保證隊(duì)列的高可用，那么做集群的意義也不太大了，還好rabbitmq支持Highly Available Queues，下面介紹queue的HA。
三、Queue HA配置
默認(rèn)情況下，集群中的隊(duì)列存在于集群中單個(gè)節(jié)點(diǎn)上，這要看創(chuàng)建隊(duì)列時(shí)聲明在那個(gè)節(jié)點(diǎn)上創(chuàng)建，而exchange和binding則默認(rèn)存在于集群中所有節(jié)點(diǎn)。
隊(duì)列可以通過鏡像來提高可用性，HA依賴rabbitmq cluster，所以隊(duì)列鏡像也不適合WAN部署，每個(gè)被鏡像的隊(duì)列包含一個(gè)master和一個(gè)或者多個(gè)slave，當(dāng)master因任何原因故障時(shí)，最老的slave被提升為新的master。
發(fā)布到隊(duì)列的消息被復(fù)制到所有的slave上，消費(fèi)者無論連接那個(gè)node，都會連接到master；如果master確認(rèn)要刪除消息，那么所有slave就會刪除隊(duì)列中消息。
隊(duì)列鏡像可以提供queue的高可用性，但不能分擔(dān)負(fù)載，因?yàn)樗袇⒓拥墓?jié)點(diǎn)都做所有的工作。
1、配置隊(duì)列鏡像
通過policy來配置鏡像，策略可在任何時(shí)候創(chuàng)建，比如先創(chuàng)建一個(gè)非鏡像的隊(duì)列，然后在鏡像，反之亦然。
鏡像隊(duì)列和非鏡像隊(duì)列的區(qū)別是非鏡像隊(duì)列沒有slaves，運(yùn)行速度也比鏡像隊(duì)列快。
設(shè)置策略然后設(shè)置ha-mode，3中模式：all、exactly、nodes
每個(gè)隊(duì)列都有一個(gè)home node，叫做queue master node
(1)、設(shè)置policy，以ha.開頭的隊(duì)列將會被鏡像到集群其他所有節(jié)點(diǎn)，一個(gè)節(jié)點(diǎn)掛掉然后重啟后需要手動同步隊(duì)列消息
rabbitmqctl set_policy ha-all-queue "^ha\." '{"ha-mode":"all"}'
(2)、設(shè)置policy，以ha.開頭的隊(duì)列將會被鏡像到集群其他所有節(jié)點(diǎn),一個(gè)節(jié)點(diǎn)掛掉然后重啟后會自動同步隊(duì)列消息（我們生產(chǎn)環(huán)境采用這個(gè)方式）
rabbitmqctl set_policy ha-all-queue "^ha\." '{"ha-mode":"all","ha-sync-mode":"automatic"}'
2、問題：
配置鏡像隊(duì)列后，其中1臺節(jié)點(diǎn)失敗，隊(duì)列內(nèi)容是不會丟失，如果整個(gè)集群重啟，隊(duì)列中的消息內(nèi)容仍然丟失，如何實(shí)現(xiàn)隊(duì)列消息內(nèi)容持久化那？
我的node也是跑在disk模式，創(chuàng)建見消息的時(shí)候也聲明了持久化，為什么還是不行那？
因?yàn)閯?chuàng)建消息的時(shí)候需要指定消息是否持久化，如果啟用了消息的持久化的話，重啟集群消息也不會丟失了，前提是創(chuàng)建的隊(duì)列也應(yīng)該是創(chuàng)建的持久化隊(duì)列。
四、客戶端連接rabbitMQ集群服務(wù)的方式：
1、客戶端可以連接集群中的任意一個(gè)節(jié)點(diǎn)，如果一個(gè)節(jié)點(diǎn)故障，客戶端自行重新連接到其他的可用節(jié)點(diǎn)；（不推薦，對客戶端不透明）
2、通過動態(tài)DNS，較短的ttl
3、通過HA+4層負(fù)載均衡器（比如haproxy+keepalived）
五、Haproxy+keepalived的部署
消息隊(duì)列作為公司的關(guān)鍵基礎(chǔ)服務(wù)，為給客戶端提供穩(wěn)定、透明的rabbitmq服務(wù)，現(xiàn)通過Haproxy+keepalived構(gòu)建高可用的rabbitmq統(tǒng)一入口，及基本的負(fù)載均衡服務(wù)。
為簡化安裝配置，現(xiàn)采用yum的方式安裝haproxy和keepalived，可參考 基于keepalived+nginx部署強(qiáng)健的高可用7層負(fù)載均衡方案
1、安裝
yum install haproxy keepalived -y
2、設(shè)置關(guān)鍵服務(wù)開機(jī)自啟動
[root@xxhaproxy101 keepalived]# chkconfig --list|grep haproxy
haproxy 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[root@xxhaproxy101 keepalived]# chkconfig haproxy on
[root@xxhaproxy101 keepalived]# chkconfig --list|grep haproxy
haproxy 0:off 1:off 2:on 3:on 4:on 5:on 6:off
3、配置將haproxy的log記錄到 /var/log/haproxy.log
[root@xxhaproxy101 haproxy]# more /etc/rsyslog.d/haproxy.conf
$ModLoad imudp
$UDPServerRun 514
local0.* /var/log/haproxy.log
[root@xxhaproxy101 haproxy]# /etc/init.d/rsyslog restart
4、haproxy的配置，2臺機(jī)器上的配置完全相同
[root@xxhaproxy101 keepalived]# more /etc/haproxy/haproxy.cfg
#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local2 notice
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode tcp
option tcplog
option dontlognull
option http-server-close
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
###haproxy statistics monitor by laijingli 20160222
listen statics 0.0.0.0:8888
mode http
log 127.0.0.1 local0 debug
transparent
stats refresh 60s
stats uri / haproxy-stats
stats realm Haproxy \ statistic
stats auth laijingli:xxxxx
#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend xx_rabbitMQ_cluster_frontend
mode tcp
option tcpka
log 127.0.0.1 local0 debug
bind 0.0.0.0:5672
use_backend xx_rabbitMQ_cluster_backend
frontend xx_rabbitMQ_cluster_management_frontend
mode tcp
option tcpka
log 127.0.0.1 local0 debug
bind 0.0.0.0:15672
use_backend xx_rabbitMQ_cluster_management_backend
#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend xx_rabbitMQ_cluster_backend
balance roundrobin
server xx_rabbitMQ135 192.168.100.135:5672 check inter 3s rise 1 fall 2
server xx_rabbitMQ136 192.168.100.136:5672 check inter 3s rise 1 fall 2
server xx_rabbitMQ137 192.168.100.137:5672 check inter 3s rise 1 fall 2
backend xx_rabbitMQ_cluster_management_backend
balance roundrobin
server xx_rabbitMQ135 192.168.100.135:15672 check inter 3s rise 1 fall 2
server xx_rabbitMQ136 192.168.100.136:15672 check inter 3s rise 1 fall 2
server xx_rabbitMQ137 192.168.100.137:15672 check inter 3s rise 1 fall 2
[root@xxhaproxy101 keepalived]#
5、貼出keepalived的配置，因HA服務(wù)器上同時(shí)運(yùn)行的http及rabbitmq的服務(wù)，故一并貼出來了,特別注意2臺服務(wù)器上的keepalived配置不一樣
[root@xxhaproxy101 keepalived]# more /etc/keepalived/keepalived.conf
####Configuration File for keepalived
####xx公司線上內(nèi)部API網(wǎng)關(guān) keepalived HA配置
####xx公司線上rabbitMQ集群keepalived HA配置
#### laijingli 20151213
global_defs {
notification_email {
laijingli2006@gmail.com
362560701@qq.com
}
notification_email_from yn_alert@xxxx.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id xxhaproxy101 ## xxhaproxy101 on master , xxhaproxy102 on backup
}
###simple check with killall -0 which is less expensive than pidof to verify that nginx is running
vrrp_script chk_nginx {
script "killall -0 nginx"
interval 1
weight 2
fall 2
rise 1
}
vrrp_instance YN_API_GATEWAY {
state MASTER ## MASTER on master , BACKUP on backup
interface em1
virtual_router_id 101 ## YN_API_GATEWAY virtual_router_id
priority 200 ## 200 on master , 199 on backup
advert_int 1
###采用單播通信，避免同一個(gè)局域網(wǎng)中多個(gè)keepalived組之間的相互影響
unicast_src_ip 192.168.100.101 ##本機(jī)ip
unicast_peer {
192.168.100.102 ##對端ip
}
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
192.168.100.99 ## VIP
}
###如果只有一塊網(wǎng)卡的話監(jiān)控網(wǎng)絡(luò)接口就沒有必要了
#track_interface {
# em1
#}
track_script {
chk_nginx
}
###狀態(tài)切換是發(fā)送郵件通知，本機(jī)記錄log，后期會觸發(fā)短信通知
notify_master /usr/local/bin/keepalived_notify.sh notify_master
notify_backup /usr/local/bin/keepalived_notify.sh notify_backup
notify_fault /usr/local/bin/keepalived_notify.sh notify_fault
notify /usr/local/bin/keepalived_notify.sh notify
smtp_alert
}
###simple check with killall -0 which is less expensive than pidof to verify that haproxy is running
vrrp_script chk_haproxy {
script "killall -0 haproxy"
interval 1
weight 2
fall 2
rise 1
}
vrrp_instance xx_rabbitMQ_GATEWAY {
state BACKUP ## MASTER on master , BACKUP on backup
interface em1
virtual_router_id 111 ## xx_rabbitMQ_GATEWAY virtual_router_id
priority 199 ## 200 on master , 199 on backup
advert_int 1
###采用單播通信，避免同一個(gè)局域網(wǎng)中多個(gè)keepalived組之間的相互影響
unicast_src_ip 192.168.100.101 ##本機(jī)ip
unicast_peer {
192.168.100.102 ##對端ip
}
authentication {
auth_type PASS
auth_pass 123456
}
virtual_ipaddress {
192.168.100.100 ## VIP
}
###如果只有一塊網(wǎng)卡的話監(jiān)控網(wǎng)絡(luò)接口就沒有必要了
#track_interface {
# em1
#}
track_script {
chk_haproxy
}
###狀態(tài)切換是發(fā)送郵件通知，本機(jī)記錄log，后期會觸發(fā)短信通知
notify_master /usr/local/bin/keepalived_notify_for_haproxy.sh notify_master
notify_backup /usr/local/bin/keepalived_notify_for_haproxy.sh notify_backup
notify_fault /usr/local/bin/keepalived_notify_for_haproxy.sh notify_fault
notify /usr/local/bin/keepalived_notify_for_haproxy.sh notify
smtp_alert
}
[root@xxhaproxy102 keepalived]# more /etc/keepalived/keepalived.conf
####Configuration File for keepalived
####xx公司線上內(nèi)部API網(wǎng)關(guān) keepalived HA配置
####xx公司線上rabbitMQ集群keepalived HA配置
#### laijingli 20151213
global_defs {
notification_email {
laijingli2006@gmail.com
362560701@qq.com
}
notification_email_from yn_alert@xxxx.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id xxhaproxy102 ## xxhaproxy101 on master , xxhaproxy102 on backup
}
###simple check with killall -0 which is less expensive than pidof to verify that nginx is running
vrrp_script chk_nginx {
script "killall -0 nginx"
interval 1
weight 2
fall 2
rise 1
}
vrrp_instance YN_API_GATEWAY {
state BACKUP ## MASTER on master , BACKUP on backup
interface em1
virtual_router_id 101 ## YN_API_GATEWAY virtual_router_id
priority 199 ## 200 on master , 199 on backup
advert_int 1
###采用單播通信，避免同一個(gè)局域網(wǎng)中多個(gè)keepalived組之間的相互影響
unicast_src_ip 192.168.100.102 ##本機(jī)ip
unicast_peer {
192.168.100.101 ##對端ip
}
authentication {
auth_type PASS
auth_pass YN_API_HA_PASS
}
virtual_ipaddress {
192.168.100.99 ## VIP
}
###如果只有一塊網(wǎng)卡的話監(jiān)控網(wǎng)絡(luò)接口就沒有必要了
#track_interface {
# em1
#}
track_script {
chk_nginx
}
###狀態(tài)切換是發(fā)送郵件通知，本機(jī)記錄log，后期會觸發(fā)短信通知
notify_master /usr/local/bin/keepalived_notify.sh notify_master
notify_backup /usr/local/bin/keepalived_notify.sh notify_backup
notify_fault /usr/local/bin/keepalived_notify.sh notify_fault
notify /usr/local/bin/keepalived_notify.sh notify
smtp_alert
}
###simple check with killall -0 which is less expensive than pidof to verify that haproxy is running
vrrp_script chk_haproxy {
script "killall -0 haproxy"
interval 1
weight 2
fall 2
rise 1
}
vrrp_instance xx_rabbitMQ_GATEWAY {
state MASTER ## MASTER on master , BACKUP on backup
interface em1
virtual_router_id 111 ## xx_rabbitMQ_GATEWAY virtual_router_id
priority 200 ## 200 on master , 199 on backup
advert_int 1
###采用單播通信，避免同一個(gè)局域網(wǎng)中多個(gè)keepalived組之間的相互影響
unicast_src_ip 192.168.100.102 ##本機(jī)ip
unicast_peer {
192.168.100.101 ##對端ip
}
authentication {
auth_type PASS
auth_pass YN_MQ_HA_PASS
}
virtual_ipaddress {
192.168.100.100 ## VIP
}
###如果只有一塊網(wǎng)卡的話監(jiān)控網(wǎng)絡(luò)接口就沒有必要了
#track_interface {
# em1
#}
track_script {
chk_haproxy
}
###狀態(tài)切換是發(fā)送郵件通知，本機(jī)記錄log，后期會觸發(fā)短信通知
notify_master /usr/local/bin/keepalived_notify_for_haproxy.sh notify_master
notify_backup /usr/local/bin/keepalived_notify_for_haproxy.sh notify_backup
notify_fault /usr/local/bin/keepalived_notify_for_haproxy.sh notify_fault
notify /usr/local/bin/keepalived_notify_for_haproxy.sh notify
smtp_alert
}
配置中用到的通知腳本，2臺服務(wù)器上完全一樣：
[root@xxhaproxy101 keepalived]# more /usr/local/bin/keepalived_notify.sh
#!/bin/bash
###keepalived notify script for record ha state transtion to log files
###將將狀態(tài)轉(zhuǎn)換過程記錄到log，便于排錯(cuò)
logfile=/var/log/keepalived.notify.log
echo --------------- >> $logfile
echo `date` [`hostname`] keepalived HA role state transition: $1 $2 $3 $4 $5 $6 >> $logfile
###將狀態(tài)轉(zhuǎn)換記錄到nginx的文件，便于通過web查看ha狀態(tài)（一定注意不要開放到公網(wǎng)）
echo `date` `hostname` $1 $2 $3 $4 $5 $6 " <br>" > /usr/share/nginx/html/index_for_nginx.html
###將nginx api和rabbitmq的ha log記錄到同一個(gè)文件里
cat /usr/share/nginx/html/index_for* > /usr/share/nginx/html/index.html
[root@xxhaproxy101 keepalived]# more /usr/local/bin/keepalived_notify_for_haproxy.sh
#!/bin/bash
###keepalived notify script for record ha state transtion to log files
###將將狀態(tài)轉(zhuǎn)換過程記錄到log，便于排錯(cuò)
logfile=/var/log/keepalived.notify.log
echo --------------- >> $logfile
echo `date` [`hostname`] keepalived HA role state transition: $1 $2 $3 $4 $5 $6 >> $logfile
###將狀態(tài)轉(zhuǎn)換記錄到nginx的文件，便于通過web查看ha狀態(tài)（一定注意不要開放到公網(wǎng)）
echo `date` `hostname` $1 $2 $3 $4 $5 $6 " <br>" > /usr/share/nginx/html/index_for_haproxy.html
###將nginx api和rabbitmq的ha log記錄到同一個(gè)文件里
cat /usr/share/nginx/html/index_for* > /usr/share/nginx/html/index.html
[root@xxhaproxy101 keepalived]#
6、haproxy監(jiān)控頁面：
http://192.168.100.101:8888/
7、查看keepalived中高可用服務(wù)運(yùn)行在那臺服務(wù)器上
http://192.168.100.101
8、通過VIP訪問rabbitMQ服務(wù)
192.168.100.100:5672
六、更多問題：
參考 xx公司rabbitmq服務(wù)客戶端使用規(guī)范
1、使用vhost來隔離不同的應(yīng)用、不同的用戶、不同的業(yè)務(wù)組
2、消息持久化，exchange、queue、message等持久化需要在客戶端聲明指定
本站僅提供存儲服務(wù)，所有內(nèi)容均由用戶發(fā)布，如發(fā)現(xiàn)有害或侵權(quán)內(nèi)容，請
點(diǎn)擊舉報(bào)。
