續上篇討論，我們有沒有比較節省的辦法實作一組 PXC Database
直到我們看到了 percona-xtradb-cluster-garbd-3.x
從官方建議我們得知，percona-xtradb-cluster-garbd-3.x 請搭配 percona-xtradb-cluster-5.7 服用
總共有三台實體機器
1. pxc-1: 192.168.1.1
2. pxc-2: 192.168.1.2
3. pxc-3: 192.168.1.3 (任何形式機器 docker, lxc, vm ; 只要在同網段能送 garbd 訊號出來即可)
4. wsrep.conf: wsrep_cluster_name=pxc-cluster
續上篇討論討論
我們先在 pxc-3 上安裝 percona-xtradb-cluster-garbd-3.x
root@pxc-3:~# aptitude install percona-xtradb-cluster-garbd-3.x
確認安裝完畢
root@pxc-3:~# dpkg -l | grep percona-xtradb-cluster-garbd-3.x ii percona-xtradb-cluster-garbd-3.x 3.21-3.jessie amd64 Garbd components of Percona XtraDB Cluster
root@pxc-3:~# whereis garbd garbd: /usr/bin/garbd
在 pxc-1 使用 bootstrap-pxc 將 mysql 啟動
root@pxc-1:~# /etc/init.d/mysql bootstrap-pxc
接著 pxc-3 上指定 garbd voting pxc-1 為 DONOR 並 SYNCED
root@pxc-3:~# /usr/bin/garbd -a gcomm://192.168.50.18:4567 -g pxc-cluster -l /tmp/garbd.txt
當 Member (garb) 加入時，甚至沒 SST 的事情
2018-01-06T20:21:16.345666Z 0 [Note] WSREP: forgetting 18001711 (tcp://192.168.50.123:4567) 2018-01-06T20:21:16.347641Z 1 [Note] WSREP: New cluster view: global state: 06ba4e9e-f315-11e7-b6ee-5381f105897f:0, view# 3: Primary, number of nodes: 1, my index: 0, p rotocol version 3 2018-01-06T20:21:16.347649Z 1 [Note] WSREP: Setting wsrep_ready to true 2018-01-06T20:21:20.290716Z 0 [Note] WSREP: (6abfb617, 'tcp://0.0.0.0:4567') connection established to 18001711 tcp://192.168.50.123:4567 2018-01-06T20:21:20.290914Z 0 [Warning] WSREP: discarding established (time wait) 18001711 (tcp://192.168.50.123:4567) 2018-01-06T20:21:21.532760Z 0 [Note] WSREP: cleaning up 18001711 (tcp://192.168.50.123:4567) 2018-01-06T20:21:24.686376Z 0 [Note] WSREP: (6abfb617, 'tcp://0.0.0.0:4567') connection established to 2b5261e6 tcp://192.168.50.123:4567 2018-01-06T20:21:24.687204Z 0 [Note] WSREP: (6abfb617, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 2018-01-06T20:21:25.186229Z 0 [Note] WSREP: declaring 2b5261e6 at tcp://192.168.50.123:4567 stable 2018-01-06T20:21:25.187176Z 0 [Note] WSREP: Node 6abfb617 state primary 2018-01-06T20:21:25.188552Z 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID. 2018-01-06T20:21:25.687047Z 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 2beb51c4-f31f-11e7-81a1-4fd8d45f8fe6 2018-01-06T20:21:25.687672Z 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2beb51c4-f31f-11e7-81a1-4fd8d45f8fe6 from 0 (garb) 2018-01-06T20:21:25.687999Z 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2beb51c4-f31f-11e7-81a1-4fd8d45f8fe6 from 1 (pxc-cluster-node-1) 2018-01-06T20:21:25.688108Z 1 [Note] WSREP: Setting wsrep_ready to true 2018-01-06T20:21:25.689550Z 0 [Note] WSREP: Member 0.0 (garb) requested state transfer from '*any*'. Selected 1.0 (pxc-cluster-node-1)(SYNCED) as donor. 2018-01-06T20:21:25.689574Z 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 0) 2018-01-06T20:21:25.690494Z 0 [Note] WSREP: 0.0 (garb): State transfer from 1.0 (pxc-cluster-node-1) complete. 2018-01-06T20:21:25.690644Z 0 [Note] WSREP: 1.0 (pxc-cluster-node-1): State transfer to 0.0 (garb) complete. 2018-01-06T20:21:25.690657Z 0 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 0) 2018-01-06T20:21:25.691247Z 0 [Note] WSREP: Member 0.0 (garb) synced with group. 2018-01-06T20:21:25.691511Z 0 [Note] WSREP: Member 1.0 (pxc-cluster-node-1) synced with group. 2018-01-06T20:21:25.691526Z 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0) 2018-01-06T20:21:25.691552Z 1 [Note] WSREP: Synchronized with group, ready for connections
接著啟動 pxc-2 mysql (wsrep.conf 參數記得要對要調整)
root@pxc-2:~# /etc/init.d/mysql start
接下來從 pxc-1 mysqld.conf 觀察 pxc-2 接入狀況
2018-01-06T20:28:31.870787Z 0 [Note] WSREP: (6abfb617, 'tcp://0.0.0.0:4567') connection established to 29bf0e70 tcp://192.168.50.129:4567
2018-01-06T20:28:31.871366Z 0 [Note] WSREP: (6abfb617, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:
2018-01-06T20:28:32.371843Z 0 [Note] WSREP: declaring 29bf0e70 at tcp://192.168.50.129:4567 stable
2018-01-06T20:28:32.372083Z 0 [Note] WSREP: declaring 2b5261e6 at tcp://192.168.50.123:4567 stable
2018-01-06T20:28:32.373326Z 0 [Note] WSREP: Node 2b5261e6 state primary
2018-01-06T20:28:32.375452Z 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
2018-01-06T20:28:32.871724Z 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 2a58080e-f320-11e7-9aac-43bf37747272
2018-01-06T20:28:32.873234Z 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2a58080e-f320-11e7-9aac-43bf37747272 from 0 (pxc-cluster-node-2)
2018-01-06T20:28:32.873578Z 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2a58080e-f320-11e7-9aac-43bf37747272 from 2 (pxc-cluster-node-1)
2018-01-06T20:28:32.874003Z 0 [Note] WSREP: STATE EXCHANGE: got state msg: 2a58080e-f320-11e7-9aac-43bf37747272 from 1 (garb)
2018-01-06T20:28:32.874156Z 1 [Note] WSREP: Setting wsrep_ready to true
2018-01-06T20:28:33.469687Z 0 [Note] WSREP: Member 0.0 (pxc-cluster-node-2) requested state transfer from '*any*'. Selected 2.0 (pxc-cluster-node-1)(SYNCED) as donor.
2018-01-06T20:28:33.469719Z 0 [Note] WSREP: Shifting SYNCED -> DONOR/DESYNCED (TO: 0)
2018-01-06T20:28:34.950529Z 0 [Note] WSREP: (6abfb617, 'tcp://0.0.0.0:4567') turning message relay requesting off
2018-01-06T20:28:43.838232Z WSREP_SST: [INFO] Streaming the backup to joiner at 192.168.50.129 4444
^[[B^[[2018-01-06T20:28:48.541928Z 0 [Note] WSREP: 2.0 (pxc-cluster-node-1): State transfer to 0.0 (pxc-cluster-node-2) complete.
2018-01-06T20:28:48.541990Z 0 [Note] WSREP: Shifting DONOR/DESYNCED -> JOINED (TO: 0)
2018-01-06T20:28:48.543943Z 0 [Note] WSREP: Member 2.0 (pxc-cluster-node-1) synced with group.
2018-01-06T20:28:48.543979Z 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 0)
2018-01-06T20:28:48.544032Z 6 [Note] WSREP: Synchronized with group, ready for connections
2018-01-06T20:28:54.161918Z 0 [Note] WSREP: 0.0 (pxc-cluster-node-2): State transfer from 2.0 (pxc-cluster-node-1) complete.
2018-01-06T20:28:54.163480Z 0 [Note] WSREP: Member 0.0 (pxc-cluster-node-2) synced with group.
確認 SYNC 後我們抓 wsrep 參數出來看
Cluster 成員確認三員
mysql> SHOW GLOBAL STATUS LIKE "wsrep_cluster_size"; +--------------------+-------+ | Variable_name | Value | +--------------------+-------+ | wsrep_cluster_size | 3 | +--------------------+-------+ 1 row in set (0.00 sec)
Cluster wsrep_local_state_comment 確認 SYNCED
mysql> SHOW GLOBAL STATUS LIKE "wsrep_local_state_comment"; +---------------------------+--------+ | Variable_name | Value | +---------------------------+--------+ | wsrep_local_state_comment | Synced | +---------------------------+--------+ 1 row in set (0.00 sec)
比較有趣的是 wsrep_incoming_addresses，因為跑 garbd 的機器上沒有 port 3306 的服務
所以 wsrep_incoming_addresses 就直接留空白了
mysql> SHOW GLOBAL STATUS LIKE "wsrep_incoming_addresses"; +--------------------------+-----------------------------------------+ | Variable_name | Value | +--------------------------+-----------------------------------------+ | wsrep_incoming_addresses | 192.168.50.129:3306,,192.168.50.18:3306 | +--------------------------+-----------------------------------------+ 1 row in set (0.00 sec)
Done.
我也碰到了，ip缺少一個的情況，我想問一下，如果garbd在3306端口上運行，是不是獲取到的ip就是不會缺少了。
garbd 沒辦法直接走 port 3306 ; 因為 galera 的判斷一定要從 port 4567
從另外兩個 cluster member 的 port service 判斷就觀察得到一些細節了
root@lab:~# netstat -ntlp | grep mysql
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 27107/mysqld
tcp 0 0 0.0.0.0:4567 0.0.0.0:* LISTEN 27107/mysqld
但也可以試試強迫指定 garbd -a gcomm://ip:3306 會發生什麼事情也是很有趣的
謝謝，不過我瀏覽了很多博客，很多人獲取到的ip都是正常的，對於怎麼獲取丟失的ip,你還有什麼解決辦法嗎？
我不認為那應該被稱作 “丟失的 IP”
因為實作的方式是 data node * 2 + galera arb * 1 (不是 data node ; 只做 Vote 行為)
wsrep_incoming_addresses 只紀錄 data node IP 是符合預期結果的
如果真的在意 wsrep_incoming_addresses 的數量的話，那就把完整的三個 Cluster Member 做完就好
Ref: https://blog.wnlin.org/how-to-setup-3-node-percona-xtradb-cluster-on-debian-jessie/
留意一下 variable wsrep_node_incoming_address 的說明
Specifies the network address from which the node expects client connections. By default, it uses the IP address from wsrep_node_address and port number 3306.
Ref: https://www.percona.com/doc/percona-xtradb-cluster/LATEST/wsrep-system-index.html
好的，你的回答很有幫助，謝謝啦