I recently made 2 nodes with mariadb galera on banana pi and one with haproxy (on Banana Pi smart router - separate machine) and i notice that when i'm connecting through haproxy node, new connections (i mean opening new connections) take a lot of time (much,much more time than direct)
Direct connect from Haproxy(Banana Pi smart router without haproxy) node works very fast but with Haproxy it takes forever
.Second thing that i want to solve is ... performance.
Sysbench test through direct connect (100 threads, no restriction or limit on connection count) gives me on both nodes
Code: Select all
OLTP test statistics:
queries performed:
read: 21546
write: 7695
other: 3078
total: 32319
transactions: 1539 (49.18 per sec.)
deadlocks: 0 (0.00 per sec.)
read/write requests: 29241 (934.43 per sec.)
other operations: 3078 (98.36 per sec.)
Test execution summary:
total time: 31.2928s
total number of events: 1539
total time taken by event execution: 3086.7654
per-request statistics:
min: 1087.02ms
avg: 2005.70ms
max: 3504.86ms
approx. 95 percentile: 2658.12ms
Threads fairness:
events (avg/stddev): 15.3900/2.28
execution time (avg/stddev): 30.8677/0.34
Code: Select all
OLTP test statistics:
queries performed:
read: 16548
write: 5910
other: 2365
total: 24823
transactions: 1180 (35.33 per sec.)
deadlocks: 3 (0.09 per sec.)
read/write requests: 22458 (672.46 per sec.)
other operations: 2365 (70.82 per sec.)
Test execution summary:
total time: 33.3966s
total number of events: 1180
total time taken by event execution: 3296.7437
per-request statistics:
min: 57.14ms
avg: 2793.85ms
max: 20223.17ms
approx. 95 percentile: 18962.81ms
Threads fairness:
events (avg/stddev): 11.8000/12.21
execution time (avg/stddev): 32.9674/0.35
Some configuration and information:
1 sysbench machine config (this one connect through haproxy)
- Intel i5, SSD, 8GB or RAM DDR3 (Linux elitebook 3.18.11-gentoo #1 SMP Mon May 18 19:01:30 CEST 2015 x86_64 Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz GenuineIntel GNU/Linux)
command that is invoked on that machine:
Code: Select all
sysbench --test=oltp --num-threads=100 --max-time=30 --max-requests=0 --mysql-table-engine=InnoDB --mysql-db=test --mysql-user=root --mysql-password=passs --mysql-host=r1.dom --mysql-engine-trx=yes --oltp-table-size=2000000 run- BanaPi R1 (smart router) with Allwinner A20(2 cores, 1GHz each), SSD, 1GB or RAM DDR3 (Linux r1 3.4.104+ #1 SMP PREEMPT Thu Jan 8 15:40:40 CET 2015 armv7l ARMv7 Processor rev 4 (v7l) sun7i GNU/Linux)
command that is invoked on that machine:
Code: Select all
sysbench --test=oltp --num-threads=100 --max-time=30 --max-requests=0 --mysql-table-engine=InnoDB --mysql-db=test --mysql-user=root --mysql-password=passs --mysql-host=node{one or two}.dom --mysql-engine-trx=yes --oltp-table-size=2000000 runhaproxy config
Code: Select all
global
# log redirection (syslog)
#log 127.0.0.1 local0
#log 127.0.0.1 local1 debug
log /dev/log local0
log /dev/log local1 notice
#apache2
tune.ssl.default-dh-param 2048
# maximum of connections for haproxy
maxconn 2048
#maxpipes 512
# chroot for security reasons
chroot /var/lib/haproxy
# user/group for haproxy process
user haproxy
group haproxy
#command to use followed hatop -s /var/lib/haproxy/stats
stats socket /var/lib/haproxy/stats mode 777 level admin
# act as a daemon
daemon
defaults
# use gloval log declaration
log global
# default check type
#mode http
# only log when closing session
# only log failed connections
# retry 3 times before setting node as failed
# redispatch traffic to other servers
option tcplog
option dontlognull
option redispatch
# maximum connection for the frontend
maxconn 1024
# timeouts
timeout client 1s
timeout client-fin 1s
timeout server 1s
timeout connect 1s
retries 3
timeout queue 1m
timeout check 20s
# enable web check health interface on port 80
listen haproxy 192.168.2.5:8080
stats enable
mode http
fullconn 1024
#stats hide-version
#stats uri /stats
#stats realm Haproxy\ Statistics
# set credentials
stats auth admin:password
# loadbalance on slaves - we need acces from eth0_101 and eth0_102
frontend MDBG
#bind 192.168.2.5:3306
bind 0.0.0.0:3306
timeout client 57600s
#timeout client-fin 57600s
default_backend mariadb-galera
backend mariadb-galera
fullconn 1024
#option splice-auto
timeout server 57600s
timeout connect 5s
# use tcp method
mode tcp
# round robin mechanism
#balance roundrobin
balance leastconn
# tcp keepalive (pipelining) on both side (clt/srv)
option tcpka
# perform http request
#option httpchk
# perform mariadb connection with haproxy user
option mysql-check user haproxy
# set all read only nodes
# inter: interval of check in milliseconds
# check port 9200 inter 2000 rise 3 fall 3
server vulcanone 172.16.1.21:3306 check inter 2000 rise 3 fall 3 maxconn 512
server vulcantwo 172.16.1.22:3306 check inter 2000 rise 3 fall 3 maxconn 512
frontend LB
bind 192.168.2.5:80
timeout client 57600s
#timeout client-fin 57600s
reqadd X-Forwarded-Proto:\ http
default_backend apache2
frontend LBS
#bind 0.0.0.0:443
bind 192.168.2.5:443 ssl crt /etc/haproxy/ssl/localhost.pem
timeout client 57600s
timeout client-fin 57600s
reqadd X-Forwarded-Proto:\ https
default_backend apache2
backend apache2
fullconn 1024
timeout server 57600s
timeout connect 5s
redirect scheme https if !{ ssl_fc }
timeout http-request 20
timeout http-keep-alive 30
mode http
balance roundrobin
cookie PHPSESSID prefix indirect nocache
option httplog
option http-server-close
option httpclose
option forwardfor
#http-request set-header X-Forwarded-Port %[dst_port]
#http-request add-header X-Forwarded-Proto https if { ssl_fc }
option httpchk HEAD / HTTP/1.1\r\nHost:localhost
server markone 172.16.1.11:80 cookie markone check maxconn 512
server marktwo 172.16.1.12:80 cookie marktwo check maxconn 512
#server web4-srv 192.168.0.124:80 check backup # backup fail-over Server, If three of the above fails this will be activated.
Code: Select all
h3x0r@elitebook ~ $ ping r1.dom
PING r1.dom (192.168.2.5) 56(84) bytes of data.
64 bytes from r1.dom (192.168.2.5): icmp_seq=1 ttl=64 time=0.335 ms
64 bytes from r1.dom (192.168.2.5): icmp_seq=2 ttl=64 time=0.255 ms
64 bytes from r1.dom (192.168.2.5): icmp_seq=3 ttl=64 time=0.320 ms
64 bytes from r1.dom (192.168.2.5): icmp_seq=4 ttl=64 time=0.194 msCode: Select all
[h3x0r@r1 ~]$ ping vulcanone.dom
PING vulcanone.dom (172.16.1.21) 56(84) bytes of data.
64 bytes from vulcanone.dom (172.16.1.21): icmp_seq=1 ttl=64 time=0.339 ms
64 bytes from vulcanone.dom (172.16.1.21): icmp_seq=2 ttl=64 time=0.225 ms
^C
--- vulcanone.dom ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.225/0.282/0.339/0.057 ms
[h3x0r@r1 ~]$ ping vulcantwo.dom
PING vulcantwo.dom (172.16.1.22) 56(84) bytes of data.
64 bytes from vulcantwo.dom (172.16.1.22): icmp_seq=1 ttl=64 time=0.432 ms
64 bytes from vulcantwo.dom (172.16.1.22): icmp_seq=2 ttl=64 time=0.283 ms
64 bytes from vulcantwo.dom (172.16.1.22): icmp_seq=3 ttl=64 time=0.329 ms
Is there any way to make haproxy more "transparentable" ? How to reduce this performance issues ?
