[笔记]HAproxy reload config file with uninterrupt session
HAProxy is a high performance load balancer. It is very light-weight, and free, making it a great option if you are in the market for a load balancer and need to keep your costs down.
Lately we’ve been making a lot of load balancer changes at work to accommodate new systems and services. Even though we have two load balancers running with keepalived taking care of any failover situations, I was thinking about how we go about reloading our configuration files. In the event of a change, the “common” way to get the changes to take effect is to run /etc/init.d/haproxy restart. This is bad for a couple major reasons:
You are temporarily shutting your load balancer down
You are severing any current connections going through the load balancer
You might say, “if you have two load balancers with keepalived, restarting the service should be fine since keepalived will handle the failover.” This, however, isn’t always true. Keepalived uses advertisements to determine when to fail over. The default advertisement interval is 1 second (configurable in keepalived.conf). The skew time helps to keep everyone from trying to transition at once. It is a number between 0 and 1, based on the formula (256 – priority) / 256. As defined in the RFC, the backup must receive an advertisement from the master every (3 * advert_int) + skew_time seconds. If it doesn’t hear anything from the master, it takes over.
Let’s assume you are using the default interval of 1 second. On my test machine, this is the duration of time it takes to restart haproxy:
time /etc/init.d/haproxy restart
Restarting haproxy haproxy
...done.
real 0m0.022s
user 0m0.000s
sys 0m0.016s
In this situation, haproxy would restart much faster than your 1 second interval. You could get lucky and happen to restart it just before the check, but luck is not consistent enough to be useful. Also, in very high-traffic situations, you’ll be causing a lot of connection issues. So we cannot rely on keepalived to solve the first problem, and it definitely doesn’t solve the second problem.
After sifting through haproxy documentation (the text-based documentation, not the man page) (/usr/share/doc/haproxy/haproxy-en.txt.gz on Ubuntu), I came across this:
313
314 global
315 daemon
316 quiet
317 nbproc 2
318 pidfile /var/run/haproxy-private.pid
319
320 # to stop only those processes among others :
321 # kill $(</var/run/haproxy-private.pid)
322
323 # to reload a new configuration with minimal service impact and without
324 # breaking existing sessions :
325 # haproxy -f haproxy.cfg -p $(</var/run/haproxy-private.pid) -st $(</var/run/haproxy-private.pid)
That last command is the one of interest. The -p asks the process to write down each of its children’s pids to the specified pid file, and the -st specifies a list of pids to send a SIGTERM to after startup. But it does this in an interesting way:
609 The '-st' and '-sf' command line options are used to inform previously running
610 processes that a configuration is being reloaded. They will receive the SIGTTOU
611 signal to ask them to temporarily stop listening to the ports so that the new
612 process can grab them. If anything wrong happens, the new process will send
613 them a SIGTTIN to tell them to re-listen to the ports and continue their normal
614 work. Otherwise, it will either ask them to finish (-sf) their work then softly
615 exit, or immediately terminate (-st), breaking existing sessions. A typical use
616 of this allows a configuration reload without service interruption :
617
618 # haproxy -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)
The end-result is a reload of the configuration file which is not visible by the customer. It also solves the second problem! Let’s look at an example of the command and look at the time compared to our above example:
# time haproxy -f /etc/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)
real 0m0.018s
user 0m0.000s
sys 0m0.004s
I’ve specified the config file I want to use and the pid file haproxy is currently using. The $(cat /var/run/haproxy.pid) takes the output of cat /var/run/haproxy.pid and passes it in to the -sf parameter as a list, which is what it is expecting. You will notice that the time is actually faster too (.012s sys, and .004s real). It may not seem like much, but if you are dealing with very high volumes of traffic, this can be pretty important. Luckily for us it doesn’t matter because we’ve been able to reload the haproxy configuration without dropping any connections and without causing any customer-facing issues.
UPDATE: There is a reload in some of the init.d scripts (I haven’t checked every OS, so this can vary), but it uses the -st option which will break existing sessions, as opposed to using -sf to do a graceful hand-off. You can modify the haproxy_reload() function to use the -sf if you want. I also find it a bit confusing that the documentation uses $(cat /path/to/pidfile) whereas this haproxy_reload() function uses $(<$PIDFILE). Either should work, but really, way to lead by example…
[笔记]HAproxy reload config file with uninterrupt session的更多相关文章
- peomethues 参数设置 监控网站 /usr/local/prometheus-2.13.0.linux-amd64/prometheus --config.file=/usr/local/prometheus-2.13.0.linux-amd64/prometheus.yml --web.listen-address=:9999 --web.enable-lifecycle
probe_http_status_code{instance="xxxx",job="web_status"} probe_http_status_code{ ...
- Example config file /etc/vsftpd.conf
# Example config file /etc/vsftpd.conf # # The default compiled in settings are fairly paranoid. Thi ...
- error in config file "/etc/rabbitmq/rabbitmq.config"
记录一次RabbitMQ配置文件配置错误 error信息: dill@ubuntu-vm:/usr/share/doc/rabbitmq-server$ sudo /usr/lib/rabbitmq/ ...
- .NET错误The 'targetFramework' attribute in the <compilation> element of the Web.config file is used only to target version 4.0 and later of the .NET Framework
错误描述: The 'targetFramework' attribute in the <compilation> element of the Web.config file is u ...
- No connection string named '***' could be found in the application config file
Code-First时更新数据库遇到妖孽问题“No connection string named '***' could be found in the application config fil ...
- Camel routes in Spring config file
The normal spring bean definition configuration file, the xsi:schemaLocation only has two: beans and ...
- The document has been modified outside of Code Composer. Would you like to reload the file
2013-06-20 10:03:32 烧写过程是合众达给出的文档 problem: I'm new to using Code Composer Studio 3.3 and am having a ...
- 为 vsftpd 启动 vsftpd:500 OOPS: bad bool value in config file for: pasv_enable
每行的值都不要有空格,否则启动时会出现错误,举个例子,假如我在listen=YES后多了个空格,那我启动时就出现.. 为 vsftpd 启动 vsftpd:500 OOPS: bad bool val ...
- Warning: World-writable config file '/etc/my.cnf' is ignored
1. 问题描述: 重启mysql服务时出现以下信息: Warning: World-writable config file '/etc/my.cnf' is ignored 出现这种情况的原因是:m ...
随机推荐
- Elasticsearch之java的基本操作一
摘要 接触ElasticSearch已经有一段了.在这期间,遇到很多问题,但在最后自己的不断探索下解决了这些问题.看到网上或多或少的都有一些介绍ElasticSearch相关知识的文档,但个人觉得 ...
- 为C# as 类型转换及Assembly.LoadFrom埋坑!
背景: 不久前,我发布了一个调试工具:发布:.NET开发人员必备的可视化调试工具(你值的拥有) 效果是这样的: 之后,有小部分用户反映,工具用不了(没反应或有异常)~~~ 然后,建议小部分用户换个电脑 ...
- java EE设计模式简介
1.何为设计模式 设计模式提供了对常见应用设计问题的解决方案.在面向对象的编程中,设计模式通常在解决与对象创建和交互相关的问题,而非整体软件架构所面对的大规模问题,它们以样板代码的形式提供了通用的解决 ...
- 【原】FMDB源码阅读(二)
[原]FMDB源码阅读(二) 本文转载请注明出处 -- polobymulberry-博客园 1. 前言 上一篇只是简单地过了一下FMDB一个简单例子的基本流程,并没有涉及到FMDB的所有方方面面,比 ...
- shell简介
Shell作为命令语言,它交互式地解释和执行用户输入的命令:作为程序设计语言,它定义了各种变量和参数,并提供了许多在高级语言中才具有的控制结构,包括循环和分支. shell使用的熟练程度反映了用户对U ...
- ASP.NET MVC5+EF6+EasyUI 后台管理系统(69)-微信公众平台开发-功能概述
系列目录 为什么要先发这个文章? 因为接下来的文章是关于微信开发的系列,心中一定要有一个概念,知道自己接下来要做什么功能. 而且微信到处都是坑,我首先要把微信与本地跑通起来才敢发布,否则中间出现坑导致 ...
- 【基于WinForm+Access局域网共享数据库的项目总结】之篇一:WinForm开发总体概述与技术实现
篇一:WinForm开发总体概述与技术实现 篇二:WinForm开发扇形图统计和Excel数据导出 篇三:Access远程连接数据库和窗体打包部署 [小记]:最近基于WinForm+Access数据库 ...
- C#通过NPOI操作Excel
参考页面: http://www.yuanjiaocheng.net/webapi/create-crud-api-1-post.html http://www.yuanjiaocheng.net/w ...
- Spring异步功能
使用 Spring 的异步功能时,实质是使用的 Servlet3 及以上版本的异步功能. Spring 的异步处理机制需要在 web.xml 中全部的 servlet 和 filter 处配置 < ...
- 邻接矩阵的深度优先遍历(java版)
这是一个有向边带权的图 顶点数组:[v0, v1, v2, v3, v4] 边数组: v0 v1 v2 v3 v4 v0 6 v1 9 3 v2 2 5 v3 1 v4 package com.dat ...