[checkmk-commits] 4901 Reworked Livestatus Proxy to be more scalable

Module: check_mk Branch: master Commit: 203bc9f512244781900368562f022d706a0e3d8c URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=203bc9f5122447… Author: Lars Michelsen &lt;lm(a)mathias-kettner.de&gt; Date: Tue Jun 27 11:54:22 2017 +0200 4901 Reworked Livestatus Proxy to be more scalable The livestatus proxy daemon has been reworked to use a multi process architecture. It now has a master process that mainly cares about monitoring the site processes. Each connected site has an own subprocess that manages all channels to this site and the clients that use these channels to communicate with the sites. When you have a look at ps/top you should see something like this: C+: OMD[heute]:~$ ps -ef | grep liveproxyd UID PID PPID C STIME TTY TIME CMD heute 9261 1 0 11:40 ? 00:00:00 liveproxyd[master] heute 9262 9261 0 11:40 ? 00:00:00 liveproxyd[heute_slave_1] heute 9263 9261 0 11:40 ? 00:00:00 liveproxyd[heute_slave_2] C-: As you can may see there is the master process that has the process <tt>1</tt> as parent process. The site processes have the master as parent. When one site process terminates for some reason the master will restart it. When the master is terminated all site processes will terminate too. In case of a restart or config reload the master will restart itself and stop all site processes and restart them again. With this change the load of the livestatus proxy will now spread over multiple CPUs. Side note: There is a new global setting <i>Logging of the Livestatus Proxy</i> that can be used to control the detail level of the log entries written to the <tt>var/log/liveproxyd.log</tt> log file. In case you experience any issues with the livestatus proxy daemon take a look at this log file and maybe increase the log level to get more details. Change-Id: If50c056ae91e155b1de9d44ddd7c6b4ce436c8e4 --- .werks/4901 | 41 +++++++++++++++++++++++++++++++++++++ tests/livestatus/test_liveproxyd.py | 14 +++++++++---- 2 files changed, 51 insertions(+), 4 deletions(-) diff --git a/.werks/4901 b/.werks/4901 new file mode 100644 index 0000000..f384e1f --- /dev/null +++ b/.werks/4901 @@ -0,0 +1,41 @@ +Title: Reworked Livestatus Proxy to be more scalable +Level: 2 +Component: liveproxy +Compatible: compat +Edition: cee +Version: 1.5.0i1 +Date: 1498556532 +Class: feature + +The livestatus proxy daemon has been reworked to use a multi process +architecture. It now has a master process that mainly cares about +monitoring the site processes. Each connected site has an own subprocess +that manages all channels to this site and the clients that use these +channels to communicate with the sites. + +When you have a look at ps/top you should see something like this: + +C+: +OMD[heute]:~$ ps -ef | grep liveproxyd +UID PID PPID C STIME TTY TIME CMD +heute 9261 1 0 11:40 ? 00:00:00 liveproxyd[master] +heute 9262 9261 0 11:40 ? 00:00:00 liveproxyd[heute_slave_1] +heute 9263 9261 0 11:40 ? 00:00:00 liveproxyd[heute_slave_2] +C-: + +As you can may see there is the master process that has the process <tt>1</tt> +as parent process. The site processes have the master as parent. + +When one site process terminates for some reason the master will restart it. +When the master is terminated all site processes will terminate too. In case +of a restart or config reload the master will restart itself and stop +all site processes and restart them again. + +With this change the load of the livestatus proxy will now spread over +multiple CPUs. + +Side note: There is a new global setting <i>Logging of the Livestatus Proxy</i> +that can be used to control the detail level of the log entries written to the +<tt>var/log/liveproxyd.log</tt> log file. In case you experience any issues +with the livestatus proxy daemon take a look at this log file and maybe increase +the log level to get more details. diff --git a/tests/livestatus/test_liveproxyd.py b/tests/livestatus/test_liveproxyd.py index 4ab3503..c57bfb8 100644 --- a/tests/livestatus/test_liveproxyd.py +++ b/tests/livestatus/test_liveproxyd.py @@ -138,10 +138,10 @@ def test_large_number_of_sites(default_cfg, site): # Currently connect to local site site_id = "site%03d" % site_num - if site_num % 2 == 0: - to_livestatus = None - else: - to_livestatus = ('127.0.0.1', 6999) + #if site_num % 2 == 0: + to_livestatus = None + #else: + # to_livestatus = ('127.0.0.1', 6999) site_sockets[site_id] = { "to_livestatus" : to_livestatus, @@ -184,6 +184,12 @@ def test_large_number_of_sites(default_cfg, site): site.execute(["cmk", "-O"]) + # Disable limits of livestatus xinetd service + #site.execute(["sed", "-i", "-r", + # "'/^\s+cps\s+=/d;/^\s+instances\s+=/d;/^\s+per_source\s+=/d'", + # "etc/xinetd.d/mk-livestatus"]) + #site.execute(["omd", "restart", "xinetd"]) + _change_liveproxyd_sites(site, liveproxyd_sites) def _num_connections_opened():