Checkmk git commits February 2019

checkmk-commits@lists.checkmk.com

18 participants
457 discussions

7143 mk_docker.py: A new docker monitoring plugin

by Moritz Kiemer

Module: check_mk Branch: master Commit: 2d43d696de115e458d82a39d20403d884cef8f47 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=2d43d696de115e… Author: Moritz Kiemer <mo(a)mathias-kettner.de> Date: Fri Feb 8 13:29:03 2019 +0100 7143 mk_docker.py: A new docker monitoring plugin We introduce a new plugin for docker monitoring, 'mk_docker.py'. It combines the functionality of the two deprecated plugins 'mk_docker_node' and 'mk_docker_container_piggybacked', while adding more configuration options. If you where using both plugins, you can use the new plugin in its default configuration to replace them. If you have been using only one of them, you can disable the opposite parts of the new plugin, either by using the bakery rule "Docker node and containers" or via the configuration file (please refer to the example files in agents/cfg_examples fo rmore details). The previous rule "Piggybacked docker containers" is deprecated - please set up a corresponding new one. To deploy this plugin you must at least have the 'docker-py' python library version 3.0.0 installed on the monitored system. You can install it using the python package manager pip (using the shell command "pip install docker-py"). CMK-1564 Change-Id: Ic8a619f7cf784ee3ebf9cde52c5d6ff865ad3e0e --- .werks/7143 | 29 ++++++++++++++ agents/plugins/mk_docker_container_piggybacked | 55 -------------------------- agents/plugins/mk_docker_node | 49 ----------------------- checks/docker_container_status | 1 + checks/docker_node_disk_usage | 1 + checks/docker_node_info | 2 + 6 files changed, 33 insertions(+), 104 deletions(-) diff --git a/.werks/7143 b/.werks/7143 new file mode 100644 index 0000000..91b885f --- /dev/null +++ b/.werks/7143 @@ -0,0 +1,29 @@ +Title: mk_docker.py: A new docker monitoring plugin +Level: 1 +Component: checks +Compatible: incomp +Edition: cre +Version: 1.6.0i1 +Date: 1550048642 +Class: feature + +We introduce a new plugin for docker monitoring, 'mk_docker.py'. +It combines the functionality of the two deprecated plugins +'mk_docker_node' and 'mk_docker_container_piggybacked', while +adding more configuration options. + +If you where using both plugins, you can use the new plugin in its +default configuration to replace them. If you have been using only +one of them, you can disable the opposite parts of the new plugin, +either by using the bakery rule "Docker node and containers" or via +the configuration file (please refer to the example files in +agents/cfg_examples fo rmore details). + +The previous rule "Piggybacked docker containers" is deprecated - +please set up a corresponding new one. + +To deploy this plugin you must have the 'docker-py' python library +installed on the monitored system. You can install it using the +python package manager pip (using the shell command +"pip install docker-py"). + diff --git a/agents/plugins/mk_docker_container_piggybacked b/agents/plugins/mk_docker_container_piggybacked deleted file mode 100755 index 36cf98c..0000000 --- a/agents/plugins/mk_docker_container_piggybacked +++ /dev/null @@ -1,55 +0,0 @@ -#!/bin/bash -# +------------------------------------------------------------------+ -# | ____ _ _ __ __ _ __ | -# | / ___| |__ ___ ___| | __ | \/ | |/ / | -# | | | | '_ \ / _ \/ __| |/ / | |\/| | ' / | -# | | |___| | | | __/ (__| < | | | | . \ | -# | \____|_| |_|\___|\___|_|\_\___|_| |_|_|\_\ | -# | | -# | Copyright Mathias Kettner 2018 mk(a)mathias-kettner.de | -# +------------------------------------------------------------------+ -# -# This file is part of Check_MK. -# The official homepage is at http://mathias-kettner.de/check_mk. -# -# check_mk is free software; you can redistribute it and/or modify it -# under the terms of the GNU General Public License as published by -# the Free Software Foundation in version 2. check_mk is distributed -# in the hope that it will be useful, but WITHOUT ANY WARRANTY; with- -# out even the implied warranty of MERCHANTABILITY or FITNESS FOR A -# PARTICULAR PURPOSE. See the GNU General Public License for more de- -# tails. You should have received a copy of the GNU General Public -# License along with GNU Make; see the file COPYING. If not, write -# to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, -# Boston, MA 02110-1301 USA. - -# $REMOTE is exported from check_mk_agent.linux - -if type docker > /dev/null 2>&1; then - NODE_NAME=$(docker info --format "{{json .Name}}") - - # For the container status, we want information about *all* containers - for CONTAINER_ID in $(docker container ls -q --all); do - echo "<<<<${CONTAINER_ID}>>>>" - docker inspect "$CONTAINER_ID" \ - --format='{{println "<<<docker_container_status>>>"}}{{json .State}}{{println}}{{println "<<<docker_container_node_name>>>"}}{{println '"$NODE_NAME"'}}{{println "<<<docker_container_labels>>>"}}{{json .Config.Labels}}{{println}}{{println "<<<docker_container_network>>>"}}{{json .NetworkSettings}}{{println}}' - - if [ "$(docker inspect -f '{{.State.Running}}' "$CONTAINER_ID")" = "true" ]; then - # Is there a regular agent available in the container? Use it! - # - # Otherwise execute the agent of the node in the context of the container. - # Using this approach we should always get at least basic information from - # the container. - # Once it comes to plugins and custom configuration the user needs to use - # a little more complex setup. Have a look at the documentation. - AGENT_PATH=$(docker container exec "$CONTAINER_ID" bash -c "type check_mk_agent" 2>/dev/null) || AGENT_PATH= - if [ -n "$AGENT_PATH" ]; then - docker container exec --env "REMOTE=$REMOTE" "$CONTAINER_ID" check_mk_agent - elif docker container exec "$CONTAINER_ID" which bash >/dev/null 2>&1; then - docker container exec --env MK_FROM_NODE=1 --env "REMOTE=$REMOTE" -i "$CONTAINER_ID" bash < "$0" - fi - fi - - echo "<<<<>>>>" - done -fi diff --git a/agents/plugins/mk_docker_node b/agents/plugins/mk_docker_node deleted file mode 100755 index 7bc1080..0000000 --- a/agents/plugins/mk_docker_node +++ /dev/null @@ -1,49 +0,0 @@ -#!/bin/bash -# +------------------------------------------------------------------+ -# | ____ _ _ __ __ _ __ | -# | / ___| |__ ___ ___| | __ | \/ | |/ / | -# | | | | '_ \ / _ \/ __| |/ / | |\/| | ' / | -# | | |___| | | | __/ (__| < | | | | . \ | -# | \____|_| |_|\___|\___|_|\_\___|_| |_|_|\_\ | -# | | -# | Copyright Mathias Kettner 2016 mk(a)mathias-kettner.de | -# +------------------------------------------------------------------+ -# -# This file is part of Check_MK. -# The official homepage is at http://mathias-kettner.de/check_mk. -# -# check_mk is free software; you can redistribute it and/or modify it -# under the terms of the GNU General Public License as published by -# the Free Software Foundation in version 2. check_mk is distributed -# in the hope that it will be useful, but WITHOUT ANY WARRANTY; with- -# out even the implied warranty of MERCHANTABILITY or FITNESS FOR A -# PARTICULAR PURPOSE. See the GNU General Public License for more de- -# tails. You should have received a copy of the GNU General Public -# License along with GNU Make; see the file COPYING. If not, write -# to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, -# Boston, MA 02110-1301 USA. - -if type docker > /dev/null 2>&1; then - echo "<<<docker_node_info:sep(58)>>>" - # sed: protect leading whitespace - docker info | sed 's/^/|/' - - echo "<<<docker_node_disk_usage:sep(0)>>>" - docker system df - - echo "<<<docker_node_images>>>" - echo "[[[images]]]" - docker images --format "{{json .}}" - - echo "[[[image_inspect]]]" - IMAGE_IDS=$(docker images | awk 'NR>1 {print $3}') - docker image inspect $IMAGE_IDS - - echo "[[[containers]]]" - # equivalent of 'docker ps' for newer docker versions: 'docker container ls' - docker ps --all --format "{{json .}}" - - echo "<<<docker_node_network:sep(0)>>>" - NETWORK_IDS=$(docker network ls -f 'driver=bridge' | awk 'NR>1 {print $1}') - docker network inspect $NETWORK_IDS -fi diff --git a/checks/docker_container_status b/checks/docker_container_status index 9920e05..69649fa 100644 --- a/checks/docker_container_status +++ b/checks/docker_container_status @@ -95,6 +95,7 @@ check_info["docker_container_status.health"] = { # '----------------------------------------------------------------------' +@append_deprecation_warning # pylint: disable=undefined-variable def check_docker_container_status(_no_item, _no_params, parsed): status = parsed.get("Status", "unknown") state = {"running": 0, "unknown": 3}.get(status, 2) diff --git a/checks/docker_node_disk_usage b/checks/docker_node_disk_usage index 21e770b..d5355e2 100644 --- a/checks/docker_node_disk_usage +++ b/checks/docker_node_disk_usage @@ -35,6 +35,7 @@ def parse_docker_node_disk_usage(info): return {r.get('type'): r for r in table_rows if r is not None} +@append_deprecation_warning # pylint: disable=undefined-variable @get_parsed_item_data def check_docker_node_disk_usage(_no_item, params, data): diff --git a/checks/docker_node_info b/checks/docker_node_info index ce80041..3d193d1 100644 --- a/checks/docker_node_info +++ b/checks/docker_node_info @@ -38,6 +38,7 @@ def parse_docker_node_info(info): return loaded or {} +@append_deprecation_warning # pylint: disable=undefined-variable def check_docker_node_info(_no_item, _no_params, parsed): if "Name" in parsed: yield 0, 'Daemon running on host %s' % parsed["Name"] @@ -56,6 +57,7 @@ check_info['docker_node_info'] = { } +@append_deprecation_warning # pylint: disable=undefined-variable def check_docker_node_containers(_no_item, params, parsed): def check_node_levels(count, levels, levels_lower): warn, crit = levels

5 years, 7 months

mk_docker.py: add docker library version to version info

by Moritz Kiemer

Module: check_mk Branch: master Commit: 4db5a08632b6947cc78bda861979eb58c78f3c28 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=4db5a08632b694… Author: Moritz Kiemer <mo(a)mathias-kettner.de> Date: Thu Feb 14 10:07:19 2019 +0100 mk_docker.py: add docker library version to version info Change-Id: I030488de8fc4b2dd5c0446c3d44edee52d1f1348 --- agents/plugins/mk_docker.py | 5 ++++- tests/unit/plugins/test_mk_docker.py | 2 +- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/agents/plugins/mk_docker.py b/agents/plugins/mk_docker.py index 5b0110a..a9b6fa1 100755 --- a/agents/plugins/mk_docker.py +++ b/agents/plugins/mk_docker.py @@ -124,7 +124,10 @@ def get_config(cfg_file): class Section(list): '''a very basic agent section class''' - version_info = {'PluginVersion': VERSION} + version_info = { + 'PluginVersion': VERSION, + 'DockerPyVersion': docker.version, + } # Should we need to parallelize one day, change this to be # more like the Section class in agent_azure, for instance diff --git a/tests/unit/plugins/test_mk_docker.py b/tests/unit/plugins/test_mk_docker.py index 685de73..63c3497 100644 --- a/tests/unit/plugins/test_mk_docker.py +++ b/tests/unit/plugins/test_mk_docker.py @@ -11,7 +11,7 @@ sys.path.insert(0, os.path.join(cmk_path(), 'agents', 'plugins')) import mk_docker # pylint: disable=import-error,wrong-import-position PLUGIN_CHECKSUMS = { - '0.1': '7e456e69affbebf7b2f070a6d6548e23', + '0.1': '2296ae916366d26d54ea449b2c2552bb', }

5 years, 7 months

add teardown for mk_logwatch tests

by Florian Kromer

Module: check_mk Branch: master Commit: 30a5036ba00fdb8062c4632e1e700ad4fc345cbd URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=30a5036ba00fdb… Author: Florian Kromer <fk(a)mathias-kettner.de> Date: Wed Feb 13 15:11:35 2019 +0100 add teardown for mk_logwatch tests Change-Id: Iac158a479a5a58ddd0e444a179fff1c634dfcfe6 --- tests/unit/agents/plugins/test_mk_logwatch.py | 60 +++++++++++++++++++++------ 1 file changed, 47 insertions(+), 13 deletions(-) diff --git a/tests/unit/agents/plugins/test_mk_logwatch.py b/tests/unit/agents/plugins/test_mk_logwatch.py index 28b8558..57059a5 100644 --- a/tests/unit/agents/plugins/test_mk_logwatch.py +++ b/tests/unit/agents/plugins/test_mk_logwatch.py @@ -5,11 +5,6 @@ import pytest from testlib import cmk_path # pylint: disable=import-error from re import compile -import imp -mk_logwatch = imp.load_source("mk_logwatch", - os.path.join(cmk_path(), 'agents', 'plugins', 'mk_logwatch')) -import mk_logwatch # pylint: disable=import-error,wrong-import-position - # consistent to check_mk/agents/cfg_examples/logwatch.cfg LOGWATCH_CONFIG_CONTENT = """ # +------------------------------------------------------------------+ @@ -94,7 +89,39 @@ CLUSTER another_cluster 1762:0:0:0:0:B03:1:AF18""" -def test_get_config_files(tmpdir): +(a)pytest.fixture(scope="module") +def agent_plugin_as_module(request): + """ + Fixture to inject source code of agent as module. Removes <source-code-file>c during teardown. + """ + # - setup - start + agent_path = os.path.abspath(os.path.join(cmk_path(), 'agents', 'plugins', 'mk_logwatch')) + # To make this setup fail safe and to prevent from potentially loading the module from a + # left-over mk_logwtachc file from previous test runs try to remove it. + try: + os.remove(agent_path + "c") + except OSError: + pass # it's ok if the file is not there + + # import the agent source file as module + # (imp.load_source() has the side effect of creating mk_logwatchc) + import imp + mk_logwatch = imp.load_source("mk_logwatch", agent_path) + import mk_logwatch # pylint: disable=import-error,wrong-import-position + + assert "get_config_files" in mk_logwatch.__dict__ + # - setup - end + yield mk_logwatch # inject module + # - teardown - start + try: + os.remove(agent_path + "c") + except OSError: + pass # it's ok if the file is not there + # - teardown - end + + +def test_get_config_files(agent_plugin_as_module, tmpdir): + mk_logwatch = agent_plugin_as_module fake_config_dir = tmpdir.mkdir("test") fake_custom_config_file = fake_config_dir.mkdir("logwatch.d").join("custom.cfg") fake_custom_config_file.write("blub") @@ -107,8 +134,9 @@ def test_get_config_files(tmpdir): assert paths == ['/logwatch.cfg', '/logwatch.d/custom.cfg'] -def test_read_config(tmpdir): +def test_read_config(agent_plugin_as_module, tmpdir): """Fakes a single logwatch config files and checks if the agent plugin reads the configuration appropriately.""" + mk_logwatch = agent_plugin_as_module # setup fake_config_file = tmpdir.mkdir("test").join("logwatch.cfg") fake_config_file.write(LOGWATCH_CONFIG_CONTENT) @@ -162,11 +190,12 @@ def test_read_config(tmpdir): ("", True, "/path/to/config/logwatch.state.local"), ("", False, "/path/to/config/logwatch.state"), ]) -def test_get_status_filename(env_var, istty, statusfile, monkeypatch, mocker): +def test_get_status_filename(agent_plugin_as_module, env_var, istty, statusfile, monkeypatch, mocker): """ May not be executed with pytest option -s set. pytest stdout redirection would colide with stdout mock. """ + mk_logwatch = agent_plugin_as_module monkeypatch.setenv("REMOTE", env_var) monkeypatch.setenv("MK_VARDIR", '/path/to/config') stdout_mock = mocker.patch("mk_logwatch.sys.stdout") @@ -182,8 +211,9 @@ def test_get_status_filename(env_var, istty, statusfile, monkeypatch, mocker): assert status_filename == statusfile -def test_read_status(tmpdir): +def test_read_status(agent_plugin_as_module, tmpdir): # setup + mk_logwatch = agent_plugin_as_module fake_status_file = tmpdir.mkdir("test").join("logwatch.state.another_cluster") fake_status_file.write("""/var/log/messages|7767698|32455445 /var/test/x12134.log|12345|32444355""") @@ -198,7 +228,8 @@ def test_read_status(tmpdir): } -def test_save_status(tmpdir): +def test_save_status(agent_plugin_as_module, tmpdir): + mk_logwatch = agent_plugin_as_module fake_status_file = tmpdir.mkdir("test").join("logwatch.state.another_cluster") fake_status_file.write("") file_path = str(fake_status_file) @@ -213,12 +244,15 @@ def test_save_status(tmpdir): @pytest.mark.parametrize("pattern_suffix, file_suffixes", [ - ("/*", ["/file.log", "/hard_linked_file_a.log", "/hard_linked_file_b.log", "/symlinked_file.log"]), - ("/**", ["/file.log", "/hard_linked_file_a.log", "/hard_linked_file_b.log", "/symlinked_file.log"]), + ("/*", + ["/file.log", "/hard_linked_file_a.log", "/hard_linked_file_b.log", "/symlinked_file.log"]), + ("/**", + ["/file.log", "/hard_linked_file_a.log", "/hard_linked_file_b.log", "/symlinked_file.log"]), ("/subdir/*", ["/subdir/another_symlinked_file.log"]), ("/symlink_to_dir/*", ["/symlink_to_dir/yet_another_file.log"]), ]) -def test_find_matching_logfiles(fake_filesystem, pattern_suffix, file_suffixes): +def test_find_matching_logfiles(agent_plugin_as_module, fake_filesystem, pattern_suffix, file_suffixes): + mk_logwatch = agent_plugin_as_module fake_fs_path = str(fake_filesystem) files = mk_logwatch.find_matching_logfiles(fake_fs_path + pattern_suffix) assert sorted(files) == [fake_fs_path + fs for fs in file_suffixes]

5 years, 7 months

ucs_bladecenter_topsystem: add regression test

by Florian Kromer

Module: check_mk Branch: master Commit: c015878d9426db890de8f29b7a7787b99a696a8f URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=c015878d9426db… Author: Florian Kromer <fk(a)mathias-kettner.de> Date: Mon Feb 11 17:33:58 2019 +0100 ucs_bladecenter_topsystem: add regression test Change-Id: I5cde32e83447d8a2aa10ba17f61a94915843557d --- .../generictests/datasets/ucs_bladecenter_topsystem.py | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/tests/unit/checks/generictests/datasets/ucs_bladecenter_topsystem.py b/tests/unit/checks/generictests/datasets/ucs_bladecenter_topsystem.py new file mode 100644 index 0000000..3add50c --- /dev/null +++ b/tests/unit/checks/generictests/datasets/ucs_bladecenter_topsystem.py @@ -0,0 +1,15 @@ +checkname = 'ucs_bladecenter_topsystem' + +info = [[ + 'topSystem', 'Address 192.168.1.1', 'CurrentTime 2015-07-15T16:40:27.600', 'Ipv6Addr ::', + 'Mode cluster', 'Name svie23ucsfi01', 'SystemUpTime 125:16:10:53' +]] + +discovery = {'': [(None, None)]} + +checks = { + '': [(None, {}, [(0, 'Address: 192.168.1.1', []), (0, 'CurrentTime: 2015-07-15T16:40:27.600', + []), (0, 'Ipv6Addr: ::', []), + (0, 'Mode: cluster', []), (0, 'Name: svie23ucsfi01', []), + (0, 'SystemUpTime: 125:16:10:53', [])])] +}

5 years, 7 months

7085 FIX Fixed parsing of special syslog messages which don' t contain a host name

by Lars Michelsen

Module: check_mk Branch: master Commit: c7ed23bc334509fa82e3e5e04a6a1d9f9fe33e93 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=c7ed23bc334509… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Wed Feb 13 10:44:12 2019 +0100 7085 FIX Fixed parsing of special syslog messages which don't contain a host name The Event Console is now able to process syslog messages that don't contain the host name field. An example for such a message is this one: C+: Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET</tt> C-: In previous versions messages like this resulted in log messages like this in the event console log (var/log/mkeventd.log): C+: 2019-02-13 09:41:07,338 [40] [cmk.mkeventd.EventServer] Got non-syslog message "Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET" (need more than 1 value to unpack) Traceback (most recent call last): File "/omd/sites/ggmcmpp1/lib/python/cmk/ec/main.py", line 2916, in create_event_from_line event.update(self.parse_syslog_info(rest)) File "/omd/sites/ggmcmpp1/lib/python/cmk/ec/main.py", line 2667, in parse_syslog_info tag, message = line.split(": ", 1) ValueError: need more than 1 value to unpack 2019-02-13 09:41:07,338 [20] [cmk.mkeventd.EventServer] Parsed message: application: core_host: facility: 1 host: host_in_downtime: False ipaddress: 1.23.45.67 pid: 0 priority: 0 text: Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41 :02 CET time: 1550047267.34 C-: A fallback event was created that had no syslog fields set and contained the whole syslog message in the text field. If you have EC rules matching on this fallback event, you will have to change these rules to match the parsed event fields. CMK-1213 Change-Id: I76fee896bbba5d6ee3f9246d5b930c762325b234 --- .werks/7085 | 64 +++++++++++++++++++++++++++++++++ cmk/ec/main.py | 20 +++++++++-- tests/unit/cmk/ec/test_event_creator.py | 15 ++++++++ 3 files changed, 96 insertions(+), 3 deletions(-) diff --git a/.werks/7085 b/.werks/7085 new file mode 100644 index 0000000..097d811 --- /dev/null +++ b/.werks/7085 @@ -0,0 +1,64 @@ +Title: Fixed parsing of special syslog messages which don't contain a host name +Level: 1 +Component: ec +Compatible: incomp +Edition: cre +Version: 1.6.0i1 +Date: 1550050614 +Class: fix + +The Event Console is now able to process syslog messages that don't contain the +host name field. An example for such a message is this one: + +C+: +Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET</tt> +C-: + +In previous versions messages like this resulted in log messages like this in +the event console log (var/log/mkeventd.log): + +C+: +2019-02-13 09:41:07,338 [40] [cmk.mkeventd.EventServer] Got non-syslog message "Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET" (need more than 1 value to unpack) +Traceback (most recent call last): + File "/omd/sites/ggmcmpp1/lib/python/cmk/ec/main.py", line 2916, in create_event_from_line + event.update(self.parse_syslog_info(rest)) + File "/omd/sites/ggmcmpp1/lib/python/cmk/ec/main.py", line 2667, in parse_syslog_info + tag, message = line.split(": ", 1) +ValueError: need more than 1 value to unpack +2019-02-13 09:41:07,338 [20] [cmk.mkeventd.EventServer] Parsed message: + application: + core_host: + facility: 1 + host: + host_in_downtime: False + ipaddress: 1.23.45.67 + pid: 0 + priority: 0 + text: Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41 +:02 CET + time: 1550047267.34 +C-: + +A fallback event was created that had no syslog fields set and contained the +whole unparsed syslog message in the text field. If you have EC rules matching +on this fallback event, you will have to change these rules to match the parsed +event fields. + +Now that the parsing has been added, events created by such a syslog message now +have the fields set as follows for our example: + +C+: +application: pfsp +core_host: +facility: 1 +host: 127.0.0.1 +host_in_downtime: False +ipaddress: 127.0.0.1' +pid: 0 +priority: 5 +text: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET +time: 1550043667.0 +C-: + +Please note that the EC uses the sender IP addresse of the syslog message to populate the host field. + diff --git a/cmk/ec/main.py b/cmk/ec/main.py index 8056204..d5d2560 100644 --- a/cmk/ec/main.py +++ b/cmk/ec/main.py @@ -1987,6 +1987,9 @@ class EventCreator(object): # Variant 1: plain syslog message without priority/facility: # May 26 13:45:01 Klapprechner CRON[8046]: message.... + # Variant 1a: plain syslog message without priority/facility/host: + # May 26 13:45:01 Klapprechner CRON[8046]: message.... + # Variant 2: syslog message including facility (RFC 3164) # <78>May 26 13:45:01 Klapprechner CRON[8046]: message.... @@ -2035,7 +2038,7 @@ class EventCreator(object): event["facility"] = prio >> 3 event["priority"] = prio & 7 - # Variant 1 + # Variant 1,1a else: event["facility"] = 1 # user event["priority"] = 5 # notice @@ -2091,9 +2094,20 @@ class EventCreator(object): event["text"] = line event['time'] = time.mktime(time.strptime(time_part, '%Y %b %d %H:%M:%S')) - # Variant 1,2,4 + # Variant 1,1a,2,4 else: - month_name, day, timeofday, host, rest = line.split(None, 4) + month_name, day, timeofday, rest = line.split(None, 3) + + # Special handling for variant 1a. Detect whether or not host + # is a hostname or syslog tag + host, tmp_rest = rest.split(None, 1) + if host.endswith(":"): + # There is no host information sent, use the source address as "host" + host = address[0] + else: + # Use the extracted host and continue with the remaining message text + rest = tmp_rest + event["host"] = host # Variant 4 diff --git a/tests/unit/cmk/ec/test_event_creator.py b/tests/unit/cmk/ec/test_event_creator.py index a895cce..16b82e2 100644 --- a/tests/unit/cmk/ec/test_event_creator.py +++ b/tests/unit/cmk/ec/test_event_creator.py @@ -35,6 +35,21 @@ def event_creator(): }, ), ( + "Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET", + { + 'application': 'pfsp', + 'core_host': '', + 'facility': 1, + 'host': '127.0.0.1', + 'host_in_downtime': False, + 'ipaddress': '127.0.0.1', + 'pid': 0, + 'priority': 5, + 'text': 'The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET', + 'time': 1550043667.0 + }, + ), + ( # Variant 2: syslog message including facility (RFC 3164) "<78>May 26 13:45:01 Klapprechner CRON[8046]: message", {

5 years, 7 months

Made helper functions private

by Lars Michelsen

Module: check_mk Branch: master Commit: 3bedac95dba3fec5f6742aaa26bd1cb290c51b71 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=3bedac95dba3fe… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Wed Feb 13 10:46:24 2019 +0100 Made helper functions private Change-Id: Ic72e6cd77411822c63211f50dcdd676d5734af75 --- cmk/ec/main.py | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/cmk/ec/main.py b/cmk/ec/main.py index d5d2560..2fe1eee 100644 --- a/cmk/ec/main.py +++ b/cmk/ec/main.py @@ -2051,11 +2051,11 @@ class EventCreator(object): if len(detail_tokens) > 1: event["sl"] = int(detail_tokens[1]) event['time'] = float(timestamp[1:]) - event.update(self.parse_syslog_info(line)) + event.update(self._parse_syslog_info(line)) # Variant 3 elif line.startswith("@"): - event.update(self.parse_monitoring_info(line)) + event.update(self._parse_monitoring_info(line)) # Variant 5 elif len(line) > 24 and line[10] == 'T': @@ -2064,22 +2064,22 @@ class EventCreator(object): # got a good solutuion for this. rfc3339_part, event['host'], line = line.split(' ', 2) event['time'] = time.mktime(time.strptime(rfc3339_part[:19], '%Y-%m-%dT%H:%M:%S')) - event.update(self.parse_syslog_info(line)) + event.update(self._parse_syslog_info(line)) # Variant 9 elif len(line) > 24 and line[12] == "T": - event.update(self.parse_rfc5424_syslog_info(line)) + event.update(self._parse_rfc5424_syslog_info(line)) # Variant 8 elif line[10] == '-' and line[19] == ' ': event['host'] = line.split(' ')[1] event['time'] = time.mktime(time.strptime(line.split(' ')[0], '%Y:%m:%d-%H:%M:%S')) rest = " ".join(line.split(' ')[2:]) - event.update(self.parse_syslog_info(rest)) + event.update(self._parse_syslog_info(rest)) # Variant 6 elif len(line.split(': ', 1)[0].split(' ')) == 1: - event.update(self.parse_syslog_info(line)) + event.update(self._parse_syslog_info(line)) # There is no datetime information in the message, use current time event['time'] = time.time() # There is no host information, use the provided address @@ -2112,11 +2112,11 @@ class EventCreator(object): # Variant 4 if rest.startswith("@"): - event.update(self.parse_monitoring_info(rest)) + event.update(self._parse_monitoring_info(rest)) # Variant 1, 2 else: - event.update(self.parse_syslog_info(rest)) + event.update(self._parse_syslog_info(rest)) month = EventServer.month_names[month_name] day = int(day) @@ -2162,7 +2162,7 @@ class EventCreator(object): return event - def parse_rfc5424_syslog_info(self, line): + def _parse_rfc5424_syslog_info(self, line): event = {} (_unused_version, timestamp, hostname, app_name, procid, _unused_msgid, rest) = line.split( @@ -2198,7 +2198,7 @@ class EventCreator(object): return event - def parse_syslog_info(self, line): + def _parse_syslog_info(self, line): event = {} # Replaced ":" by ": " here to make tags with ":" possible. This # is needed to process logs generated by windows agent logfiles @@ -2217,7 +2217,7 @@ class EventCreator(object): event["pid"] = pid return event - def parse_monitoring_info(self, line): + def _parse_monitoring_info(self, line): event = {} # line starts with '@' if line[11] == ';':

5 years, 7 months

Moved WATO specific group loading/saving to correct module

by Lars Michelsen

Module: check_mk Branch: master Commit: c2ac79f53e4abbc3584492e329ac5efbc2924c19 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=c2ac79f53e4abb… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Wed Feb 13 21:03:35 2019 +0100 Moved WATO specific group loading/saving to correct module To clarify the purpose of these functions the load_group_information stuff has been moved to the cmk.gui.watolib.groups module. Change-Id: Icd4a97a5c86338657d27153f6e69edb48890b34b --- cmk/gui/plugins/userdb/hook_auth.py | 3 +- cmk/gui/plugins/userdb/ldap_connector.py | 4 +- cmk/gui/plugins/wato/bi.py | 3 +- cmk/gui/plugins/wato/check_mk_configuration.py | 3 +- cmk/gui/plugins/wato/utils/__init__.py | 27 +++- cmk/gui/plugins/webapi/webapi.py | 3 +- cmk/gui/userdb.py | 64 -------- cmk/gui/wato/pages/folders.py | 6 +- cmk/gui/wato/pages/groups.py | 5 +- cmk/gui/wato/pages/password_store.py | 5 +- cmk/gui/wato/pages/users.py | 7 +- cmk/gui/watolib/groups.py | 195 ++++++++++++++++++++++++- cmk/gui/watolib/host_attributes.py | 145 +----------------- 13 files changed, 237 insertions(+), 233 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=c2ac79f53e…

5 years, 7 months

Cleanup WATO group loading functions

by Lars Michelsen

Module: check_mk Branch: master Commit: 55c784573959fb5060423d3601b1fb7f90af1547 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=55c784573959fb… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Thu Feb 14 07:42:03 2019 +0100 Cleanup WATO group loading functions The loading of groups has been splitted into group type specific functions. The fact that all types of groups are currently being loaded from a single file should be only locally known in the cmk.gui.watolib.groups module before we can clean this up more. Change-Id: Ie4b36a27bb7ab8383e9c980b021d370668b10b71 --- cmk/gui/plugins/userdb/hook_auth.py | 4 +- cmk/gui/plugins/userdb/ldap_connector.py | 4 +- cmk/gui/plugins/wato/__init__.py | 4 +- cmk/gui/plugins/wato/bi.py | 8 +-- cmk/gui/plugins/wato/check_mk_configuration.py | 26 +++------ cmk/gui/plugins/wato/utils/__init__.py | 77 +++++++++++++++++--------- cmk/gui/plugins/webapi/webapi.py | 22 ++++++-- cmk/gui/wato/__init__.py | 1 - cmk/gui/wato/mkeventd.py | 7 +-- cmk/gui/wato/pages/folders.py | 6 +- cmk/gui/wato/pages/groups.py | 50 ++++++++++++----- cmk/gui/wato/pages/notifications.py | 4 +- cmk/gui/wato/pages/password_store.py | 6 +- cmk/gui/wato/pages/users.py | 8 +-- cmk/gui/watolib/groups.py | 24 ++++++-- tests/unit/cmk/gui/watolib/test_groups.py | 27 ++++++++- tests/unit/cmk/gui/watolib/test_rulespecs.py | 8 +-- 17 files changed, 186 insertions(+), 100 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=55c7845739…

5 years, 7 months

Add tests to verify the current group loading logic

by Lars Michelsen

Module: check_mk Branch: master Commit: 60f41cf39b16bc6da72d95aef93556f4a7b8de98 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=60f41cf39b16bc… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Wed Feb 13 21:51:35 2019 +0100 Add tests to verify the current group loading logic Change-Id: Ieed3740a650ac355e4af3810030161715e710238 --- tests/unit/cmk/gui/watolib/test_groups.py | 82 +++++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) diff --git a/tests/unit/cmk/gui/watolib/test_groups.py b/tests/unit/cmk/gui/watolib/test_groups.py new file mode 100644 index 0000000..b1cd738 --- /dev/null +++ b/tests/unit/cmk/gui/watolib/test_groups.py @@ -0,0 +1,82 @@ +# encoding: utf-8 + +import pytest + +import cmk.utils.paths +import cmk.gui.watolib.groups as groups + + +(a)pytest.fixture(autouse=True) +def patch_config_paths(monkeypatch, tmp_path): + cmk_confd = tmp_path / "check_mk" / "conf.d" + monkeypatch.setattr(cmk.utils.paths, "check_mk_config_dir", str(cmk_confd)) + (cmk_confd / "wato").mkdir(parents=True) + + gui_confd = tmp_path / "check_mk" / "multisite.d" + monkeypatch.setattr(cmk.utils.paths, "default_config_dir", str(gui_confd.parent)) + (gui_confd / "wato").mkdir(parents=True) + + +def test_load_group_information_empty(tmp_path): + assert groups.load_group_information() == {'contact': {}, 'host': {}, 'service': {}} + + +def test_load_group_information(tmp_path): + with open(cmk.utils.paths.check_mk_config_dir + "/wato/groups.mk", "w") as f: + f.write("""# encoding: utf-8 + +if type(define_contactgroups) != dict: + define_contactgroups = {} +define_contactgroups.update({'all': u'Everything'}) + +if type(define_hostgroups) != dict: + define_hostgroups = {} +define_hostgroups.update({'all_hosts': u'All hosts :-)'}) + +if type(define_servicegroups) != dict: + define_servicegroups = {} +define_servicegroups.update({'all_services': u'All särvices'}) +""") + + with open(cmk.utils.paths.default_config_dir + "/multisite.d/wato/groups.mk", "w") as f: + f.write("""# encoding: utf-8 + +multisite_hostgroups = { + "all_hosts": { + "ding": "dong", + }, +} + +multisite_servicegroups = { + "all_services": { + "d1ng": "dong", + }, +} + +multisite_contactgroups = { + "all": { + "d!ng": "dong", + }, +} +""") + + assert groups.load_group_information() == { + 'contact': { + 'all': { + 'alias': u'Everything', + "d!ng": "dong", + } + }, + 'host': { + 'all_hosts': { + 'alias': u'All hosts :-)', + "ding": "dong", + } + }, + 'service': { + 'all_services': { + 'alias': u'All s\xe4rvices', + "d1ng": "dong", + } + }, + }

5 years, 7 months

Simplify GroupChoice valuespec

by Lars Michelsen

Module: check_mk Branch: master Commit: 88255cf7e9fd44c6b63734fbd91ae71795e7f3a9 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=88255cf7e9fd44… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Wed Feb 13 20:36:39 2019 +0100 Simplify GroupChoice valuespec The with_foreign_groups handling is not needed anymore by any call site. Change-Id: If401a15cc1b341d0dd39602dca1271ade9bc3786 --- cmk/gui/userdb.py | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/cmk/gui/userdb.py b/cmk/gui/userdb.py index 1099fe9..df0cc9d 100644 --- a/cmk/gui/userdb.py +++ b/cmk/gui/userdb.py @@ -1141,17 +1141,14 @@ def _load_gui_groups(): class GroupChoice(DualListChoice): - def __init__(self, what, with_foreign_groups=True, **kwargs): - DualListChoice.__init__(self, **kwargs) + def __init__(self, what, **kwargs): + super(GroupChoice, self).__init__(choices=self._load_groups, **kwargs) self.what = what - self._choices = lambda: self.load_groups(with_foreign_groups) - def load_groups(self, with_foreign_groups): + def _load_groups(self): all_groups = load_group_information() this_group = all_groups.get(self.what, {}) - return sorted([(k, t['alias'] and t['alias'] or k) - for (k, t) in this_group.items() - if with_foreign_groups or k in config.user.contact_groups()], + return sorted([(k, t['alias'] and t['alias'] or k) for (k, t) in this_group.items()], key=lambda x: x[1].lower())

5 years, 7 months

← Newer
1
...
22
23
24
25
26
27
28
...
46
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Checkmk git commits February 2019