[checkmk-commits] 7085 FIX Fixed parsing of special syslog messages which don' t contain a host name

Module: check_mk Branch: master Commit: c7ed23bc334509fa82e3e5e04a6a1d9f9fe33e93 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=c7ed23bc334509… Author: Lars Michelsen &lt;lm(a)mathias-kettner.de&gt; Date: Wed Feb 13 10:44:12 2019 +0100 7085 FIX Fixed parsing of special syslog messages which don't contain a host name The Event Console is now able to process syslog messages that don't contain the host name field. An example for such a message is this one: C+: Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET</tt> C-: In previous versions messages like this resulted in log messages like this in the event console log (var/log/mkeventd.log): C+: 2019-02-13 09:41:07,338 [40] [cmk.mkeventd.EventServer] Got non-syslog message "Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET" (need more than 1 value to unpack) Traceback (most recent call last): File "/omd/sites/ggmcmpp1/lib/python/cmk/ec/main.py", line 2916, in create_event_from_line event.update(self.parse_syslog_info(rest)) File "/omd/sites/ggmcmpp1/lib/python/cmk/ec/main.py", line 2667, in parse_syslog_info tag, message = line.split(": ", 1) ValueError: need more than 1 value to unpack 2019-02-13 09:41:07,338 [20] [cmk.mkeventd.EventServer] Parsed message: application: core_host: facility: 1 host: host_in_downtime: False ipaddress: 1.23.45.67 pid: 0 priority: 0 text: Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41 :02 CET time: 1550047267.34 C-: A fallback event was created that had no syslog fields set and contained the whole syslog message in the text field. If you have EC rules matching on this fallback event, you will have to change these rules to match the parsed event fields. CMK-1213 Change-Id: I76fee896bbba5d6ee3f9246d5b930c762325b234 --- .werks/7085 | 64 +++++++++++++++++++++++++++++++++ cmk/ec/main.py | 20 +++++++++-- tests/unit/cmk/ec/test_event_creator.py | 15 ++++++++ 3 files changed, 96 insertions(+), 3 deletions(-) diff --git a/.werks/7085 b/.werks/7085 new file mode 100644 index 0000000..097d811 --- /dev/null +++ b/.werks/7085 @@ -0,0 +1,64 @@ +Title: Fixed parsing of special syslog messages which don't contain a host name +Level: 1 +Component: ec +Compatible: incomp +Edition: cre +Version: 1.6.0i1 +Date: 1550050614 +Class: fix + +The Event Console is now able to process syslog messages that don't contain the +host name field. An example for such a message is this one: + +C+: +Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET</tt> +C-: + +In previous versions messages like this resulted in log messages like this in +the event console log (var/log/mkeventd.log): + +C+: +2019-02-13 09:41:07,338 [40] [cmk.mkeventd.EventServer] Got non-syslog message "Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET" (need more than 1 value to unpack) +Traceback (most recent call last): + File "/omd/sites/ggmcmpp1/lib/python/cmk/ec/main.py", line 2916, in create_event_from_line + event.update(self.parse_syslog_info(rest)) + File "/omd/sites/ggmcmpp1/lib/python/cmk/ec/main.py", line 2667, in parse_syslog_info + tag, message = line.split(": ", 1) +ValueError: need more than 1 value to unpack +2019-02-13 09:41:07,338 [20] [cmk.mkeventd.EventServer] Parsed message: + application: + core_host: + facility: 1 + host: + host_in_downtime: False + ipaddress: 1.23.45.67 + pid: 0 + priority: 0 + text: Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41 +:02 CET + time: 1550047267.34 +C-: + +A fallback event was created that had no syslog fields set and contained the +whole unparsed syslog message in the text field. If you have EC rules matching +on this fallback event, you will have to change these rules to match the parsed +event fields. + +Now that the parsing has been added, events created by such a syslog message now +have the fields set as follows for our example: + +C+: +application: pfsp +core_host: +facility: 1 +host: 127.0.0.1 +host_in_downtime: False +ipaddress: 127.0.0.1' +pid: 0 +priority: 5 +text: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET +time: 1550043667.0 +C-: + +Please note that the EC uses the sender IP addresse of the syslog message to populate the host field. + diff --git a/cmk/ec/main.py b/cmk/ec/main.py index 8056204..d5d2560 100644 --- a/cmk/ec/main.py +++ b/cmk/ec/main.py @@ -1987,6 +1987,9 @@ class EventCreator(object): # Variant 1: plain syslog message without priority/facility: # May 26 13:45:01 Klapprechner CRON[8046]: message.... + # Variant 1a: plain syslog message without priority/facility/host: + # May 26 13:45:01 Klapprechner CRON[8046]: message.... + # Variant 2: syslog message including facility (RFC 3164) # <78>May 26 13:45:01 Klapprechner CRON[8046]: message.... @@ -2035,7 +2038,7 @@ class EventCreator(object): event["facility"] = prio >> 3 event["priority"] = prio & 7 - # Variant 1 + # Variant 1,1a else: event["facility"] = 1 # user event["priority"] = 5 # notice @@ -2091,9 +2094,20 @@ class EventCreator(object): event["text"] = line event['time'] = time.mktime(time.strptime(time_part, '%Y %b %d %H:%M:%S')) - # Variant 1,2,4 + # Variant 1,1a,2,4 else: - month_name, day, timeofday, host, rest = line.split(None, 4) + month_name, day, timeofday, rest = line.split(None, 3) + + # Special handling for variant 1a. Detect whether or not host + # is a hostname or syslog tag + host, tmp_rest = rest.split(None, 1) + if host.endswith(":"): + # There is no host information sent, use the source address as "host" + host = address[0] + else: + # Use the extracted host and continue with the remaining message text + rest = tmp_rest + event["host"] = host # Variant 4 diff --git a/tests/unit/cmk/ec/test_event_creator.py b/tests/unit/cmk/ec/test_event_creator.py index a895cce..16b82e2 100644 --- a/tests/unit/cmk/ec/test_event_creator.py +++ b/tests/unit/cmk/ec/test_event_creator.py @@ -35,6 +35,21 @@ def event_creator(): }, ), ( + "Feb 13 08:41:07 pfsp: The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET", + { + 'application': 'pfsp', + 'core_host': '', + 'facility': 1, + 'host': '127.0.0.1', + 'host_in_downtime': False, + 'ipaddress': '127.0.0.1', + 'pid': 0, + 'priority': 5, + 'text': 'The configuration was changed on leader blatldc1-xxx to version 1.1366 by blatldc1-xxx/admin at 2019-02-13 09:41:02 CET', + 'time': 1550043667.0 + }, + ), + ( # Variant 2: syslog message including facility (RFC 3164) "<78>May 26 13:45:01 Klapprechner CRON[8046]: message", {