Checkmk git commits March 2019

checkmk-commits@lists.checkmk.com

17 participants
451 discussions

7062 Improvement of the graphing with PNP4Nagios for various checks

by Tom Baerwinkel

Module: check_mk Branch: master Commit: 7f276d4fe9bdc3278518efd4f46ef4b02b8ea06d URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=7f276d4fe9bdc3… Author: Tom Baerwinkel <tb(a)mathias-kettner.de> Date: Fri Mar 1 14:55:00 2019 +0100 7062 Improvement of the graphing with PNP4Nagios for various checks When PNP4Nagios is used Check_MK tries to use autogenerated graph templates to paint the graphs of a check. If the capabilities for Check_MKs autogeneration of graph templates is insufficient Check_MK provides a fallback to handcrafted templates. This werk extends the capabilities of Check_MKs autogeneration of PNP templates and makes a lot of handcrafted PNP templates unnecessary. Therefore, these templates are removed. This results in an overall more pleasant and uniform graphing. PNP4Nagios is used if the raw edition is used or if the enterprise edition is used together with the Nagios core. Change-Id: I31c680f89d307e054ff2428b2065a01de484c555 --- .werks/7062 | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/.werks/7062 b/.werks/7062 new file mode 100644 index 0000000..b07a480 --- /dev/null +++ b/.werks/7062 @@ -0,0 +1,21 @@ +Title: Improvement of the graphing with PNP4Nagios for various checks +Level: 1 +Component: checks +Compatible: compat +Edition: cre +Version: 1.6.0i1 +Date: 1551447666 +Class: feature + +When PNP4Nagios is used Check_MK tries to use autogenerated graph templates +to paint the graphs of a check. If the capabilities for Check_MKs autogeneration +of graph templates is insufficient Check_MK provides a fallback to handcrafted +templates. + +This werk extends the capabilities of Check_MKs autogeneration of PNP templates +and makes a lot of handcrafted PNP templates unnecessary. Therefore, these +templates are removed. This results in an overall more pleasant and uniform +graphing. + +PNP4Nagios is used if the raw edition is used or if the enterprise edition is +used together with the Nagios core.

5 years, 4 months

Reintroduce the brocade_fcport pnp template

by Tom Baerwinkel

Module: check_mk Branch: master Commit: bcb04430188e3f484fc94bd1af48e30e5d0dacb5 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=bcb04430188e3f… Author: Tom Baerwinkel <tb(a)mathias-kettner.de> Date: Fri Mar 1 14:31:55 2019 +0100 Reintroduce the brocade_fcport pnp template It was accidentally removed in a previous commit CMK-1739 Change-Id: I4ec09aa03e3f085d4fd849f497f96eb14c819bca --- pnp-templates/check_mk-brocade_fcport.php | 143 ++++++++++++++++++++++++++++++ 1 file changed, 143 insertions(+) diff --git a/pnp-templates/check_mk-brocade_fcport.php b/pnp-templates/check_mk-brocade_fcport.php new file mode 100644 index 0000000..854225d --- /dev/null +++ b/pnp-templates/check_mk-brocade_fcport.php @@ -0,0 +1,143 @@ +<?php +# +------------------------------------------------------------------+ +# | ____ _ _ __ __ _ __ | +# | / ___| |__ ___ ___| | __ | \/ | |/ / | +# | | | | '_ \ / _ \/ __| |/ / | |\/| | ' / | +# | | |___| | | | __/ (__| < | | | | . \ | +# | \____|_| |_|\___|\___|_|\_\___|_| |_|_|\_\ | +# | | +# | Copyright Mathias Kettner 2014 mk(a)mathias-kettner.de | +# +------------------------------------------------------------------+ +# +# This file is part of Check_MK. +# The official homepage is at http://mathias-kettner.de/check_mk. +# +# check_mk is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by +# the Free Software Foundation in version 2. check_mk is distributed +# in the hope that it will be useful, but WITHOUT ANY WARRANTY; with- +# out even the implied warranty of MERCHANTABILITY or FITNESS FOR A +# PARTICULAR PURPOSE. See the GNU General Public License for more de- +# tails. You should have received a copy of the GNU General Public +# License along with GNU Make; see the file COPYING. If not, write +# to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, +# Boston, MA 02110-1301 USA. + + +# The number of data source various due to different +# settings (such as averaging). We rather work with names +# than with numbers. +$RRD = array(); +foreach ($NAME as $i => $n) { + $RRD[$n] = "$RRDFILE[$i]:$DS[$i]:MAX"; + $RRDAVG[$n] = "$RRDFILE[$i]:$DS[$i]:AVERAGE"; + $WARN[$n] = $WARN[$i]; + $CRIT[$n] = $CRIT[$i]; + $MIN[$n] = $MIN[$i]; + $MAX[$n] = $MAX[$i]; +} + + +# 1. GRAPH: THROUGHPUT IN MB/s + +$ds_name[1] = 'Traffic'; +$opt[1] = "--vertical-label \"MByte/sec\" -X0 -b 1024 --title \"Traffic for $hostname / $servicedesc\" "; + +$def[1] = "" + . "HRULE:0#c0c0c0 " + . "DEF:in=$RRD[in] " + . "DEF:out=$RRD[out] " + . "CDEF:inmb=in,1048576,/ " + . "CDEF:outmb=out,1048576,/ " + . "DEF:inavg=$RRDAVG[in] " + . "DEF:outavg=$RRDAVG[out] " + . "CDEF:inmbavg=inavg,1048576,/ " + . "CDEF:outmbavg=outavg,1048576,/ " + . "AREA:inmb#60a020:\"in \" " + . "GPRINT:inmb:LAST:\"%5.3lf MB/s last\" " + . "GPRINT:inmbavg:AVERAGE:\"%5.3lf MB/s avg\" " + . "GPRINT:inmb:MAX:\"%5.3lf MB/s max\\n\" " + . "CDEF:out_draw=outmb,-1,* " + . "AREA:out_draw#2060a0:\"out \" " + . "GPRINT:outmb:LAST:\"%5.3lf MB/s last\" " + . "GPRINT:outmbavg:AVERAGE:\"%5.3lf MB/s avg\" " + . "GPRINT:outmb:MAX:\"%5.3lf MB/s max\\n\" " + ; + +if (isset($RRD['in_avg'])) { +$def[1] .= "" + . "DEF:inaverage=$RRD[in_avg] " + . "DEF:outaverage=$RRD[out_avg] " + . "CDEF:inaveragemb=inaverage,1048576,/ " + . "CDEF:outaveragemb=outaverage,1048576,/ " + . "DEF:inaverage_avg=$RRDAVG[in_avg] " + . "DEF:outaverage_avg=$RRDAVG[out_avg] " + . "CDEF:inaveragemb_avg=inaverage_avg,1048576,/ " + . "CDEF:outaveragemb_avg=outaverage_avg,1048576,/ " + . "CDEF:outaveragemb_draw=outaverage,-1048576,/ " + . "LINE:inaveragemb_avg#a0d040:\"in (avg) \" " + . "GPRINT:inaveragemb:LAST:\"%5.3lf MB/s last\" " + . "GPRINT:inaveragemb_avg:AVERAGE:\"%5.3lf MB/s avg\" " + . "GPRINT:inaveragemb:MAX:\"%5.3lf MB/s max\\n\" " + . "LINE:outaveragemb_draw#40a0d0:\"out (avg)\" " + . "GPRINT:outaveragemb:LAST:\"%5.3lf MB/s last\" " + . "GPRINT:outaveragemb_avg:AVERAGE:\"%5.3lf MB/s avg\" " + . "GPRINT:outaveragemb:MAX:\"%5.3lf MB/s max\\n\" " + ; +} + +if ($WARN['in']) { + $def[1] .= "HRULE:$WARN[in]#ffff00:\"Warning (in)\" "; + $def[1] .= "HRULE:-$WARN[out]#ffff00:\"Warning (out)\" "; +} +if ($CRIT['in']) { + $def[1] .= "HRULE:$CRIT[in]#ff0000:\"Critical (in)\" "; + $def[1] .= "HRULE:-$CRIT[out]#ff0000:\"Critical (out)\" "; +} +if ($MAX['in']) { + $speedmb = $MAX['in'] / 1048576.0; + $speedtxt = sprintf("%.1f MB/s", $speedmb); + $def[1] .= "HRULE:$speedmb#ff80c0:\"Portspeed\: $speedtxt\" "; + $def[1] .= "HRULE:-$speedmb#ff80c0 "; + # $opt[1] .= " -u $speedmb -l -$speedmb"; +} + +# 2. GRAPH: FRAMES +$ds_name[2] = 'Frames'; +$opt[2] = "--vertical-label \"Frames/sec\" -b 1024 --title \"Frames per second\" "; +$def[2] = "" + . "HRULE:0#c0c0c0 " + . "DEF:in=$RRD[rxframes] " + . "DEF:out=$RRD[txframes] " + . "DEF:inavg=$RRDAVG[rxframes] " + . "DEF:outavg=$RRDAVG[txframes] " + . "AREA:in#a0d040:\"in \" " + . "GPRINT:in:LAST:\"%5.1lf/s last\" " + . "GPRINT:inavg:AVERAGE:\"%5.1lf/s avg\" " + . "GPRINT:in:MAX:\"%5.1lf/s max\\n\" " + . "CDEF:out_draw=out,-1,* " + . "AREA:out_draw#40a0d0:\"out \" " + . "GPRINT:out:LAST:\"%5.1lf/s last\" " + . "GPRINT:outavg:AVERAGE:\"%5.1lf/s avgargs\" " + . "GPRINT:out:MAX:\"%5.1lf/s max\\n\" " + ; + +# 3. GRAPH: ERRORS + +$ds_name[3] = 'Error counter'; +$opt[3] = "--vertical-label \"Error counter\" --title \"Problems\" "; +$def[3] = "" + . "DEF:rxcrcs=$RRD[rxcrcs] " + . "DEF:notxcredits=$RRD[notxcredits] " + . "DEF:rxencoutframes=$RRD[rxencoutframes] " + . "DEF:c3discards=$RRD[c3discards] " + . "LINE1:rxcrcs#c00000:\"CRC Errors \" " + . "GPRINT:rxcrcs:LAST:\"last\: %4.0lf/s \" " + . "LINE1:notxcredits#ff8000:\"No Tx Credits \" " + . "GPRINT:notxcredits:LAST:\"last\: %4.0lf/s\\n\" " + . "LINE1:rxencoutframes#ff0080:\"ENC-Out Frames \" " + . "GPRINT:rxencoutframes:LAST:\"last\: %4.0lf/s \" " + . "LINE1:c3discards#ffa0a0:\"Class 3 Discards\" " + . "GPRINT:c3discards:LAST:\"last\: %4.0lf/s\\n\" " + ; +?>

5 years, 4 months

7105 FIX Check API: Make usage of dsname, unit,

by Simon Betz

human_readable_func and infoname in check_levels clearer Message-ID: <5c7d2885.rNRJY8l1hK6VbNFf%si(a)mathias-kettner.de> User-Agent: Heirloom mailx 12.5 6/20/10 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Module: check_mk Branch: master Commit: 29979d693a91890b95e192fc4f39629b5018d56c URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=29979d693a9189… Author: Simon Betz <si(a)mathias-kettner.de> Date: Fri Mar 1 11:59:22 2019 +0100 7105 FIX Check API: Make usage of dsname, unit, human_readable_func and infoname in check_levels clearer CMK-1771 Change-Id: I321d030d58ab6350f81951c5f2670f99a52cd8a6 --- .werks/7105 | 39 ++++++++++ checks/atto_fibrebridge_fcport | 8 +- checks/aws_elb | 7 +- checks/aws_s3_requests | 6 +- checks/brocade_sfp | 14 ++-- checks/cpu_util.include | 44 ++++++++--- checks/ddn_s2a_statsdelay | 26 +++---- checks/diskstat.include | 6 +- checks/esx_vsphere_vm | 6 +- checks/ibm_svc_systemstats | 2 +- checks/if.include | 5 +- checks/inotify | 4 +- checks/k8s_pods_cm | 25 ++++--- checks/k8s_stats | 11 ++- checks/mem | 11 ++- checks/safenet_hsm | 2 +- checks/safenet_ntls | 10 ++- checks/tinkerforge | 2 +- checks/ucd_cpu_util | 13 +++- checks/ucs_c_rack_server_util | 9 +-- checks/varnish | 4 +- checks/vms_cpu | 13 +++- cmk_base/check_api.py | 61 +++++++++------ .../generictests/datasets/azure_databases.py | 2 +- .../esx_vsphere_hostsystem_esx51_regression.py | 2 +- .../datasets/esx_vsphere_hostsystem_regression.py | 2 +- .../unit/checks/generictests/datasets/k8s_stats.py | 16 ++-- .../datasets/lparstat_aix_1_regression.py | 8 +- .../datasets/lparstat_aix_2_regression.py | 8 +- .../datasets/lparstat_aix_regression.py | 24 +++--- .../datasets/ucs_c_rack_server_util.py | 16 ++-- .../datasets/varnish_new_format_regression.py | 38 +++++----- .../datasets/varnish_old_format_regression.py | 86 +++++++++++----------- .../generictests/datasets/vms_cpu_regression.py | 16 ++-- tests/unit/checks/test_kernel_check.py | 14 ++-- tests/unit/checks/test_lnx_if.py | 4 +- tests/unit/checks/test_statgrab_cpu_check.py | 16 ++-- tests/unit/cmk_base/test_check_api.py | 31 +++++--- 38 files changed, 367 insertions(+), 244 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=29979d693a…

5 years, 4 months

omdlib: Moved backup logic to dedicated module

by Lars Michelsen

Module: check_mk Branch: master Commit: 12fa8887de49a1e73b452badc0df882cfe4a4137 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=12fa8887de49a1… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Sat Mar 2 19:10:55 2019 +0100 omdlib: Moved backup logic to dedicated module Change-Id: Ic92bd6d69d55c71400f03691ba77440d61ad435b --- omd/packages/omd/omdlib/backup.py | 214 ++++++++++++++++++++++++++++++++++++ omd/packages/omd/omdlib/main.py | 224 ++++---------------------------------- 2 files changed, 235 insertions(+), 203 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=12fa8887de…

5 years, 4 months

Move some more backup related code to omdlib.backup

by Lars Michelsen

Module: check_mk Branch: master Commit: 85cfa9084c5a3391977e91b7c9bb70829319018f URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=85cfa9084c5a33… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Sat Mar 2 19:49:49 2019 +0100 Move some more backup related code to omdlib.backup Change-Id: Ib42c952853554170c045c50e82847ef6efca322a --- omd/packages/omd/omdlib/backup.py | 23 +++++++++++++++++++++++ omd/packages/omd/omdlib/main.py | 15 +++------------ 2 files changed, 26 insertions(+), 12 deletions(-) diff --git a/omd/packages/omd/omdlib/backup.py b/omd/packages/omd/omdlib/backup.py index 661c728..70ca46b 100644 --- a/omd/packages/omd/omdlib/backup.py +++ b/omd/packages/omd/omdlib/backup.py @@ -30,6 +30,7 @@ import errno import socket import tarfile import fnmatch +from typing import Tuple # pylint: disable=unused-import def backup_site_to_tarfile(site, fh, mode, options, verbose): @@ -212,3 +213,25 @@ class BackupTarFile(tarfile.TarFile): if self._sock: self._resume_all_rrds() self._sock.close() + + +def get_site_and_version_from_backup(tar): + # type: (tarfile.TarFile) -> Tuple[str, str] + """Get the first file of the tar archive. Expecting <site>/version symlink + for validation reasons.""" + site_tarinfo = tar.next() + if site_tarinfo is None: + raise Exception("Failed to detect version of backed up site.") + + try: + sitename, version_name = site_tarinfo.name.split("/", 1) + except ValueError: + raise Exception("Failed to detect version of backed up site. " + "Maybe the backup is from an incompatible version.") + + if version_name == "version": + version = site_tarinfo.linkname.split('/')[-1] + else: + raise Exception("Failed to detect version of backed up site.") + + return sitename, version diff --git a/omd/packages/omd/omdlib/main.py b/omd/packages/omd/omdlib/main.py index a68023d..a860480 100644 --- a/omd/packages/omd/omdlib/main.py +++ b/omd/packages/omd/omdlib/main.py @@ -3610,19 +3610,10 @@ def main_restore(site, args, options=None): except tarfile.ReadError as e: bail_out("Failed to open the backup: %s" % e) - # Get the first file of the tar archive. Expecting <site>/version symlink - # for validation reasons. - site_tarinfo = tar.next() try: - sitename, version_name = site_tarinfo.name.split("/", 1) - except ValueError: - bail_out("Failed to detect version of backed up site. " - "Maybe the backup is from an incompatible version.") - - if version_name == "version": - version = site_tarinfo.linkname.split('/')[-1] - else: - bail_out("Failed to detect version of backed up site.") + sitename, version = omdlib.backup.get_site_and_version_from_backup(tar) + except Exception as e: + bail_out(e) if not version_exists(version): bail_out("You need to have version %s installed to be able to restore "

5 years, 4 months

The code was not race condition safe. Added a try block to not fail for OSError 2

by Alex Zurhake

Module: check_mk Branch: master Commit: 10a7937409e01cf13b52ca4e9bf299111da43b45 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=10a7937409e01c… Author: Alex Zurhake <az(a)mathias-kettner.de> Date: Mon Mar 4 12:14:41 2019 +0100 The code was not race condition safe. Added a try block to not fail for OSError 2 Change-Id: Ie99d83df49e8fb3f2a541e1a69c49da7c9bc4b06 --- .../integration/notifications/test_unreachable_notifications.py | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/tests/integration/notifications/test_unreachable_notifications.py b/tests/integration/notifications/test_unreachable_notifications.py index cc440c9..8cd4623 100644 --- a/tests/integration/notifications/test_unreachable_notifications.py +++ b/tests/integration/notifications/test_unreachable_notifications.py @@ -117,8 +117,13 @@ def initial_state(site, scenario): site.live.command("[%d] ROTATE_LOGFILE" % time.time()) def rotated_log(): - return os.path.exists(site.path("var/check_mk/core/history")) \ - and inode_before != os.stat(site.path("var/check_mk/core/history")).st_ino + try: + return inode_before != os.stat(site.path("var/check_mk/core/history")).st_ino + except OSError as e: + if e.errno == 2: + return False + else: + raise e wait_until(rotated_log, timeout=10) else:

5 years, 4 months

7224 FIX Backup/Restore: Fix vanishing files terminating a backup

by Lars Michelsen

Module: check_mk Branch: master Commit: c647025b6b00ac08f9be79daa694913ed6b4fcca URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=c647025b6b00ac… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Mon Mar 4 09:01:39 2019 +0100 7224 FIX Backup/Restore: Fix vanishing files terminating a backup The "omd backup" mechanism recursively backups all site related files of a Check_MK site. When files in a site directory, that is currently being processed vanish, this could lead to failed backups with errors like this: C+: Site backup failed: Traceback (most recent call last): File "/omd/versions/1.5.0p8.cee/bin/omd", line 4553, in command_function(args, command_options) File "/omd/versions/1.5.0p8.cee/bin/omd", line 3711, in main_backup backup_site_to_tarfile(fh, tar_mode, options) File "/omd/versions/1.5.0p8.cee/bin/omd", line 3686, in backup_site_to_tarfile backup_site_files_to_tarfile(tar, options) File "/omd/versions/1.5.0p8.cee/bin/omd", line 3556, in backup_site_files_to_tarfile tar.add(g_sitedir, g_sitename, exclude=filter_files) File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2032, in add recursive, exclude, filter) File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2032, in add recursive, exclude, filter) File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2032, in add recursive, exclude, filter) File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2032, in add recursive, exclude, filter) File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2032, in add recursive, exclude, filter) File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2009, in add tarinfo = self.gettarinfo(name, arcname) File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 1881, in gettarinfo statres = os.lstat(name) OSError: [Errno 2] No such file or directory: '/omd/sites/xyz/var/pnp4nagios/perfdata/hastenichgesehn/Interface_BG4_2OG_VLAN_421.xml.new' C-: After this change the backup continues in such a situation excluding the just vanished file. Change-Id: Ic76bc983ebd221687e2f249bf6bcb3389f348aa2 --- .werks/7224 | 43 ++++++++++++++++++++ omd/packages/omd/omdlib/backup.py | 18 ++++++++- tests/unit/omdlib/test_backup.py | 85 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 144 insertions(+), 2 deletions(-) diff --git a/.werks/7224 b/.werks/7224 new file mode 100644 index 0000000..b067d0e --- /dev/null +++ b/.werks/7224 @@ -0,0 +1,43 @@ +Title: Backup/Restore: Fix vanishing files terminating a backup +Level: 2 +Component: omd +Compatible: compat +Edition: cre +Version: 1.6.0i1 +Date: 1551685160 +Class: fix + +The "omd backup" mechanism recursively backups all site related +files of a Check_MK site. When files in a site directory, that +is currently being processed vanish, this could lead to failed +backups with errors like this: + +C+: +Site backup failed: Traceback (most recent call last): +File "/omd/versions/1.5.0p8.cee/bin/omd", line 4553, in +command_function(args, command_options) +File "/omd/versions/1.5.0p8.cee/bin/omd", line 3711, in main_backup +backup_site_to_tarfile(fh, tar_mode, options) +File "/omd/versions/1.5.0p8.cee/bin/omd", line 3686, in backup_site_to_tarfile +backup_site_files_to_tarfile(tar, options) +File "/omd/versions/1.5.0p8.cee/bin/omd", line 3556, in backup_site_files_to_tarfile +tar.add(g_sitedir, g_sitename, exclude=filter_files) +File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2032, in add +recursive, exclude, filter) +File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2032, in add +recursive, exclude, filter) +File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2032, in add +recursive, exclude, filter) +File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2032, in add +recursive, exclude, filter) +File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2032, in add +recursive, exclude, filter) +File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 2009, in add +tarinfo = self.gettarinfo(name, arcname) +File "/omd/versions/1.5.0p8.cee/lib/python2.7/tarfile.py", line 1881, in gettarinfo +statres = os.lstat(name) +OSError: [Errno 2] No such file or directory: '/omd/sites/xyz/var/pnp4nagios/perfdata/hastenichgesehn/Interface_BG4_2OG_VLAN_421.xml.new' +C-: + +After this change the backup continues in such a situation +excluding the just vanished file. diff --git a/omd/packages/omd/omdlib/backup.py b/omd/packages/omd/omdlib/backup.py index 70ca46b..268574c 100644 --- a/omd/packages/omd/omdlib/backup.py +++ b/omd/packages/omd/omdlib/backup.py @@ -106,7 +106,7 @@ class BackupTarFile(tarfile.TarFile): """We need to use our tarfile class here to perform a rrdcached SUSPEND/RESUME to prevent writing to individual RRDs during backups.""" - def __init__(self, *args, **kwargs): + def __init__(self, name, mode, fileobj, **kwargs): self._site = kwargs.pop("site") self._verbose = kwargs.pop("verbose") self._site_stopped = self._site.is_stopped() @@ -114,7 +114,21 @@ class BackupTarFile(tarfile.TarFile): self._sock = None self._sites_path = os.path.realpath("/omd/sites") - super(BackupTarFile, self).__init__(*args, **kwargs) + super(BackupTarFile, self).__init__(name, mode, fileobj, **kwargs) + + # We override this function to workaround an issue in the builtin add() method in + # case it is called in recursive mode and a file vanishes between the os.listdir() + # and the first file access (often seen os.lstat()) during backup. Instead of failing + # like this we want to skip those files silently during backup. + def add(self, name, arcname=None, recursive=True, exclude=None, filter=None): # pylint: disable=redefined-builtin + try: + super(BackupTarFile, self).add(name, arcname, recursive, exclude, filter) + except OSError as e: + if e.errno != errno.ENOENT or arcname == self._site.name: + raise + + if self._verbose: + sys.stdout.write("Skipping vanished file: %s\n" % arcname) def addfile(self, tarinfo, fileobj=None): # In case of a stopped site or stopped rrdcached there is no diff --git a/tests/unit/omdlib/test_backup.py b/tests/unit/omdlib/test_backup.py new file mode 100644 index 0000000..d6b6942 --- /dev/null +++ b/tests/unit/omdlib/test_backup.py @@ -0,0 +1,85 @@ +# encoding: utf-8 +# pylint: disable=redefined-outer-name + +import tarfile +import pytest # type: ignore +from pathlib2 import Path +import omdlib +import omdlib.main +import omdlib.backup + + +(a)pytest.fixture() +def site(tmp_path, monkeypatch): + monkeypatch.setattr(omdlib, "__version__", "1.3.3i7.cee") + omd_root = tmp_path / "site" + omd_root.mkdir(parents=True, exist_ok=True) + (omd_root / "version").symlink_to("../versions/%s" % omdlib.__version__) + + class UnitTestSite(omdlib.main.SiteContext): + @property + def dir(self): + return str(omd_root) + + return UnitTestSite("unit") + + +def test_backup_site_to_tarfile(site, tmp_path): + # Write some file for testing the backup procedure + with Path(site.dir + "/test123").open("w", encoding="utf-8") as f: + f.write(u"uftauftauftata") + + tar_path = tmp_path / "backup.tar" + with tar_path.open("wb") as backup_tar: + omdlib.backup.backup_site_to_tarfile(site, backup_tar, mode="w:", options={}, verbose=False) + + with tar_path.open("rb") as backup_tar: + tar = tarfile.open(fileobj=backup_tar, mode="r:*") + sitename, version = omdlib.backup.get_site_and_version_from_backup(tar) + assert sitename == "unit" + assert version == "1.3.3i7.cee" + + names = [tarinfo.name for tarinfo in tar] + assert "unit/test123" in names + + +def test_backup_site_to_tarfile_broken_link(site, tmp_path): + Path(site.dir + "/link").symlink_to("agag") + + tar_path = tmp_path / "backup.tar" + with tar_path.open("wb") as backup_tar: + omdlib.backup.backup_site_to_tarfile(site, backup_tar, mode="w:", options={}, verbose=False) + + with tar_path.open("rb") as backup_tar: + tar = tarfile.open(fileobj=backup_tar, mode="r:*") + _sitename, _version = omdlib.backup.get_site_and_version_from_backup(tar) + + link = tar.getmember("unit/link") + assert link.linkname == "agag" + + +def test_backup_site_to_tarfile_vanishing_files(site, tmp_path, monkeypatch): + test_dir = Path(site.dir) / "xyz" + test_file = test_dir / "test_file" + test_dir.mkdir(parents=True, exist_ok=True) # pylint: disable=no-member + test_file.touch() # pylint: disable=no-member + + orig_add = omdlib.backup.BackupTarFile.add + + def add(self, name, arcname=None, recursive=True, exclude=None, filter=None): # pylint: disable=redefined-builtin + # The add() was called for test_dir which then calls os.listdir() and + # add() for all found entries. Remove the test_file here to simulate + # a vanished file during this step. + if arcname == "unit/xyz/test_file": + test_file.unlink() # pylint: disable=no-member + orig_add(self, name, arcname, recursive, exclude, filter) + + monkeypatch.setattr(omdlib.backup.BackupTarFile, "add", add) + + tar_path = tmp_path / "backup.tar" + with tar_path.open("wb") as backup_tar: + omdlib.backup.backup_site_to_tarfile(site, backup_tar, mode="w:", options={}, verbose=False) + + with tar_path.open("rb") as backup_tar: + tar = tarfile.open(fileobj=backup_tar, mode="r:*") + _sitename, _version = omdlib.backup.get_site_and_version_from_backup(tar)

5 years, 4 months

FEED-3195: Update windows agent updater modules

by Andreas Umbreit

Module: check_mk Branch: master Commit: cd9d99687bb851097fbb7779f748464947835d21 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=cd9d99687bb851… Author: Andreas Umbreit <au(a)mathias-kettner.de> Date: Tue Feb 19 15:56:56 2019 +0100 FEED-3195: Update windows agent updater modules Change-Id: I85dbe0bce1c9d93e0d9836a42390d1c47737d4b0 --- .../windows/frozen_binaries/pyinstaller-deps.make | 93 +++++++++++---------- .../src/pip/PyInstaller-3.3.1.tar.gz | Bin 3458638 -> 0 bytes .../frozen_binaries/src/pip/PyInstaller-3.4.tar.gz | Bin 0 -> 3487849 bytes ...ny.whl => altgraph-0.16.1-py2.py3-none-any.whl} | Bin 23260 -> 20311 bytes .../pip/certifi-2018.11.29-py2.py3-none-any.whl | Bin 0 -> 154154 bytes .../src/pip/certifi-2018.4.16-py2.py3-none-any.whl | Bin 150367 -> 0 bytes .../src/pip/cffi-1.11.5-cp27-cp27m-win32.whl | Bin 153593 -> 0 bytes .../src/pip/cffi-1.12.1-cp27-cp27m-win32.whl | Bin 0 -> 157185 bytes .../pip/cryptography-2.2.2-cp27-cp27m-win32.whl | Bin 1107198 -> 0 bytes .../src/pip/cryptography-2.5-cp27-cp27m-win32.whl | Bin 0 -> 1262466 bytes .../frozen_binaries/src/pip/dis3-0.1.2.tar.gz | Bin 14466 -> 0 bytes .../src/pip/dis3-0.1.3-py2-none-any.whl | Bin 0 -> 7471 bytes .../frozen_binaries/src/pip/future-0.16.0.tar.gz | Bin 824484 -> 0 bytes .../frozen_binaries/src/pip/future-0.17.1.tar.gz | Bin 0 -> 829119 bytes .../src/pip/idna-2.6-py2.py3-none-any.whl | Bin 56450 -> 0 bytes .../src/pip/idna-2.8-py2.py3-none-any.whl | Bin 0 -> 58594 bytes .../src/pip/macholib-1.11-py2.py3-none-any.whl | Bin 0 -> 36562 bytes .../src/pip/macholib-1.9-py2.py3-none-any.whl | Bin 40512 -> 0 bytes .../src/pip/pefile-2017.11.5.tar.gz | Bin 61774 -> 0 bytes .../frozen_binaries/src/pip/pefile-2018.8.8.tar.gz | Bin 0 -> 62505 bytes .../src/pip/pyOpenSSL-17.5.0-py2.py3-none-any.whl | Bin 53583 -> 0 bytes .../src/pip/pyOpenSSL-19.0.0-py2.py3-none-any.whl | Bin 0 -> 53333 bytes .../frozen_binaries/src/pip/pycparser-2.18.tar.gz | Bin 245897 -> 0 bytes .../frozen_binaries/src/pip/pycparser-2.19.tar.gz | Bin 0 -> 158295 bytes ...-win32.whl => pywin32-224-cp27-cp27m-win32.whl} | Bin 6769375 -> 6836723 bytes .../pip/pywin32_ctypes-0.2.0-py2.py3-none-any.whl | Bin 0 -> 28532 bytes .../src/pip/requests-2.18.4-py2.py3-none-any.whl | Bin 88704 -> 0 bytes .../src/pip/requests-2.21.0-py2.py3-none-any.whl | Bin 0 -> 57987 bytes ....whl => setuptools-40.8.0-py2.py3-none-any.whl} | Bin 566761 -> 575457 bytes .../src/pip/six-1.11.0-py2.py3-none-any.whl | Bin 10702 -> 0 bytes .../src/pip/six-1.12.0-py2.py3-none-any.whl | Bin 0 -> 10586 bytes .../src/pip/urllib3-1.22-py2.py3-none-any.whl | Bin 132332 -> 0 bytes .../src/pip/urllib3-1.24.1-py2.py3-none-any.whl | Bin 0 -> 118086 bytes .../src/{python-2.7.12.msi => python-2.7.15.msi} | Bin 18907136 -> 19304448 bytes 34 files changed, 49 insertions(+), 44 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=cd9d99687b…

5 years, 4 months

7165 FIX cmk-update-agent.exe: Apply Werk #6932 bugfix to Windows

by Andreas Umbreit

Module: check_mk Branch: master Commit: 672252349405218011cf298b104eb6dc962e9998 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=67225234940521… Author: Andreas Umbreit <au(a)mathias-kettner.de> Date: Mon Feb 25 17:09:11 2019 +0100 7165 FIX cmk-update-agent.exe: Apply Werk #6932 bugfix to Windows (Jira: FEED-3195) Change-Id: I31f43fc3cdb871d1ed5f4998627003ae06dae876 --- .werks/7165 | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/.werks/7165 b/.werks/7165 new file mode 100644 index 0000000..59bc5ba --- /dev/null +++ b/.werks/7165 @@ -0,0 +1,10 @@ +Title: cmk-update-agent.exe: Apply Werk #6932 bugfix to Windows +Level: 1 +Component: agents +Class: fix +Compatible: compat +Edition: cee +State: unknown +Version: 1.6.0i1 +Date: 1551110636 +

5 years, 4 months

7204 mk_jolokia.py: Auto-detect product type

by Moritz Kiemer

Module: check_mk Branch: master Commit: 363fcec610db70d7c4f8338b29f82dc6de66fbb5 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=363fcec610db70… Author: Moritz Kiemer <mo(a)mathias-kettner.de> Date: Mon Mar 4 08:14:27 2019 +0100 7204 mk_jolokia.py: Auto-detect product type The plugin now detects the product type from the jolokia info section, if the user did not provide it explicitly. Change-Id: I819878034834837347a9b17f31a3339ebbcbade5 --- .werks/7204 | 12 ++++++++++++ agents/plugins/mk_jolokia.py | 7 +++++-- 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/.werks/7204 b/.werks/7204 new file mode 100644 index 0000000..eaab79b --- /dev/null +++ b/.werks/7204 @@ -0,0 +1,12 @@ +Title: mk_jolokia.py: Auto-detect product type if not configured +Level: 1 +Component: checks +Compatible: compat +Edition: cre +Version: 1.6.0i1 +Date: 1551683726 +Class: feature + +The plugin now detects the product type from the jolokia info section, +if the user did not provide it explicitly. + diff --git a/agents/plugins/mk_jolokia.py b/agents/plugins/mk_jolokia.py index 039a003..3bdb85f 100755 --- a/agents/plugins/mk_jolokia.py +++ b/agents/plugins/mk_jolokia.py @@ -154,7 +154,8 @@ DEFAULT_CONFIG_TUPLES = ( ("service_url", None), ("service_user", None), ("service_password", None), - ("product", None, "Product description. Available: %s" % \ + ("product", None, "Product description. Available: %s. If not provided," \ + " we try to detect the product from the jolokia info section." % \ ", ".join(QUERY_SPECS_SPECIFIC.keys())), ("timeout", 1.0, "Connection/read timeout for requests."), ("custom_vars", []), @@ -492,8 +493,10 @@ def generate_jolokia_info(inst): info = data.get('info', {}) version = info.get('version', "unknown") product = info.get('product', "unknown") - if inst.product: + if inst.product is not None: product = inst.product + else: + inst.product = product agentversion = data.get('agent', "unknown") yield inst.name, product, version, agentversion

5 years, 4 months

← Newer
1
...
39
40
41
42
43
44
45
46
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Checkmk git commits March 2019