Checkmk git commits October 2017

checkmk-commits@lists.checkmk.com

10 participants
288 discussions

5359 FIX Reduced size of PDF exports containing graphs

by Lars Michelsen

Module: check_mk Branch: master Commit: 8c08e511dd4c6b43a44b4fe96a1fd5557f2a32ac URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=8c08e511dd4c6b… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Mon Oct 2 14:34:11 2017 +0200 5359 FIX Reduced size of PDF exports containing graphs The size of PDF exports increased dramatically when graphs were shown. This was caused by the gradients used in the rendered metric areas. For example a PDF with 10 graphs had a size of 6 MB with gradients compared to 260 KB without gradients. The gradients have now been dropped completely. Change-Id: I1e6b05a26a6b11da3b490d9ed12bf49c4aff57eb --- .werks/5359 | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/.werks/5359 b/.werks/5359 new file mode 100644 index 0000000..b1fde2d --- /dev/null +++ b/.werks/5359 @@ -0,0 +1,16 @@ +Title: Reduced size of PDF exports containing graphs +Level: 2 +Component: reporting +Class: fix +Compatible: compat +Edition: cee +State: unknown +Version: 1.5.0i1 +Date: 1506947471 + +The size of PDF exports increased dramatically when graphs were shown. This +was caused by the gradients used in the rendered metric areas. + +For example a PDF with 10 graphs had a size of 6 MB with gradients compared +to 260 KB without gradients. The gradients have now been dropped completely. +

6 years, 11 months

Prevent Windows agent make outside chroot jail

by Jukka Aro

Module: check_mk Branch: master Commit: 0d0c71a51896912095dd69ccb77ad26709700e02 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=0d0c71a5189691… Author: Jukka Aro <ja(a)mathias-kettner.de> Date: Fri Sep 29 12:57:28 2017 +0200 Prevent Windows agent make outside chroot jail Force to use make-agent script instead. Change-Id: Ib59a77e47c7cc67d4eac748acd49527db61ab2b8 --- agents/windows/Makefile | 10 ++++++++++ agents/windows/do-chroot | 5 +++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/agents/windows/Makefile b/agents/windows/Makefile index 1b0afb6..8f7aa27 100644 --- a/agents/windows/Makefile +++ b/agents/windows/Makefile @@ -40,6 +40,16 @@ MKDIR = mkdir -p POSTCOMPILE = mv -f -u $(DEPDIR)/$*.Td $(DEPDIR)/$*.d +EMPTY := +ifneq (,$(MAKECMDGOALS)) + SEPARATOR := $(EMPTY) $(EMPTY) +endif + +# Check if called inside chroot jail. +ifneq (WinAgent,$(CHROOTNAME)) + $(error "Not allowed outside chroot. Please call './make-agent$(SEPARATOR)$(MAKECMDGOALS)' instead") +endif + ifeq ($(shell uname),Linux) CXX = i686-w64-mingw32-g++-posix CXX_64 = x86_64-w64-mingw32-g++-posix diff --git a/agents/windows/do-chroot b/agents/windows/do-chroot index f1db6ec..be4ccd8 100755 --- a/agents/windows/do-chroot +++ b/agents/windows/do-chroot @@ -6,6 +6,7 @@ CMD="$1" export HOSTNAME=cmk-windows-agent-chroot unset LANG +CHROOTNAME=WinAgent BASE="$(dirname "$(dirname "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)")")" MNT=$BASE/agents/windows/chroot @@ -27,9 +28,9 @@ if [ ! -e $MNT/build/Makefile ] ; then fi if [ -n "$CMD" ] ; then - chroot chroot bash -c "$CMD" + chroot chroot /usr/bin/env CHROOTNAME=$CHROOTNAME bash -c "$CMD" else - chroot chroot bash -l + chroot chroot /usr/bin/env CHROOTNAME=$CHROOTNAME bash -l fi umount $MNT/build

6 years, 11 months

Build Windows agent within chroot jail

by Jukka Aro

Module: check_mk Branch: master Commit: 09575f2994a022642e1fdfaec0084600adb5ff8b URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=09575f2994a022… Author: Jukka Aro <ja(a)mathias-kettner.de> Date: Thu Sep 28 16:50:24 2017 +0200 Build Windows agent within chroot jail * Add necessary packages to debootstrap. * Mount entire sources under chroot/build to make symlinks work. * Add optinal parameters to do-chroot for commands to be executed within chroot jail. * Add 'make-agent' script for building agent within chroot jail. Change-Id: I72db1294938b6868a00356c02ab489f4b93a7440 --- agents/windows/Makefile | 7 ----- agents/windows/do-chroot | 15 ++++++++--- agents/windows/make-agent | 53 +++++++++++++++++++++++++++++++++++++ agents/windows/make-chroot | 66 +++++++++++++++++++++++++++++++++++++++++++--- 4 files changed, 128 insertions(+), 13 deletions(-) diff --git a/agents/windows/Makefile b/agents/windows/Makefile index 383c5c2..1b0afb6 100644 --- a/agents/windows/Makefile +++ b/agents/windows/Makefile @@ -215,10 +215,3 @@ clean: mrproper: clean rm -f *.exe - -setup: - sudo apt-get install \ - nsis \ - lcab \ - debootstrap \ - debian-archive-keyring diff --git a/agents/windows/do-chroot b/agents/windows/do-chroot index be7fb50..f1db6ec 100755 --- a/agents/windows/do-chroot +++ b/agents/windows/do-chroot @@ -1,9 +1,14 @@ #!/bin/bash + +# $1: cmd(s) to be executed within chroot jail +# if non-existent, enter chroot jail with 'bash -l' +CMD="$1" + export HOSTNAME=cmk-windows-agent-chroot unset LANG -BASE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -MNT=$BASE/chroot +BASE="$(dirname "$(dirname "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)")")" +MNT=$BASE/agents/windows/chroot if [ ! -e $MNT/proc/self ] ; then mount -t proc proc $MNT/proc @@ -21,7 +26,11 @@ if [ ! -e $MNT/build/Makefile ] ; then mount -o bind $BASE $MNT/build fi -chroot chroot bash -l +if [ -n "$CMD" ] ; then + chroot chroot bash -c "$CMD" +else + chroot chroot bash -l +fi umount $MNT/build umount $MNT/dev/pts diff --git a/agents/windows/make-agent b/agents/windows/make-agent new file mode 100755 index 0000000..9b45984 --- /dev/null +++ b/agents/windows/make-agent @@ -0,0 +1,53 @@ +#!/bin/bash +# The script builds Windows agent within a chroot jail. +# +# Possible targets given as parameter(s) to this script: +# setup - install dependencies for chroot +# chroot - create a chroot for building Windows agent +# * - other targets passed to make within chroot, see Makefile + +targets="$@" # no parameter -> default make target +setup_target=setup +chroot_target=chroot + +check_no_of_targets() { + if [ $1 != 1 ] ; then + echo "You cannot combine $2 with other targets." + exit 1 + fi +} + +# setup target: install dependencies for chroot +if [[ " $targets " =~ " $setup_target " ]] ; then + check_no_of_targets $# $setup_target + sudo apt-get install \ + nsis \ + lcab \ + debootstrap \ + debian-archive-keyring + exit +fi + +# chroot target: create a chroot for building Windows agent +if [[ " $targets " =~ " $chroot_target " ]] ; then + check_no_of_targets $# $chroot_target + sudo ./make-chroot + exit +fi + +# all other targets: assume chroot exists and pass targets to make inside chroot + +base="$(dirname "$(dirname "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)")")" +mnt=$base/agents/windows/chroot + +if [ ! -d "$mnt/build" ] ; then + echo "Did you forget to call '${BASH_SOURCE[0]} chroot' first?" + exit 1 +fi + +sudo ./do-chroot "cd build/agents/windows; make $targets" + +# Some cleanup: chown build artifacts to the original user. +find . -maxdepth 1 -regextype egrep \ +-regex '(.*\.(exe|msi|rc|res))|.*\/obj.*|.*build_version|.*plugins|.*msibuild' \ +-exec sudo chown -R $USER:$USER {} \; diff --git a/agents/windows/make-chroot b/agents/windows/make-chroot index aa42a5a..f420455 100755 --- a/agents/windows/make-chroot +++ b/agents/windows/make-chroot @@ -10,8 +10,8 @@ MIRROR=http://de.archive.ubuntu.com/ubuntu export HOSTNAME=cmk-windows-agent-chroot export LC_ALL=en_US.UTF-8 LANGUAGE=en_US.UTF-8 LANG=en_US.UTF-8 -BASE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -MNT=$BASE/chroot +BASE="$(dirname "$(dirname "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)")")" +MNT=$BASE/agents/windows/chroot cleanup_chroot() { PIDS=$(lsof -Fp $MNT 2>/dev/null | cut -dp -f2) @@ -42,10 +42,63 @@ rm -rf $MNT/* echo "+ INSTALLING SYSTEM" +package_names=( + "ca-certificates" + "gcc" + "gettext" + "git" + "lcab" + "libauthen-sasl-perl" + "libc6-dev" + "libencode-locale-perl" + "libfile-listing-perl" + "libfont-afm-perl" + "libglib2.0-dev" + "libgsf-1-dev" + "libgtk2.0-dev" + "libhtml-form-perl" + "libhtml-format-perl" + "libhtml-parser-perl" + "libhtml-tagset-perl" + "libhtml-tree-perl" + "libhttp-cookies-perl" + "libhttp-daemon-perl" + "libhttp-date-perl" + "libhttp-message-perl" + "libhttp-negotiate-perl" + "libio-html-perl" + "libio-socket-ssl-perl" + "liblwp-mediatypes-perl" + "liblwp-protocol-https-perl" + "libmailtools-perl" + "libnet-http-perl" + "libnet-smtp-ssl-perl" + "libnet-ssleay-perl" + "libperl-dev" + "libtimedate-perl" + "liburi-perl" + "libwww-perl" + "libwww-robotrules-perl" + "libxml-parser-perl" + "linux-base" + "locales" + "make" + "mingw-w64" + "nsis" + "openssl" + "patch" + "perl-openssl-defaults" + "pkg-config" + "python2.7" + "uuid-dev" + "wine-stable" +) +packages=$(IFS=,; echo "${package_names[*]}") + # TODO: Would use --force-check-gpg, but not supported on wheezy (root server) debootstrap \ --components=main,universe \ - --include=linux-base,locales,mingw-w64,wine-stable \ + --include="$packages" \ --arch amd64 $DISTRO $MNT $MIRROR # Need to be installed for installation. @@ -78,6 +131,13 @@ PS1='\${debian_chroot:+(\$debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033 PS1="\[\e]0;\${debian_chroot:+(\$debian_chroot)}\u@\h: \w\a\]\$PS1" EOF +# Debian pkg installation does not seem to link python executable +if [ ! -h $MNT/usr/bin/python ] ; then + cd $MNT/usr/bin + ln -s python2.7 python + cd - +fi + mkdir $MNT/build echo "+ CLEANING UP"

6 years, 11 months

check_mkevents: Only query for the needed columns

by Lars Michelsen

Module: check_mk Branch: master Commit: d6a9745815adc1fab1613b331f347156f37e1cc4 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=d6a9745815adc1… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Mon Oct 2 13:53:35 2017 +0200 check_mkevents: Only query for the needed columns Change-Id: I04535ec19b2b97772c283507665be118b17d5c5a --- active_checks/check_mkevents.cc | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/active_checks/check_mkevents.cc b/active_checks/check_mkevents.cc index 3155fad..fba3c4f 100644 --- a/active_checks/check_mkevents.cc +++ b/active_checks/check_mkevents.cc @@ -223,7 +223,10 @@ int main(int argc, char **argv) { // Create query message string query_message; - query_message += "GET events\nFilter: event_host "; + query_message += "GET events\n"; + query_message += "Columns: event_phase event_state event_text\n"; + + query_message += "Filter: event_host "; if (strchr(host, '/') != nullptr) { query_message += "in "; query_message += prepare_host_match_list(host);

6 years, 11 months

EC: Removed debug code

by Lars Michelsen

Module: check_mk Branch: master Commit: 43d8f6ed6ad5186fc2a663d08a0ad6431c9cdd74 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=43d8f6ed6ad518… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Mon Oct 2 13:40:10 2017 +0200 EC: Removed debug code Change-Id: Iec5b25707c87eb79baaa44044e8f6f5d1e441952 --- bin/mkeventd | 2 -- 1 file changed, 2 deletions(-) diff --git a/bin/mkeventd b/bin/mkeventd index c162969..de1d389 100755 --- a/bin/mkeventd +++ b/bin/mkeventd @@ -1783,8 +1783,6 @@ class EventServer(ECServerThread): host_downtimes = {} for event in g_event_status.events(): - if "host_in_downtime" not in event: - self.logger.warning(repr(event)) if not event["host_in_downtime"]: continue # only care about events created in downtime

6 years, 11 months

Refactor the processing of Windows eventlog

by Jukka Aro

Module: check_mk Branch: master Commit: a48ee8e59b95a998c1407d922db3c6f891f054cd URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=a48ee8e59b95a9… Author: Jukka Aro <ja(a)mathias-kettner.de> Date: Thu Sep 21 16:51:24 2017 +0200 Refactor the processing of Windows eventlog Key changes: * Extract the actual processing (done twice!) in a separate function. * Use functions in anonymous namespace for functionality that does not need to access class members. * Improve readability: * Rename functions and variables to make them more descriptive. * Favour pass-by-value function parameters and real return values over output reference parameters with void return value. * Favour std::unique_ptr over std::shared_ptr (make explicit who owns an object). Change-Id: I7db5ee676c80bd4f4a695f9401104420ac69acf0 --- agents/windows/EventLog.cc | 7 +- agents/windows/EventLog.h | 2 +- agents/windows/EventLogVista.cc | 6 +- agents/windows/EventLogVista.h | 2 +- agents/windows/IEventLog.h | 2 +- agents/windows/build_version | 2 +- agents/windows/check_mk_agent-64.exe | Bin 1564160 -> 1563648 bytes agents/windows/check_mk_agent-64.unstripped.exe | Bin 22765638 -> 22700573 bytes agents/windows/check_mk_agent-64.unversioned.exe | Bin 1564160 -> 1563648 bytes agents/windows/check_mk_agent.exe | Bin 1547264 -> 1546752 bytes agents/windows/check_mk_agent.msi | Bin 3385344 -> 3384320 bytes agents/windows/check_mk_agent.unstripped.exe | Bin 19613197 -> 19551772 bytes agents/windows/check_mk_agent.unversioned.exe | Bin 1547264 -> 1546752 bytes agents/windows/crash.exe | Bin 358864 -> 358864 bytes agents/windows/install_agent-64.exe | Bin 681081 -> 680694 bytes agents/windows/install_agent.exe | Bin 675979 -> 675746 bytes agents/windows/nowin.exe | Bin 361000 -> 361000 bytes agents/windows/plugins/mk_jolokia.exe | Bin 10106814 -> 10106814 bytes agents/windows/sections/SectionEventlog.cc | 106 ++++++++++++----------- agents/windows/sections/SectionEventlog.h | 8 +- 20 files changed, 70 insertions(+), 65 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=a48ee8e59b…

6 years, 11 months

EC: Refactored tables to dedicated objects

by Lars Michelsen

Module: check_mk Branch: master Commit: c026f125cda058d5ca264314e989fe522ac8d2ee URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=c026f125cda058… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Fri Sep 29 13:28:47 2017 +0200 EC: Refactored tables to dedicated objects Change-Id: I65c56763920dc6984c0ac4073eeb71e3736ef52d --- bin/mkeventd | 295 +++++++++++++++++++++++++++++++++++------------------------ 1 file changed, 176 insertions(+), 119 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=c026f125cd…

6 years, 11 months

Cleaned up livestatus query logging

by Lars Michelsen

Module: check_mk Branch: master Commit: 313b3d8635305b157437758c574270283a98df90 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=313b3d8635305b… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Fri Sep 29 12:24:55 2017 +0200 Cleaned up livestatus query logging Change-Id: I5bd3c02f05b71fe96d5119d476d196dc62415309 --- bin/mkeventd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/mkeventd b/bin/mkeventd index 8605a02..3dd042e 100755 --- a/bin/mkeventd +++ b/bin/mkeventd @@ -574,7 +574,7 @@ class Query(object): def __repr__(self): - return repr("\\n".join(self._raw_query)) + return repr("\n".join(self._raw_query)) #.

6 years, 11 months

EC: Moved all query parsing stuff to query classes

by Lars Michelsen

Module: check_mk Branch: master Commit: 048c487f1915a22eff03f741c91d4d3693bee1d0 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=048c487f1915a2… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Fri Sep 29 12:58:00 2017 +0200 EC: Moved all query parsing stuff to query classes Change-Id: I2a0961a6198085a15aca272d1ffd082d7d7c2d1c --- bin/mkeventd | 371 +++++++++++++++++++++++++++++++++-------------------------- 1 file changed, 209 insertions(+), 162 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=048c487f19…

6 years, 11 months

EC: Improved performance of row filtering (Use precalculated colum indexes )

by Lars Michelsen

Module: check_mk Branch: master Commit: 6ae9c8ed8e4ca0523daeb5dd865d3170658d79dd URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=6ae9c8ed8e4ca0… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Mon Oct 2 11:33:52 2017 +0200 EC: Improved performance of row filtering (Use precalculated colum indexes) Change-Id: I56b0b6bb90b0bdf3e046e72ee8154356c8863335 --- bin/mkeventd | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/bin/mkeventd b/bin/mkeventd index 3a12aa9..565adac 100755 --- a/bin/mkeventd +++ b/bin/mkeventd @@ -793,7 +793,9 @@ def log_event_history_to_mongodb(event, what, who, addinfo): 'addinfo': addinfo, }) -def get_event_history_from_mongodb(filters, limit): +def get_event_history_from_mongodb(query): + filters, limit = query.filters, query.limit + history_entries = [] if not g_mongo_conn: @@ -994,7 +996,8 @@ def flush_event_history_files(): expire_logfiles(True) -def get_event_history_from_file(filters, limit): +def get_event_history_from_file(query): + filters, limit = query.filters, query.limit history_entries = [] log_dir = g_state_dir + "/history" if not os.path.exists(log_dir): @@ -1055,7 +1058,7 @@ def get_event_history_from_file(filters, limit): logger.info("Skipping logfile %s.log because of time filter" % ts) continue # skip this file - new_entries = parse_history_file(path, filters, greptexts, limit) + new_entries = parse_history_file(path, query, greptexts, limit) history_entries += new_entries if limit != None: limit -= len(new_entries) @@ -1064,7 +1067,7 @@ def get_event_history_from_file(filters, limit): -def parse_history_file(path, filters, greptexts, limit): +def parse_history_file(path, query, greptexts, limit): entries = [] line_no = 0 # If we have greptexts we pre-filter the file using the extremely @@ -1088,7 +1091,7 @@ def parse_history_file(path, filters, greptexts, limit): parts = line.decode('utf-8').rstrip('\n').split('\t') convert_history_line(parts) values = [line_no] + parts - if g_status_server.table_history.filter_row(headers, filters, values): + if g_status_server.table_history.filter_row(query, values): entries.append(values) except Exception as e: logger.exception("Invalid line '%s' in history file %s: %s" % (line, path, e)) @@ -3301,8 +3304,7 @@ class StatusTable(object): # Apply filters # TODO: History filtering is done in history load code. Check for improvements if query.filters and query.table_name != "history": - # TODO: Hand over whole query object to filter_row() - match = self.filter_row(query.table.column_names, query.filters, row) + match = self.filter_row(query, row) if not match: continue @@ -3325,15 +3327,11 @@ class StatusTable(object): return result_row - def filter_row(self, headers, filters, list_row): - # TODO(sp) This is a real performance killer: We should just calculate - # the column indices for column names *once*, not doing some dict/list - # Kung Fu for every row... - row = dict(zip(headers, list_row)) - for column, opfunc, argument in filters: - if not opfunc(row[column], argument): + def filter_row(self, query, row): + for column, opfunc, argument in query.filters: + if not opfunc(row[query.table.column_indices[column]], argument): return None - return list_row + return row @@ -3400,9 +3398,9 @@ class StatusTableHistory(StatusTable): def _enumerate(self, query): if g_config['archive_mode'] == 'mongodb': - return get_event_history_from_mongodb(query.filters, query.limit) + return get_event_history_from_mongodb(query) else: - return get_event_history_from_file(query.filters, query.limit) + return get_event_history_from_file(query)

6 years, 11 months

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Checkmk git commits October 2017