Checkmk git commits October 2017

checkmk-commits@lists.checkmk.com

10 participants
288 discussions

5361 FIX Fixed cancelling events by "text" (regression #5057,

by Lars Michelsen

since 1.4.0p11) Message-ID: <59d48ab3.jrGWgD6ibsthLec+%lm(a)mathias-kettner.de> User-Agent: Heirloom mailx 12.5 6/20/10 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Module: check_mk Branch: master Commit: 703bb61ec6a0c1e5365425968e3c70081193cba5 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=703bb61ec6a0c1… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Wed Oct 4 09:07:57 2017 +0200 5361 FIX Fixed cancelling events by "text" (regression #5057, since 1.4.0p11) Change-Id: Ic9791980bdcf1fe5dc5d89b073f49faa330301f5 --- .werks/5361 | 11 +++++++++++ bin/mkeventd | 2 +- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/.werks/5361 b/.werks/5361 new file mode 100644 index 0000000..f6546b0 --- /dev/null +++ b/.werks/5361 @@ -0,0 +1,11 @@ +Title: Fixed cancelling events by "text" (regression #5057, since 1.4.0p11) +Level: 1 +Component: ec +Class: fix +Compatible: compat +Edition: cre +State: unknown +Version: 1.5.0i1 +Date: 1507100833 + + diff --git a/bin/mkeventd b/bin/mkeventd index a1e85a9..57f26ae 100755 --- a/bin/mkeventd +++ b/bin/mkeventd @@ -2448,7 +2448,7 @@ class EventServer(ECServerThread): cp = True match_groups = match(rule.get("match_ok", ""), event["text"], complete = False) - if match_groups != False and cp and cancel_by_application: + if match_groups != False and cp and ("cancel_application" not in rule or cancel_by_application): if debug: self.logger.info(" found cancelling event") if match_groups == True:

6 years, 11 months

Update binaries for Windows agent

by Jukka Aro

Module: check_mk Branch: master Commit: b73e4506bb303b1dfea223548a1ce7290bc8f2f2 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=b73e4506bb303b… Author: Jukka Aro <ja(a)mathias-kettner.de> Date: Mon Oct 2 15:18:54 2017 +0200 Update binaries for Windows agent Change-Id: I1141a7e891ce10ef5182c08db8a45c3613d4a228 --- agents/windows/build_version | 2 +- agents/windows/check_mk_agent-64.exe | Bin 1563648 -> 1568768 bytes agents/windows/check_mk_agent-64.unstripped.exe | Bin 22700573 -> 22974157 bytes agents/windows/check_mk_agent-64.unversioned.exe | Bin 1563648 -> 1568768 bytes agents/windows/check_mk_agent.exe | Bin 1546752 -> 1552384 bytes agents/windows/check_mk_agent.msi | Bin 3384320 -> 3395072 bytes agents/windows/check_mk_agent.unstripped.exe | Bin 19551772 -> 19784372 bytes agents/windows/check_mk_agent.unversioned.exe | Bin 1546752 -> 1552384 bytes agents/windows/crash.exe | Bin 358864 -> 358864 bytes agents/windows/install_agent-64.exe | Bin 680694 -> 683168 bytes agents/windows/install_agent.exe | Bin 675746 -> 678265 bytes agents/windows/nowin.exe | Bin 361000 -> 361000 bytes 12 files changed, 1 insertion(+), 1 deletion(-) diff --git a/agents/windows/build_version b/agents/windows/build_version index 5bbb296..5672c98 100644 --- a/agents/windows/build_version +++ b/agents/windows/build_version @@ -1 +1 @@ -2936 +2938 diff --git a/agents/windows/check_mk_agent-64.exe b/agents/windows/check_mk_agent-64.exe index c9779dd..4884f6a 100755 Binary files a/agents/windows/check_mk_agent-64.exe and b/agents/windows/check_mk_agent-64.exe differ diff --git a/agents/windows/check_mk_agent-64.unstripped.exe b/agents/windows/check_mk_agent-64.unstripped.exe index bf01487..9313547 100755 Binary files a/agents/windows/check_mk_agent-64.unstripped.exe and b/agents/windows/check_mk_agent-64.unstripped.exe differ diff --git a/agents/windows/check_mk_agent-64.unversioned.exe b/agents/windows/check_mk_agent-64.unversioned.exe index 2e79ffb..1c0c693 100755 Binary files a/agents/windows/check_mk_agent-64.unversioned.exe and b/agents/windows/check_mk_agent-64.unversioned.exe differ diff --git a/agents/windows/check_mk_agent.exe b/agents/windows/check_mk_agent.exe index 276b9ff..e8eb7ee 100755 Binary files a/agents/windows/check_mk_agent.exe and b/agents/windows/check_mk_agent.exe differ diff --git a/agents/windows/check_mk_agent.msi b/agents/windows/check_mk_agent.msi index 50d6cfe..46ca8f7 100755 Binary files a/agents/windows/check_mk_agent.msi and b/agents/windows/check_mk_agent.msi differ diff --git a/agents/windows/check_mk_agent.unstripped.exe b/agents/windows/check_mk_agent.unstripped.exe index 36eaac5..6d31405 100755 Binary files a/agents/windows/check_mk_agent.unstripped.exe and b/agents/windows/check_mk_agent.unstripped.exe differ diff --git a/agents/windows/check_mk_agent.unversioned.exe b/agents/windows/check_mk_agent.unversioned.exe index 37c2b74..d077a3f 100755 Binary files a/agents/windows/check_mk_agent.unversioned.exe and b/agents/windows/check_mk_agent.unversioned.exe differ diff --git a/agents/windows/crash.exe b/agents/windows/crash.exe index 667d940..19ccc3d 100755 Binary files a/agents/windows/crash.exe and b/agents/windows/crash.exe differ diff --git a/agents/windows/install_agent-64.exe b/agents/windows/install_agent-64.exe index e038745..70bec4d 100755 Binary files a/agents/windows/install_agent-64.exe and b/agents/windows/install_agent-64.exe differ diff --git a/agents/windows/install_agent.exe b/agents/windows/install_agent.exe index c84f5da..f89335c 100755 Binary files a/agents/windows/install_agent.exe and b/agents/windows/install_agent.exe differ diff --git a/agents/windows/nowin.exe b/agents/windows/nowin.exe index c56d6a4..bc20b5d 100755 Binary files a/agents/windows/nowin.exe and b/agents/windows/nowin.exe differ

6 years, 11 months

Removed html.log() call

by Lars Michelsen

Module: check_mk Branch: master Commit: 1c5fc9cc80357dde879e0a59a4f44feb8de71696 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=1c5fc9cc80357d… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Wed Oct 4 07:56:09 2017 +0200 Removed html.log() call - logging is not related to html rendering - method was not aware of log levels There is now a dedicated "log" module to manage the logging of the GUI. Use something like this now: C+: from log import logger logger.error(...) C-: You can use all the features of the default python logging module loggers now. Change-Id: Iaa12a99e4e70b4f4cf7925770d01758d813cdfaa --- web/htdocs/html_mod_python.py | 7 ------- web/htdocs/htmllib.py | 4 ---- 2 files changed, 11 deletions(-) diff --git a/web/htdocs/html_mod_python.py b/web/htdocs/html_mod_python.py index 9623666..923f131 100644 --- a/web/htdocs/html_mod_python.py +++ b/web/htdocs/html_mod_python.py @@ -229,13 +229,6 @@ class html_mod_python(htmllib.html): self.javascript('update_header_timer()') self.top_heading_right() - def log(self, *args): - for arg in args: - if type(arg) in (str, unicode): - text = arg - else: - text = repr(arg) - logger.warning(text) def http_redirect(self, url): self.set_http_header('Location', url) diff --git a/web/htdocs/htmllib.py b/web/htdocs/htmllib.py index 721b528..0364e18 100644 --- a/web/htdocs/htmllib.py +++ b/web/htdocs/htmllib.py @@ -1735,10 +1735,6 @@ class html(HTMLGenerator, RequestHandler): self.lowlevel_write("%s" % self.render_pre(formatted)) - def log(self, *args): - raise NotImplementedError() - - # # URL building #

6 years, 11 months

Changed html.log calls to use logger directly

by Lars Michelsen

Module: check_mk Branch: master Commit: 1f519ce4192a43ba3ad16f2b541393416246b5df URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=1f519ce4192a43… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Wed Oct 4 07:54:21 2017 +0200 Changed html.log calls to use logger directly Change-Id: Icd880beff95483f5457a124bc71753aaef71a22c --- .bugs/2970 | 7 +++++-- web/htdocs/metrics.py | 5 ++--- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/.bugs/2970 b/.bugs/2970 index fa0a97c..1aefb29 100644 --- a/.bugs/2970 +++ b/.bugs/2970 @@ -1,9 +1,12 @@ Title: Manually defined log level for web.log is not used Component: wato -State: open +Class: bug +State: works4me Date: 2017-09-22 10:36:07 Targetversion: 1.4.0 -Class: bug If the log level is set manually in WATO nothing changes in the global.mk. It seems that the setting is not used at all. + +2017-10-04 07:49:37: changed state open -> works4me +The setting "User Interface > Logging" is stored in etc/check_mk/multisite.d/wato/global.mk by saving the option log_levels. It works as expected in 1.4 and master. diff --git a/web/htdocs/metrics.py b/web/htdocs/metrics.py index 208b44f..dec841d 100644 --- a/web/htdocs/metrics.py +++ b/web/htdocs/metrics.py @@ -411,7 +411,7 @@ def parse_perf_data(perf_data_string, check_command=None): try: parts = split_perf_data(perf_data_string) except ValueError, e: - html.log("Failed to parse perfdata string: %s" % perf_data_string) + logger.exception("Failed to parse perfdata string: %s", perf_data_string) return [], check_command if not parts: @@ -464,8 +464,7 @@ def parse_perf_data(perf_data_string, check_command=None): perf_data.append(perf_data_tuple) except: - html.log("Failed to parse perfdata '%s': %s" % - (perf_data_string, traceback.format_exc())) + logger.exception("Failed to parse perfdata '%s'", perf_data_string) if config.debug: raise

6 years, 11 months

LDAP: Use regular logger interface

by Lars Michelsen

Module: check_mk Branch: master Commit: 6547837c03784ddf4997473100b504d071d05fa7 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=6547837c03784d… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Wed Oct 4 08:19:40 2017 +0200 LDAP: Use regular logger interface Change-Id: I39b1ca0bf7300dfcd42ed52f8e82b21975450d4c --- web/plugins/userdb/ldap.py | 56 +++++++++++++++++++++------------------------- 1 file changed, 25 insertions(+), 31 deletions(-) diff --git a/web/plugins/userdb/ldap.py b/web/plugins/userdb/ldap.py index ea8cee3..ee40b24 100644 --- a/web/plugins/userdb/ldap.py +++ b/web/plugins/userdb/ldap.py @@ -178,7 +178,7 @@ class LDAPUserConnector(UserConnector): self._ldap_obj = None self._ldap_obj_config = None - self._logger = log.logger.getChild("ldap") + self._logger = log.logger.getChild("ldap.Connection(%s)" % self.id()) self._user_cache = {} self._group_cache = {} @@ -215,11 +215,6 @@ class LDAPUserConnector(UserConnector): return self._config['id'] - # TODO: Clean this up and make log calls different log levels - def log(self, s): - self._logger.debug('LDAP [%s]: %s' % (self.id(), s)) - - def connect_server(self, server): try: uri = self.format_ldap_uri(server) @@ -272,10 +267,10 @@ class LDAPUserConnector(UserConnector): and not "no_persistent" in self._config \ and self._ldap_obj \ and self._config == self._ldap_obj_config: - self.log('LDAP CONNECT - Using existing connecting') + self._logger.debug('LDAP CONNECT - Using existing connecting') return # Use existing connections (if connection settings have not changed) else: - self.log('LDAP CONNECT - Connecting...') + self._logger.debug('LDAP CONNECT - Connecting...') ldap_test_module() @@ -326,12 +321,12 @@ class LDAPUserConnector(UserConnector): locator.m_logger = self._logger try: server = locator.locate(domain) - self.log(' DISCOVERY: Discovered server %r from %r' % (server, domain)) + self._logger.debug(' DISCOVERY: Discovered server %r from %r' % (server, domain)) return server except Exception, e: - self.log(' DISCOVERY: Failed to discover a server from domain %r' % domain) + self._logger.debug(' DISCOVERY: Failed to discover a server from domain %r' % domain) log_exception() - self.log(' DISCOVERY: Try to use domain DNS name %r as server' % domain) + self._logger.debug(' DISCOVERY: Try to use domain DNS name %r as server' % domain) return domain @@ -352,12 +347,12 @@ class LDAPUserConnector(UserConnector): def bind(self, user_dn, password, catch = True, conn = None): if conn is None: conn = self._ldap_obj - self.log('LDAP_BIND %s' % user_dn) + self._logger.debug('LDAP_BIND %s' % user_dn) try: conn.simple_bind_s(user_dn.encode("utf-8"), password) - self.log(' SUCCESS') + self._logger.debug(' SUCCESS') except ldap.LDAPError, e: - self.log(' FAILED (%s: %s)' % (e.__class__.__name__, e)) + self._logger.debug(' FAILED (%s: %s)' % (e.__class__.__name__, e)) if catch: raise MKLDAPException(_('Unable to authenticate with LDAP (%s)') % e) else: @@ -464,7 +459,7 @@ class LDAPUserConnector(UserConnector): def ldap_paged_async_search(self, base, scope, filt, columns): - self.log(' PAGED ASYNC SEARCH') + self._logger.debug(' PAGED ASYNC SEARCH') page_size = self._config.get('page_size', 1000) if ldap_compat: @@ -508,7 +503,7 @@ class LDAPUserConnector(UserConnector): if columns == None: columns = [] - self.log('LDAP_SEARCH "%s" "%s" "%s" "%r"' % (base, scope, filt, columns)) + self._logger.debug('LDAP_SEARCH "%s" "%s" "%s" "%r"' % (base, scope, filt, columns)) start_time = time.time() # In some environments, the connection to the LDAP server does not seem to @@ -549,17 +544,17 @@ class LDAPUserConnector(UserConnector): except (ldap.SERVER_DOWN, ldap.TIMEOUT, MKLDAPException), e: last_exc = e if implicit_connect and tries_left: - self.log(' Received %r. Retrying with clean connection...' % e) + self._logger.debug(' Received %r. Retrying with clean connection...' % e) self.disconnect() time.sleep(0.5) else: - self.log(' Giving up.') + self._logger.debug(' Giving up.') break duration = time.time() - start_time if not success: - self.log(' FAILED') + self._logger.debug(' FAILED') if config.debug: raise MKLDAPException(_('Unable to successfully perform the LDAP search ' '(Base: %s, Scope: %s, Filter: %s, Columns: %s): %s') % @@ -569,7 +564,7 @@ class LDAPUserConnector(UserConnector): else: raise MKLDAPException(_('Unable to successfully perform the LDAP search (%s)') % last_exc) - self.log(' RESULT length: %d, duration: %0.3f' % (len(result), duration)) + self._logger.debug(' RESULT length: %d, duration: %0.3f' % (len(result), duration)) return result @@ -948,8 +943,7 @@ class LDAPUserConnector(UserConnector): self.bind(user_dn, password) result = username.encode('utf-8') except: - self.log(" Exception during authentication (User: %s): %s" % - (username, traceback.format_exc())) + self._logger.exception(" Exception during authentication (User: %s)", username) result = False self.default_bind(self._ldap_obj) @@ -995,7 +989,7 @@ class LDAPUserConnector(UserConnector): self.set_last_sync_time() if not self.has_user_base_dn_configured(): - self.log("Not trying sync (no \"user base DN\" configured)") + self._logger.debug("Not trying sync (no \"user base DN\" configured)") return # silently skip sync without configuration register_user_attribute_sync_plugins() @@ -1007,8 +1001,8 @@ class LDAPUserConnector(UserConnector): start_time = time.time() connection_id = self.id() - self.log('SYNC STARTED') - self.log(' SYNC PLUGINS: %s' % ', '.join(self._config['active_plugins'].keys())) + self._logger.info('SYNC STARTED') + self._logger.debug(' SYNC PLUGINS: %s' % ', '.join(self._config['active_plugins'].keys())) ldap_users = self.get_users() @@ -1052,11 +1046,11 @@ class LDAPUserConnector(UserConnector): mode_create, user = load_user(user_id) user_connection_id = cleanup_connection_id(user.get('connector')) if user_connection_id != connection_id: - self.log(' SKIP SYNC "%s" (name conflict after adding suffix ' + self._logger.debug(' SKIP SYNC "%s" (name conflict after adding suffix ' 'with user from "%s" connector)' % (user_id, user_connection_id)) continue # added suffix, still name conflict else: - self.log(' SKIP SYNC "%s" (name conflict with user from "%s" connector)' % (user_id, user_connection_id)) + self._logger.debug(' SKIP SYNC "%s" (name conflict with user from "%s" connector)' % (user_id, user_connection_id)) continue # name conflict, different connector self.execute_active_sync_plugins(user_id, ldap_user, user) @@ -1106,7 +1100,7 @@ class LDAPUserConnector(UserConnector): changes.append(_("LDAP [%s]: Modified user %s (%s)") % (connection_id, user_id, ', '.join(details))) duration = time.time() - start_time - self.log('SYNC FINISHED - Duration: %0.3f sec' % duration) + self._logger.info('SYNC FINISHED - Duration: %0.3f sec', duration) if changes: wato.add_change("edit-users", "<br>\n".join(changes), add_user=False) @@ -1926,7 +1920,7 @@ def synchronize_profile_to_sites(connection, user_id, profile): for site_id in config.sitenames() if not config.site_is_local(site_id) ] - connection.log('Credentials changed: %s. Trying to sync to %d sites' % + connection._logger.debug('Credentials changed: %s. Trying to sync to %d sites' % (user_id, len(remote_sites))) num_disabled = 0 @@ -1954,12 +1948,12 @@ def synchronize_profile_to_sites(connection, user_id, profile): num_succeeded += 1 else: num_failed += 1 - connection.log(' FAILED [%s]: %s' % (site_id, result)) + connection._logger.debug(' FAILED [%s]: %s' % (site_id, result)) # Add pending entry to make sync possible later for admins if config.wato_enabled: wato.add_change("edit-users", _('Password changed (sync failed: %s)') % result, add_user=False, sites=[site_id], need_restart=False) - connection.log(' Disabled: %d, Succeeded: %d, Failed: %d' % + connection._logger.debug(' Disabled: %d, Succeeded: %d, Failed: %d' % (num_disabled, num_succeeded, num_failed))

6 years, 11 months

5378 FIX Make Windows section webservices backwards compatible

by Jukka Aro

Module: check_mk Branch: master Commit: 74786c1d9e1a5f3a7194bd78debbd73a3750203a URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=74786c1d9e1a5f… Author: Jukka Aro <ja(a)mathias-kettner.de> Date: Mon Oct 2 15:27:31 2017 +0200 5378 FIX Make Windows section webservices backwards compatible Version 1.4.0 'corrected' the naming of section wmi_webservices. Unfortunately, this broke the backwards compatibility of old ini files that still call this 'webservices'. The fix maps the old name to the new one so that both old and new ini files work. Moreover, the agent bakery was still writing ini files with the old name. Now the agent bakery is also fixed. Change-Id: Iecfe16c55a6fdfb0b47b1a939a30d11c4707bc92 --- .werks/5378 | 11 +++++++++++ agents/windows/Configurable.h | 15 +++++++++++---- agents/windows/SectionManager.cc | 21 ++++++++++++++++++--- 3 files changed, 40 insertions(+), 7 deletions(-) diff --git a/.werks/5378 b/.werks/5378 new file mode 100644 index 0000000..e1bb0b6 --- /dev/null +++ b/.werks/5378 @@ -0,0 +1,11 @@ +Title: Windows agent: Ensure backwards compatibility of section webservices +Level: 1 +Component: checks +Compatible: compat +Edition: cee +Version: 1.5.0i1 +Date: 1506950547 +Class: fix + +Version 1.4.0 'corrected' the naming of section wmi_webservices. Unfortunately, this broke the backwards compatibility of old ini files that still call this 'webservices'. The fix maps the old name to the new one so that both old and new ini files work. Moreover, the agent bakery was still writing ini files with the old name. Now the agent bakery is also fixed. + diff --git a/agents/windows/Configurable.h b/agents/windows/Configurable.h index 3edc96e..936f273 100644 --- a/agents/windows/Configurable.h +++ b/agents/windows/Configurable.h @@ -203,17 +203,20 @@ template <typename ContainerT, typename BlockModeT = BlockMode::Nop<ContainerT>, typename AddModeT = AddMode::Append<ContainerT>> class SplittingListConfigurable : public ListConfigurable<ContainerT, BlockModeT, AddModeT> { - char _split_char; typedef ListConfigurable<ContainerT, BlockModeT, AddModeT> SuperT; - typedef typename ContainerT::value_type DataT; + typedef std::function<std::string(const std::string &)> MapFunction; public: SplittingListConfigurable(Configuration &config, const char *section, const char *key, const WinApiAdaptor &winapi, + const MapFunction &mapFunction = + [](const std::string &s) { return s; }, char split_char = ' ') - : SuperT(config, section, key, winapi), _split_char(split_char) {} + : SuperT(config, section, key, winapi) + , _mapFunction(mapFunction) + , _split_char(split_char) {} virtual void feed(const std::string &key, const std::string &value) override { @@ -221,7 +224,7 @@ public: std::stringstream str(value); std::string item; while (getline(str, item, _split_char)) { - SuperT::feed(key, item); + SuperT::feed(key, _mapFunction(item)); } } @@ -233,6 +236,10 @@ public: } out << "\n"; } + +private: + const MapFunction _mapFunction; + char _split_char; }; #endif // Configurable_h diff --git a/agents/windows/SectionManager.cc b/agents/windows/SectionManager.cc index 6e27d32..534430f 100644 --- a/agents/windows/SectionManager.cc +++ b/agents/windows/SectionManager.cc @@ -1,4 +1,5 @@ #include "SectionManager.h" +#include <unordered_map> #include "Configuration.h" #include "Environment.h" #include "sections/SectionCheckMK.h" @@ -21,12 +22,26 @@ #include "sections/SectionWMI.h" #include "sections/SectionWinperf.h" +namespace { +// Fix possible backwards incompatibility of section names by mapping +// 'old' names to 'new' ones. +std::string mapSectionName(const std::string &sectionName) { + const std::unordered_map<std::string, std::string> mappedSectionNames = { + {"webservices", "wmi_webservices"}}; + const auto it = mappedSectionNames.find(sectionName); + return it == mappedSectionNames.end() ? sectionName : it->second; +} + +} // namespace + SectionManager::SectionManager(Configuration &config, Logger *logger, const WinApiAdaptor &winapi) : _ps_use_wmi(config, "ps", "use_wmi", false, winapi) - , _enabled_sections(config, "global", "sections", winapi) - , _disabled_sections(config, "global", "disabled_sections", winapi) - , _realtime_sections(config, "global", "realtime_sections", winapi) + , _enabled_sections(config, "global", "sections", winapi, mapSectionName) + , _disabled_sections(config, "global", "disabled_sections", winapi, + mapSectionName) + , _realtime_sections(config, "global", "realtime_sections", winapi, + mapSectionName) , _script_local_includes(config, "local", "include", winapi) , _script_plugin_includes(config, "plugin", "include", winapi) , _winperf_counters(config, "winperf", "counters", winapi)

6 years, 11 months

Temporarily disable makefile chroot check'

by Lars Michelsen

Module: check_mk Branch: master Commit: 23ab59f4855cb375ce25d06a86d96aa25250d580 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=23ab59f4855cb3… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Mon Oct 2 21:25:09 2017 +0200 Temporarily disable makefile chroot check' The setversion target is currently called by the build system. We change this when integrating the chroot based build into the build system soon. Change-Id: I42c458209bec7c5599cf04fb9490bb915ad0b1a7 --- agents/windows/Makefile | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/agents/windows/Makefile b/agents/windows/Makefile index 8f7aa27..1c58698 100644 --- a/agents/windows/Makefile +++ b/agents/windows/Makefile @@ -46,9 +46,9 @@ ifneq (,$(MAKECMDGOALS)) endif # Check if called inside chroot jail. -ifneq (WinAgent,$(CHROOTNAME)) - $(error "Not allowed outside chroot. Please call './make-agent$(SEPARATOR)$(MAKECMDGOALS)' instead") -endif +#ifneq (WinAgent,$(CHROOTNAME)) +# $(error "Not allowed outside chroot. Please call './make-agent$(SEPARATOR)$(MAKECMDGOALS)' instead") +#endif ifeq ($(shell uname),Linux) CXX = i686-w64-mingw32-g++-posix

6 years, 11 months

5360 FIX Improved GUI performance when having a large number of custom views or dashboards

by Lars Michelsen

Module: check_mk Branch: master Commit: 91b50e926ac45537f40f8bef5e82fd3201847d1e URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=91b50e926ac455… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Mon Oct 2 15:49:21 2017 +0200 5360 FIX Improved GUI performance when having a large number of custom views or dashboards Change-Id: I363390977425d705016df1f145480a1105058569 --- .werks/5360 | 10 +++++ web/htdocs/visuals.py | 105 ++++++++++++++++++++++++++++++++++++++------------ 2 files changed, 90 insertions(+), 25 deletions(-) diff --git a/.werks/5360 b/.werks/5360 new file mode 100644 index 0000000..27e027d --- /dev/null +++ b/.werks/5360 @@ -0,0 +1,10 @@ +Title: Improved GUI performance when having a large number of custom views or dashboards +Level: 1 +Component: multisite +Compatible: compat +Edition: cre +Version: 1.5.0i1 +Date: 1506952133 +Class: fix + + diff --git a/web/htdocs/visuals.py b/web/htdocs/visuals.py index eabd0ad..731be69 100644 --- a/web/htdocs/visuals.py +++ b/web/htdocs/visuals.py @@ -99,6 +99,40 @@ def load_plugins(force): # | | # '----------------------------------------------------------------------' +class UserVisualsCache(object): + """Realizes a in memory cache (per apache process). This has been introduced to improve the + situation where there are hundrets of custom visuals (views here). These visuals are rarely + changed, but read and evaluated(!) during each page request which costs a lot of time.""" + def __init__(self): + super(UserVisualsCache, self).__init__() + self._cache = {} + + + def get(self, path): + try: + cached_modification_timestamp, cached_user_visuals = self._cache[path] + + try: + current_modification_timestamp = os.stat(path).st_mtime + except IOError: + return None # Invalidate cache (file to readable) + + if os.stat(path).st_mtime <= cached_modification_timestamp: + return cached_user_visuals # Use cache + else: + return None # Cache is outdated + except KeyError: + return None # Not cached yet + + + def add(self, path, modification_timestamp, user_visuals): + self._cache[path] = modification_timestamp, user_visuals + + +_user_visuals_cache = UserVisualsCache() + + + def save(what, visuals, user_id = None): if user_id == None: user_id = config.user.id @@ -109,6 +143,7 @@ def save(what, visuals, user_id = None): uservisuals[name] = visual config.save_user_file('user_' + what, uservisuals, user = user_id) + # FIXME: Currently all user visual files of this type are locked. We could optimize # this not to lock all files but only lock the files the user is about to modify. def load(what, builtin_visuals, skip_func = None, lock=False): @@ -128,6 +163,14 @@ def load(what, builtin_visuals, skip_func = None, lock=False): visuals[('', name)] = visual # Now scan users subdirs for files "user_*.mk" + visuals.update(load_user_visuals(what, builtin_visuals, skip_func, lock)) + + return visuals + + +def load_user_visuals(what, builtin_visuals, skip_func, lock): + visuals = {} + subdirs = os.listdir(config.config_dir) for user in subdirs: try: @@ -148,39 +191,51 @@ def load(what, builtin_visuals, skip_func = None, lock=False): if not userdb.user_exists(user): continue - user_visuals = store.load_data_from_file(path, {}, lock) - for name, visual in user_visuals.items(): - visual["owner"] = user - visual["name"] = name + user_visuals = _user_visuals_cache.get(path) + if user_visuals is None: + modification_timestamp = os.stat(path).st_mtime + user_visuals = load_visuals_of_a_user(what, builtin_visuals, skip_func, lock, path, user) + _user_visuals_cache.add(path, modification_timestamp, user_visuals) - if skip_func and skip_func(visual): - continue + visuals.update(user_visuals) - # Maybe resolve inherited attributes. This was a feature for several versions - # to make the visual texts localizable. This has been removed because the visual - # texts can now be localized using the custom localization strings. - # This is needed for backward compatibility to make the visuals without these - # attributes get the attributes from their builtin visual. - builtin_visual = visuals.get(('', name)) - if builtin_visual: - for attr in [ 'title', 'linktitle', 'topic', 'description' ]: - if attr not in visual and attr in builtin_visual: - visual[attr] = builtin_visual[attr] + except SyntaxError, e: + raise MKGeneralException(_("Cannot load %s from %s: %s") % (what, path, e)) - # Repair visuals with missing 'title' or 'description' - visual.setdefault("title", name) - visual.setdefault("description", "") + return visuals - # Declare custom permissions - declare_visual_permission(what, name, visual) - visuals[(user, name)] = visual +def load_visuals_of_a_user(what, builtin_visuals, skip_func, lock, path, user): + user_visuals = {} + for name, visual in store.load_data_from_file(path, {}, lock).items(): + visual["owner"] = user + visual["name"] = name + if skip_func and skip_func(visual): + continue - except SyntaxError, e: - raise MKGeneralException(_("Cannot load %s from %s: %s") % (what, path, e)) + # Maybe resolve inherited attributes. This was a feature for several versions + # to make the visual texts localizable. This has been removed because the visual + # texts can now be localized using the custom localization strings. + # This is needed for backward compatibility to make the visuals without these + # attributes get the attributes from their builtin visual. + builtin_visual = builtin_visuals.get(name) + if builtin_visual: + for attr in [ 'title', 'linktitle', 'topic', 'description' ]: + if attr not in visual and attr in builtin_visual: + visual[attr] = builtin_visual[attr] + + # Repair visuals with missing 'title' or 'description' + visual.setdefault("title", name) + visual.setdefault("description", "") + + # Declare custom permissions + declare_visual_permission(what, name, visual) + + user_visuals[(user, name)] = visual + + return user_visuals - return visuals def declare_visual_permission(what, name, visual): permname = "%s.%s" % (what[:-1], name)

6 years, 11 months

Reserved 10 Werk IDS

by Jukka Aro

Module: check_mk Branch: master Commit: a832a09a024d74b183aeb0b8a257bdc392423380 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=a832a09a024d74… Author: Jukka Aro <ja(a)mathias-kettner.de> Date: Mon Oct 2 15:20:45 2017 +0200 Reserved 10 Werk IDS Change-Id: I042c774998e96af41ac4da2e6c5079a032bd64a0 --- .werks/first_free | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.werks/first_free b/.werks/first_free index 4a9e81e..e9e79fa 100644 --- a/.werks/first_free +++ b/.werks/first_free @@ -1 +1 @@ -5378 +5388

6 years, 11 months

Store: Fixed useless two times read of file in load_data_from_file()

by Lars Michelsen

Module: check_mk Branch: master Commit: 9ac380fd0419527cc4b09bc505c5957e83d0391c URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=9ac380fd041952… Author: Lars Michelsen <lm(a)mathias-kettner.de> Date: Mon Oct 2 14:58:16 2017 +0200 Store: Fixed useless two times read of file in load_data_from_file() Change-Id: Iaab5747a3b20d817e71ae57007309a25750ca9de --- lib/store.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/store.py b/lib/store.py index 08300f0..f88545d 100644 --- a/lib/store.py +++ b/lib/store.py @@ -116,7 +116,7 @@ def load_data_from_file(path, default=None, lock=False): # May be created empty during locking return default else: - return ast.literal_eval(file(path).read()) + return ast.literal_eval(content) except IOError, e: if e.errno == 2: # IOError: [Errno 2] No such file or directory return default

6 years, 11 months

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Checkmk git commits October 2017