Checkmk git commits April 2019

checkmk-commits@lists.checkmk.com

15 participants
599 discussions

Time slices selector and RRD datacolumn helper function

by Óscar Nájera

Module: check_mk Branch: master Commit: 1f9b5df726c147f26e04267f33f5896abc8e0fab URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=1f9b5df726c147… Author: Óscar Nájera <on(a)mathias-kettner.de> Date: Wed Apr 3 13:54:57 2019 +0200 Time slices selector and RRD datacolumn helper function CMK-1761 Change-Id: Icc7547bd9c9fb7d3637d27b5b760dfdb1174a91f --- cmk/utils/prediction.py | 9 ++++ cmk_base/prediction.py | 48 ++++++++++++-------- .../cmk_base/test_services_prediction.py | 15 +------ tests/testlib/__init__.py | 12 +++++ tests/unit/cmk_base/test_prediction.py | 51 ++++++++++++++++++++++ 5 files changed, 103 insertions(+), 32 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=1f9b5df726…

5 years, 3 months

Isolate calculation of data for prediction

by Óscar Nájera

Module: check_mk Branch: master Commit: dc83877e6110a66705a46543c238a5431ad9ee3b URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=dc83877e6110a6… Author: Óscar Nájera <on(a)mathias-kettner.de> Date: Tue Apr 2 17:50:04 2019 +0200 Isolate calculation of data for prediction - Determination of time windows happens in local time and in isolation from the livestatus query. Tests are moved in time to evaluate this. - Once unix timestamps are defined livestatus is queried. - Fix Integration test for predictive levels - New test just on retrieval and up-sampling - Add new time-zoned tests CMK-1761 Change-Id: I8610ac1e4179c574fd585e6702707841d8a2fdab --- cmk/utils/prediction.py | 3 +- cmk_base/prediction.py | 62 ++++++-------- .../cmk_base/test-files/America/New_York/everyday | 1 + .../cmk_base/test-files/{ => Europe/Berlin}/26 | 2 +- .../test-files/{ => Europe/Berlin}/26.info | 0 .../cmk_base/test-files/Europe/Berlin/everyday | 1 + .../test-files/{ => Europe/Berlin}/everyday.info | 0 .../test-files/{ => Europe/Berlin}/thursday | 2 +- .../test-files/{ => Europe/Berlin}/thursday.info | 0 tests/integration/cmk_base/test-files/UTC/sunday | 1 + tests/integration/cmk_base/test-files/everyday | 1 - .../cmk_base/test_services_prediction.py | 98 ++++++++++++++++++---- 12 files changed, 115 insertions(+), 56 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=dc83877e61…

5 years, 3 months

Use TimeSeries class to retrieve and work with RRDdata

by Óscar Nájera

Module: check_mk Branch: master Commit: 33bc1508975f19ea18167ac438ca0cbcf8bff7a3 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=33bc1508975f19… Author: Óscar Nájera <on(a)mathias-kettner.de> Date: Tue Apr 2 16:32:35 2019 +0200 Use TimeSeries class to retrieve and work with RRDdata - Now that time series can be up-sampled consolidation does not need to deal with re-scaling arrays of data for alignment of time series. - Refactor RRDdata consolidation in predictive levels CMK-1761 Change-Id: Ic03fa75b87bbfff4b7ab5afd8999d48e93335145 --- cmk/gui/prediction.py | 5 +- cmk/utils/prediction.py | 28 +++++--- cmk_base/prediction.py | 78 +++++++++------------- .../cmk_base/test_services_prediction.py | 21 +++--- tests/unit/cmk_base/test_prediction.py | 24 +++---- 5 files changed, 77 insertions(+), 79 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=33bc150897…

5 years, 3 months

Time windows

by Óscar Nájera

Module: check_mk Branch: master Commit: 3453a2246914b1a012dc5d0fd5fcf6a09cd841f7 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=3453a2246914b1… Author: Óscar Nájera <on(a)mathias-kettner.de> Date: Wed Apr 3 17:18:31 2019 +0200 Time windows Change-Id: I76b94d88007e8d8a11553aab52329e9244cdaf3a --- cmk_base/prediction.py | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/cmk_base/prediction.py b/cmk_base/prediction.py index 6bd2c36..10c266d 100644 --- a/cmk_base/prediction.py +++ b/cmk_base/prediction.py @@ -39,30 +39,29 @@ import cmk.utils.prediction logger = cmk.utils.log.get_logger(__name__) -def day_start(timestamp): - return (timestamp - cmk.utils.prediction.timezone_at(timestamp)) % 86400 +def window_start(timestamp, span): + """If time is partitioned in SPAN intervals, how many seconds is TIMESTAMP away from the start - -def hour_start(timestamp): - return (timestamp - cmk.utils.prediction.timezone_at(timestamp)) % 3600 + It works well across time zones, but has an unfair behavior with daylight savings time.""" + return (timestamp - cmk.utils.prediction.timezone_at(timestamp)) % span def group_by_wday(t): wday = time.localtime(t).tm_wday - return defines.weekday_ids()[wday], day_start(t) + return defines.weekday_ids()[wday], window_start(t, 86400) def group_by_day(t): - return "everyday", day_start(t) + return "everyday", window_start(t, 86400) def group_by_day_of_month(t): mday = time.localtime(t).tm_mday - return str(mday), day_start(t) + return str(mday), window_start(t, 86400) def group_by_everyhour(t): - return "everyhour", hour_start(t) + return "everyhour", window_start(t, 3600) prediction_periods = { @@ -101,7 +100,7 @@ def get_prediction_timegroup(t, period_info): # Convert to local timezone timegroup, rel_time = period_info["groupby"](t) from_time = t - rel_time - until_time = t - rel_time + period_info["slice"] + until_time = from_time + period_info["slice"] return timegroup, from_time, until_time, rel_time

5 years, 3 months

New TimeSeries class to describe RRDdata

by Óscar Nájera

Module: check_mk Branch: master Commit: 67759e66b5edef4b7006fc3bc6a8dcd3376bd135 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=67759e66b5edef… Author: Óscar Nájera <on(a)mathias-kettner.de> Date: Tue Apr 2 16:19:33 2019 +0200 New TimeSeries class to describe RRDdata This class allows to track performance data and binds to each value the measurement time stamp. It allows for up-sampling data by backward filling values CMK-1761 Change-Id: I81646be8f009cd1bbd4efec0ca9c298abe002c4d --- cmk/utils/prediction.py | 60 +++++++++++++++++++++++++ tests/unit/cmk/utils/test_prediction_commons.py | 20 +++++++++ 2 files changed, 80 insertions(+) diff --git a/cmk/utils/prediction.py b/cmk/utils/prediction.py index cdd1b4c..4c79d3e 100644 --- a/cmk/utils/prediction.py +++ b/cmk/utils/prediction.py @@ -51,6 +51,66 @@ def timezone_at(timestamp): return time.timezone +def rrd_timestamps(twindow): + start, end, step = twindow + if step == 0: + return [] + return [t + step for t in range(start, end, step)] + + +class TimeSeries(object): + """Describes the returned time series returned by livestatus + + - Timestamped values are valid for the measurement interval: + [timestamp-step; timestamp[ + which means they are at the end of the interval. + - The Series describes the interval [start; end[ + - Start has no associated value to it. + """ + + def __init__(self, data): + self.start = data[0] + self.end = data[1] + self.step = data[2] + self.values = data[3:] + + @property + def twindow(self): + return self.start, self.end, self.step + + def bfill_upsample(self, twindow, shift): + """Upsample by backward filling values + + twindow : 3-tuple, (start, end, step) + description of target time interval + """ + upsa = [] + i = 0 + start, end, step = twindow + current_times = rrd_timestamps(self.twindow) + if start != self.start or end != self.end or step != self.step: + for t in range(start, end, step): + if t >= current_times[i] + shift: + i += 1 + upsa.append(self.values[i]) + + return upsa + + return self.values + + def time_data_pairs(self): + return list(zip(rrd_timestamps(self.twindow), self.values)) + + def __repr__(self): + return str(list(self.twindow) + self.values) + + def __eq__(self, other): + if not isinstance(other, TimeSeries): + return NotImplemented + + return self.start == other.start and self.end == other.end and self.step == other.step and self.values == other.values + + def get_rrd_data(hostname, service_description, varname, cf, fromtime, untiltime): """Fetch RRD historic metrics data of a specific service, within the specified time range diff --git a/tests/unit/cmk/utils/test_prediction_commons.py b/tests/unit/cmk/utils/test_prediction_commons.py index f538c3c..b15ed07 100644 --- a/tests/unit/cmk/utils/test_prediction_commons.py +++ b/tests/unit/cmk/utils/test_prediction_commons.py @@ -3,6 +3,26 @@ import pytest import cmk.utils.prediction as prediction +(a)pytest.mark.parametrize("twindow, result", [((0, 0, 0), []), + ((100, 200, 25), [125, 150, 175, 200])]) +def test_rrdtimestamps(twindow, result): + assert prediction.rrd_timestamps(twindow) == result + + +(a)pytest.mark.parametrize("rrddata, twindow, shift, upsampled", [ + ([10, 20, 10, 20], (10, 20, 10), 0, [20]), + ([10, 20, 10, 20], (10, 20, 5), 0, [20, 20]), + ([10, 20, 10, 20], (20, 30, 5), 10, [20, 20]), + ([0, 120, 40, 25, 65, 105], (300, 400, 10), 300, [25, 25, 25, 25, 65, 65, 65, 65, 105, 105]), + ([0, 120, 40, 25, None, 105], + (300, 400, 10), 300, [25, 25, 25, 25, None, None, None, None, 105, 105]), + ([0, 120, 40, 25, 65, 105], (330, 410, 10), 300, [25, 65, 65, 65, 65, 105, 105, 105]), +]) +def test_time_series_upsampling(rrddata, twindow, shift, upsampled): + ts = prediction.TimeSeries(rrddata) + assert ts.bfill_upsample(twindow, shift) == upsampled + + @pytest.mark.parametrize("ref_value, stdev, sig, params, levels_factor, result", [ (2, 0.5, 1, ("absolute", (3, 5)), 0.5, (3.5, 4.5)), (2, 0.5, -1, ("relative", (20, 50)), 0.5, (1.6, 1)),

5 years, 3 months

Fix precedence when cleaning prediction files

by Óscar Nájera

Module: check_mk Branch: master Commit: b9d15775f717ec756df35c84db02abcb73d2e684 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=b9d15775f717ec… Author: Óscar Nájera <on(a)mathias-kettner.de> Date: Thu Apr 4 12:04:26 2019 +0200 Fix precedence when cleaning prediction files Change-Id: I3921a0e09a4918cd8124ec193cdb27f0c13c94d6 --- cmk/utils/prediction.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmk/utils/prediction.py b/cmk/utils/prediction.py index 64188f2..8aab80d 100644 --- a/cmk/utils/prediction.py +++ b/cmk/utils/prediction.py @@ -187,7 +187,7 @@ def clean_prediction_files(pred_file, force=False): # which was never handled correctly so that the prediction could never be used again until # manual removal of the files. Clean this up. for file_path in [pred_file, pred_file + '.info']: - if os.path.exists(file_path) and os.stat(file_path).st_size == 0 or force: + if os.path.exists(file_path) and (os.stat(file_path).st_size == 0 or force): logger.verbose("Removing obsolete prediction %s", os.path.basename(file_path)) os.remove(file_path)

5 years, 3 months

[CMK-1890] - automatic installer of MSI loaded to the ProgramData/update

by Sergey Kipnis

Module: check_mk Branch: master Commit: 085102cbb29d468779f13f2f076adebc5652aa2e URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=085102cbb29d46… Author: Sergey Kipnis <sk(a)mathias-kettner.de> Date: Wed Apr 3 10:57:32 2019 +0200 [CMK-1890] - automatic installer of MSI loaded to the ProgramData/update - install_api.* keeps all functionality required(partially moved from service_apu.*) - service_api.* left as an empty stubs to be removed - full Unit Test support of both Top Level and Low Level functionality - revamped service main to be compatible with API - diagnostic extended to support new api and new logic - update to be tested MANUALLY. To be changed with upcoming commits - install API fixed according to C++ guidelines Change-Id: I8745858b3f160661019d120e361da15ff4c597b0 --- agents/wnx/install/resources/check_mk.dat | 2 +- agents/wnx/src/engine/cfg.cpp | 5 + agents/wnx/src/engine/cfg.h | 5 +- agents/wnx/src/engine/engine.vcxproj | 2 + agents/wnx/src/engine/engine.vcxproj.filters | 6 + agents/wnx/src/engine/install_api.cpp | 198 ++++++++++++++++++++++++++ agents/wnx/src/engine/install_api.h | 53 +++++++ agents/wnx/src/engine/service_api.cpp | 109 +------------- agents/wnx/src/engine/service_api.h | 21 +-- agents/wnx/src/engine/windows_service_api.cpp | 40 +++--- agents/wnx/src/main/check_mk_service.cpp | 35 +++-- agents/wnx/watest/test-install-auto.cpp | 105 ++++++++++++++ agents/wnx/watest/test-service.cpp | 40 ------ agents/wnx/watest/watest.vcxproj | 1 + agents/wnx/watest/watest.vcxproj.filters | 3 + 15 files changed, 426 insertions(+), 199 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=085102cbb2…

5 years, 3 months

Fixed macro expansion in in Livestatus.

by Sven Panne

Module: check_mk Branch: master Commit: 93f08352f54306634fff74b60912a44a7d108b9a URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=93f08352f54306… Author: Sven Panne <sp(a)mathias-kettner.de> Date: Thu Apr 4 10:12:30 2019 +0200 Fixed macro expansion in in Livestatus. Unterminated dollar signs were handled incorrectly: The part of the text before such a dollar sign was repeated, e.g. "foo$bar" was incorreclty expanded to "foofoo$bar", although it should remain unchanged. CMK-1925 Change-Id: If18c9083ffeeff42c8f4e52a748e43f32a959347 --- livestatus/src/OffsetStringMacroColumn.cc | 2 +- livestatus/src/test/test_OffsetStringMacroColumn.cc | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/livestatus/src/OffsetStringMacroColumn.cc b/livestatus/src/OffsetStringMacroColumn.cc index e255474..e902b76 100644 --- a/livestatus/src/OffsetStringMacroColumn.cc +++ b/livestatus/src/OffsetStringMacroColumn.cc @@ -56,7 +56,7 @@ std::string OffsetStringMacroColumn::expandMacros(const std::string &raw, result += std::string(scan, dollar - scan); const char *otherdollar = strchr(dollar + 1, '$'); if (otherdollar == nullptr) { // unterminated macro, do not expand - result += scan; + result += dollar; break; } std::string macroname = diff --git a/livestatus/src/test/test_OffsetStringMacroColumn.cc b/livestatus/src/test/test_OffsetStringMacroColumn.cc index ea5e8c6..b98766c 100644 --- a/livestatus/src/test/test_OffsetStringMacroColumn.cc +++ b/livestatus/src/test/test_OffsetStringMacroColumn.cc @@ -202,7 +202,7 @@ TEST_F(OffsetStringHostMacroColumnTest, border_cases) { EXPECT_EQ("$", expanded_host_notes()); set_host_notes("foo$bar"); - EXPECT_EQ("foofoo$bar", expanded_host_notes()); // TODO(sp) WRONG!!! + EXPECT_EQ("foo$bar", expanded_host_notes()); set_host_notes("checking $USER0$..."); EXPECT_EQ("checking $USER0$...", expanded_host_notes()); @@ -319,7 +319,7 @@ TEST_F(OffsetStringServiceMacroColumnTest, border_cases) { EXPECT_EQ("$", expanded_service_notes()); set_service_notes("foo$bar"); - EXPECT_EQ("foofoo$bar", expanded_service_notes()); // TODO(sp) WRONG!!! + EXPECT_EQ("foo$bar", expanded_service_notes()); set_service_notes("checking $USER0$..."); EXPECT_EQ("checking $USER0$...", expanded_service_notes());

5 years, 3 months

6970 FIX Fixed macro expansion in Livestatus

by Sven Panne

Module: check_mk Branch: master Commit: 7672a969d4e5ddd1cefdae3a077f31a2a7ae1c0c URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=7672a969d4e5dd… Author: Sven Panne <sp(a)mathias-kettner.de> Date: Thu Apr 4 11:19:35 2019 +0200 6970 FIX Fixed macro expansion in Livestatus Unterminated dollar signs were handled incorrectly: The part of the text before such a dollar sign was repeated, e.g. "foo$bar" was incorreclty expanded to "foofoo$bar", although it should remain unchanged. This affected only the Livestatus NEB, not the CMC. CMK-1925 Change-Id: If4afe3c9a1f2d702877f008b1ecd6ae0865c95d9 --- .werks/6970 | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/.werks/6970 b/.werks/6970 new file mode 100644 index 0000000..931ec07 --- /dev/null +++ b/.werks/6970 @@ -0,0 +1,15 @@ +Title: Fixed macro expansion in Livestatus +Level: 1 +Component: livestatus +Compatible: compat +Edition: cre +Version: 1.6.0i1 +Date: 1554369501 +Class: fix + +Unterminated dollar signs were handled incorrectly: The part of the text +before such a dollar sign was repeated, e.g. "foo$bar" was incorreclty +expanded to "foofoo$bar", although it should remain unchanged. This +affected only the Livestatus NEB, not the CMC. + +CMK-1925

5 years, 3 months

[CMK-1850] - msi-update now supports new MSI file from Windows build machine

by Konstantin Büttner

Module: check_mk Branch: master Commit: 5daeaebfa7e40124cd6358acc193653be2dc6ad5 URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=5daeaebfa7e401… Author: Konstantin Büttner <kb(a)mathias-kettner.de> Date: Wed Apr 3 15:56:10 2019 +0200 [CMK-1850] - msi-update now supports new MSI file from Windows build machine - old msi-update is saved as msi-update-legacy - build script makes baked_container.msi too - and yapf Change-Id: I24c5b47b3a7a50206ef3e308fa485a51cd0eb7a8 --- agents/windows/msibuild/msi-update | 184 ++++++++++++-------- agents/windows/msibuild/msi-update-legacy | 277 ++++++++++++++++++++++++++++++ agents/wnx/build_release.cmd | 1 + 3 files changed, 387 insertions(+), 75 deletions(-) Diff: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commitdiff;h=5daeaebfa7…

5 years, 3 months

← Newer
1
...
45
46
47
48
49
50
51
...
60
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

Checkmk git commits April 2019