[checkmk-commits] [tribe29/checkmk] b8eeb1: Draft batch processing for InfluxDB

Branch: refs/heads/2.1.0 Home: https://github.com/tribe29/checkmk Commit: b8eeb187c3aaad95c19f6d851a51fa41264be726 https://github.com/tribe29/checkmk/commit/b8eeb187c3aaad95c19f6d851a51fa412… Author: Mathias Laurin <mathias.laurin(a)tribe29.com> Date: 2023-01-26 (Thu, 26 Jan 2023) Changed paths: M cmk/base/.f12 Log Message: ----------- Draft batch processing for InfluxDB This patch configures and performs batch processing. In order for the feature to work correctly, however, we need to wake up the helper more often or the "batch_max_duration" will not behave as expected on small installations with the default 60 seconds check interval. Now: t0 : check -> buffer metrics t0 + 1 min : check -> send previous batch (older than batch_max_duration) and buffer new metrics t0 + 2 min : ... Expected: t0 : check -> buffer metrics t0 + 10 sec : send t0 + 1 min : check -> buffer metrics + 10 sec : send CMK-11427 Change-Id: I3c1f62433155a6bfd27a5b9b6e4578cdd3d1a4bc Commit: 430f553a015f1e13bb9e783d0cabe2d577b09f36 https://github.com/tribe29/checkmk/commit/430f553a015f1e13bb9e783d0cabe2d57… Author: Mathias Laurin <mathias.laurin(a)tribe29.com> Date: 2023-01-26 (Thu, 26 Jan 2023) Changed paths: M livestatus/src/Queue.h M livestatus/src/module.cc M livestatus/src/test/test_Queue.cc Log Message: ----------- Merge Queue:pop and Queue:try_pop and add timeout CMK-11860 Change-Id: Ia255aee68c77a4950c440c824fe8cc855a6fd44d Commit: e5d695b6fbd12116f215837a99a95d516dcd1cf5 https://github.com/tribe29/checkmk/commit/e5d695b6fbd12116f215837a99a95d516… Author: Mathias Laurin <mathias.laurin(a)tribe29.com> Date: 2023-01-26 (Thu, 26 Jan 2023) Changed paths: M livestatus/src/Queue.h Log Message: ----------- Call user-defined function when loop is idle This lets us add some housekeeping (such as the batch handling of data for influx db) in DataExportThread. CMK-11860 Change-Id: I2c60be645b4024881784ee2f80268e9745a1dd79 Commit: 9ce85521ea9b3858167273b708519c2103b17985 https://github.com/tribe29/checkmk/commit/9ce85521ea9b3858167273b708519c210… Author: Mathias Laurin <mathias.laurin(a)tribe29.com> Date: 2023-01-26 (Thu, 26 Jan 2023) Changed paths: M omd/Licenses.csv A omd/packages/httplib/cpp-httplib-0.11.3.tar.gz R omd/packages/httplib/cpp-httplib-0.8.9.tar.gz Log Message: ----------- Upgrade cpp-httplib from 0.8.9 to 0.11.3 Change-Id: Ia7ea5b7445ca2b04b455c5ea5b20af83d9e05d31 Commit: bf1a4ae9f73262329d4ed91aa680e524575bcedb https://github.com/tribe29/checkmk/commit/bf1a4ae9f73262329d4ed91aa680e5245… Author: Mathias Laurin <mathias.laurin(a)tribe29.com> Date: 2023-01-26 (Thu, 26 Jan 2023) Changed paths: A .werks/15157 Log Message: ----------- 15157 Zip metrics sent to InfluxDB The batch of metrics from Werk 15156 is now compressed before it is sent. This follows the "Optimize writes to InfluxDB" best practice documented on influxdata.com and should improve the overall performance of the conenctor. CMK-11861 Change-Id: I36921f28a8ccce05f08c160e79e9b64de9d4cd45 Commit: a6471cbd33cae4a5cd48c1caa5450564d02d36cc https://github.com/tribe29/checkmk/commit/a6471cbd33cae4a5cd48c1caa5450564d… Author: Mathias Laurin <mathias.laurin(a)tribe29.com> Date: 2023-01-26 (Thu, 26 Jan 2023) Changed paths: A .werks/15156 Log Message: ----------- 15156 Send metrics to InfluxDB in batch Metrics are collected and sent together after either about 5000 metrics have been enqueued or 10 seconds, whichever comes first. This follows the "Optimize writes to InfluxDB" best practice documented on influxdata.com and should improve the overall performance of the connector. Change-Id: Icb89215101255305843ae751cf3f8ed151f73249 Compare: https://github.com/tribe29/checkmk/compare/b8d0b5c26adc...a6471cbd33ca