[checkmk-commits] [tribe29/checkmk] 4bfc21: Do not save labels files pretty printed

Branch: refs/heads/1.6.0 Home: https://github.com/tribe29/checkmk Commit: 4bfc2109b3d111543e4cb0f9bab440cb1f2366bc https://github.com/tribe29/checkmk/commit/4bfc2109b3d111543e4cb0f9bab440cb1… Author: Lars Michelsen <lm(a)tribe29.com> Date: 2019-10-31 (Thu, 31 Oct 2019) Changed paths: M cmk/utils/labels.py Log Message: ----------- Do not save labels files pretty printed Change-Id: Id072865cfe964350108f0ba5f48dfe4e9c5ebe24 Commit: 7739a27ce8d5ea01ab9b2fe707a780910223eb7b https://github.com/tribe29/checkmk/commit/7739a27ce8d5ea01ab9b2fe707a780910… Author: Lars Michelsen <lm(a)tribe29.com> Date: 2019-10-31 (Thu, 31 Oct 2019) Changed paths: M cmk_base/ip_lookup.py Log Message: ----------- Faster IP lookup cache writing Storing the file in pretty printed mode takes a lot more time when working with larger cache files. We should really think about changing the default to pretty=False. Change-Id: Ic8a63a2259927f2231888aaf9051d65f2478f9b9 Commit: 230ffd7083cd6becce1dfce2d9064555af4ad912 https://github.com/tribe29/checkmk/commit/230ffd7083cd6becce1dfce2d9064555a… Author: Lars Michelsen <lm(a)tribe29.com> Date: 2019-10-31 (Thu, 31 Oct 2019) Changed paths: M cmk_base/ip_lookup.py M tests/unit/cmk_base/test_ip_lookup.py Log Message: ----------- Some more IP cache cleanups * Better name for cache object factory * Extract cache clearing logic * Minor fix: Also clean up the in-memory cache * Add test for clearing logic * Extract host/family enumeration * Add test for host/family enumeration Change-Id: I02232a0467d20d65dc9db1ddca334775a26a0522 Commit: f27c63ffe43405a94d349992c7480d0612b08c66 https://github.com/tribe29/checkmk/commit/f27c63ffe43405a94d349992c7480d061… Author: Lars Michelsen <lm(a)tribe29.com> Date: 2019-10-31 (Thu, 31 Oct 2019) Changed paths: M cmk_base/ip_lookup.py Log Message: ----------- Simplify host / family lookup computation Change-Id: I86debb91216d97f466a5cd165cc99e80c3f8e240 Commit: 31d1ee7746ab764f564f9e6d8a8fc12ebd003b6b https://github.com/tribe29/checkmk/commit/31d1ee7746ab764f564f9e6d8a8fc12eb… Author: Lars Michelsen <lm(a)tribe29.com> Date: 2019-10-31 (Thu, 31 Oct 2019) Changed paths: M cmk_base/caching.py M cmk_base/ip_lookup.py M tests/unit/cmk_base/test_ip_lookup.py M tests/unit/conftest.py Log Message: ----------- Improve "Update DNS cache" performance * The IPLookupCache is now managed by a special form of the DictCache which also cares about writing the persisted cache. This is not ideal, but seemed to be the least invasive way. * The cache updates the DNS cache for each individual lookup by default as it did it before. * But this object can now be set to "do not persist" mode to just capture the updates in memory during subsequent lookups like it's done in ip_lookup.update_dns_cache(). * After the lookups are done, the save() method can be called once to perform a single bulk operation The operation is now a 100 times faster (tested with ~2000 hosts and a fast responding DNS server. Made some cleanups on the way and added several tests for the affected functions. Change-Id: Ib927cff7556ae07348e457973a1e0c6fa355a2ef Commit: a095fe6d03a20a13d17c150e807e104d27389d28 https://github.com/tribe29/checkmk/commit/a095fe6d03a20a13d17c150e807e104d2… Author: Lars Michelsen <lm(a)tribe29.com> Date: 2019-11-01 (Fri, 01 Nov 2019) Changed paths: A .werks/10378 Log Message: ----------- 10378 FIX Improve "Update DNS cache" / cmk --update-dns-cache performance The operation was performing a lot of needless operations. The update of several thousands of hosts DNS entries took way too long. The exact time needed depends mostly on the number of hosts, cache size, DNS lookup performance and so on. But even with good responding servers the operation took too long, because the cache was updated in a very inefficient way when calling the "Update DNS cache" action in WATO or "cmk --update-dns-cache" from the command line. The operation is now more than a 100 times faster (tested with ~2000 hosts and a fast responding DNS server). Change-Id: Ie9a2cb589c7861934311043d1e8161a6de4d519e Compare: https://github.com/tribe29/checkmk/compare/6e590f27165d...a095fe6d03a2