[checkmk-commits] [tribe29/checkmk] 7525e4: python3 modules: No verbose copying anymore

Branch: refs/heads/master Home: https://github.com/tribe29/checkmk Commit: 7525e4dac6ca89ba928b62724653307d1f56816b https://github.com/tribe29/checkmk/commit/7525e4dac6ca89ba928b62724653307d1… Author: Lars Michelsen <lm(a)tribe29.com> Date: 2022-11-30 (Wed, 30 Nov 2022) Changed paths: M omd/packages/python3-modules/python3-modules.make Log Message: ----------- python3 modules: No verbose copying anymore Change-Id: I206ce807c59cf9ff023db932c23fe845ab2ef1aa Commit: d9bc1d8a9f96cf2b58a57ca9d39015ab6130524d https://github.com/tribe29/checkmk/commit/d9bc1d8a9f96cf2b58a57ca9d39015ab6… Author: Lars Michelsen <lm(a)tribe29.com> Date: 2022-11-30 (Wed, 30 Nov 2022) Changed paths: M omd/packages/python3-modules/python3-modules.make Log Message: ----------- Reduce extremely high load peaks when rebuilding python3-modules We build the different distros in parallel already. Now, when we do too much parallelization in the individual jobs, we create too much load on our build nodes. The grpcio package is hacking the setup.py logic to enable parallel compilation. By default it uses multiprocessing.cpu_count(), like you can see here: https://github.com/grpc/grpc/blob/master/src/python/grpcio/_parallel_compil… This is way too much for our scenario since it multiplies with the number of concurrent distros being built. Some quick measurement on my local machine: time python setup.py build (...) real 2m20,337s user 24m50,186s sys 1m27,513s time GRPC_PYTHON_BUILD_EXT_COMPILER_JOBS=4 python setup.py build (...) real 3m44,693s user 13m1,628s sys 0m48,248s time GRPC_PYTHON_BUILD_EXT_COMPILER_JOBS=2 python setup.py build (...) real 6m52,043s user 12m11,269s sys 0m45,709s A limit of 4 seems to be a good value at least on my machine. Let's see how it works out in the CI. CMK-11767 Change-Id: I005ae4d5bb21760b5e50b1e2ba069da1aa403290 Compare: https://github.com/tribe29/checkmk/compare/c2b3439f25b2...d9bc1d8a9f96