[checkmk-commits] [Checkmk/checkmk] 896f52: Pass down spec #1

Branch: refs/heads/master Home: https://github.com/Checkmk/checkmk Commit: 896f5230aa24a4749bcd802b0356fbea80fd5d95 https://github.com/Checkmk/checkmk/commit/896f5230aa24a4749bcd802b0356fbea8… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/_openapi.py M cmk/gui/openapi/restful_objects/decorators.py M cmk/gui/wsgi/applications/rest_api.py Log Message: ----------- Pass down spec #1 The end goal is to get rid of the global SPEC object. This is the first of a few steps to get there. Change-Id: I3c594933cbcd220def3bb4463c670eeb98180b39 Commit: cda0988204cce16a53331662b6ff460ed44ee714 https://github.com/Checkmk/checkmk/commit/cda0988204cce16a53331662b6ff460ed… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/__main__.py M cmk/gui/openapi/_openapi.py M cmk/gui/openapi/restful_objects/type_defs.py M cmk/gui/wsgi/applications/rest_api.py Log Message: ----------- Pass down spec #2 The end goal is to get rid of the global SPEC object. This is the first of a few steps to get there. Fix a few typing issues along the way. They were not visible before because the global SPEC is of type Any :-(. Change-Id: Iaaefddd73473d24cfc674a099066155c748db5a3 Commit: 855740886219e6535df31992ddfad0af7b201cc8 https://github.com/Checkmk/checkmk/commit/855740886219e6535df31992ddfad0af7… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/restful_objects/code_examples.py M cmk/gui/openapi/restful_objects/decorators.py Log Message: ----------- Pass down spec #3 The end goal is to get rid of the global SPEC object. This is the first of a few steps to get there. Change-Id: Ie6c954dfe24a5846728c69b49c33de6965e9a36a Commit: add6ccf8abf181e4b3cfd3c6bf52a50014988f54 https://github.com/Checkmk/checkmk/commit/add6ccf8abf181e4b3cfd3c6bf52a5001… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/__init__.py M cmk/gui/openapi/__main__.py M cmk/gui/openapi/_openapi.py M cmk/gui/wsgi/applications/rest_api.py Log Message: ----------- Clarify function name Change-Id: Ie215d0a42873b6fe4671023c32f751a15fd16a0c Commit: 88e68e247ebebb891fc5c5dca013954dc6fcc623 https://github.com/Checkmk/checkmk/commit/88e68e247ebebb891fc5c5dca013954dc… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/__main__.py M cmk/gui/openapi/_openapi.py M cmk/gui/openapi/restful_objects/__init__.py M cmk/gui/openapi/restful_objects/code_examples.py M cmk/gui/openapi/restful_objects/specification.py M cmk/gui/wsgi/applications/rest_api.py Log Message: ----------- Clean up global SPEC instance There is just one call site now, so we can get rid of the global state. Change-Id: Ia2a463539a3b3f26952b41e1537eabfe68623ba0 Commit: 0600caa5c38c9a47a52837786240e1904fde6a85 https://github.com/Checkmk/checkmk/commit/0600caa5c38c9a47a52837786240e1904… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/__init__.py M cmk/gui/openapi/__main__.py R cmk/gui/openapi/_openapi.py A cmk/gui/openapi/spec_generator.py M cmk/gui/wsgi/applications/rest_api.py Log Message: ----------- Move spec generating code to dedicated module The end goal is to extract the whole spec computation out of the GUI application. The intermediate target is to isolate all the code we need to generate the spec in a dedicated module which is also only imported when needed. This change moves the generic piece of the code to a new module where we will move a few other pieces to in the folloing commits. CMK-15981 Change-Id: I77dccc7419305049dc1e95486fb4d106435f5f4f Commit: 3c15efabe826e812a609c3608018336c0a3f120f https://github.com/Checkmk/checkmk/commit/3c15efabe826e812a609c3608018336c0… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/restful_objects/decorators.py M cmk/gui/openapi/spec_generator.py Log Message: ----------- Move endpoint spec generating code to spec_generator CMK-15981 Change-Id: If1cdfa0175a13b9f2bbd0e39436d599f42c29135 Commit: 7c573eb3a62a175a5f1c24b98e72b5a6c0d32d66 https://github.com/Checkmk/checkmk/commit/7c573eb3a62a175a5f1c24b98e72b5a6c… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: R cmk/gui/openapi/__main__.py M cmk/gui/openapi/spec_generator.py M cmk/gui/wsgi/applications/rest_api.py Log Message: ----------- Consolidate more spec serving logic in serve_spec This step makes it easier to separate the generation of the spec from serving the spec later on. CMK-15981 Change-Id: Ief08d474eeb2f0ceef3cabd4d4b09c3fd9a5c92e Commit: 4aad48a3ed95dffafe0b4ecefcc185116720d217 https://github.com/Checkmk/checkmk/commit/4aad48a3ed95dffafe0b4ecefcc185116… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/spec_generator.py M cmk/gui/wsgi/applications/rest_api.py Log Message: ----------- Remove a depdendency on request environment from spec generation This step makes it easier to separate the generation of the spec from serving the spec later on. CMK-15981 Change-Id: I2279a2b3664b5f303c55318bbdae70fb3161ead4 Commit: 6024ee5d10d2fd9e23bab5d64904b95c775d7898 https://github.com/Checkmk/checkmk/commit/6024ee5d10d2fd9e23bab5d64904b95c7… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/restful_objects/type_defs.py Log Message: ----------- Remove unused target CMK-15981 Change-Id: I0db78ae28e08a4c751bcea5b24715a4e97af8998 Commit: 4fa46d656ca6c2063440efd877e64935b02308bc https://github.com/Checkmk/checkmk/commit/4fa46d656ca6c2063440efd877e64935b… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M bin/.f12 A bin/cmk-compute-api-spec M cmk/gui/openapi/restful_objects/specification.py M cmk/gui/openapi/spec_generator.py M cmk/gui/wsgi/applications/rest_api.py M tests/testlib/pylint_checker_cmk_module_layers.py Log Message: ----------- Add dedicated command to compute REST API spec: cmk-compute-api-spec Previously the REST API specification was computed and cached in every apache process. This caused several issues: a) After spawning a new apache, the spec needed to be recomputed for every process. This caused a delay in the first request hitting an apache process asking for it. b) It was held in memory by every process consuming a few MB. c) The invalidation of the cache and computation of new spec could not be triggered manually. With this change the spec is now stored in the site and made available to all apache processes from there. With the dedicated command `cmk-compute-api-spec` the computation can now be triggered in specific situations automatically or manually for debugging. CMK-15981 Change-Id: I7c7a04b15d7715024f4ef213689edbf0ad630795 Commit: 70fb282764406d589211205526659e0a12bc0d7a https://github.com/Checkmk/checkmk/commit/70fb282764406d589211205526659e0a1… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/wsgi/applications/rest_api.py Log Message: ----------- Drop dead code Change-Id: Ib8b244e26b3a6694a4034619fae5d1a2f0532693 Commit: cceca10ef530e0f6fc9331144569f794abeb297d https://github.com/Checkmk/checkmk/commit/cceca10ef530e0f6fc9331144569f794a… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: A tests/integration/cmk/gui/openapi/test_spec_generator.py Log Message: ----------- Add integration test to ensure cmk-compute-api-spec works CMK-15981 Change-Id: Icc64b14bf029d64db6b24e2a2ccea8ab5766833d Commit: 20a97f9af177a3106bcfa4a522902b44a77db1a9 https://github.com/Checkmk/checkmk/commit/20a97f9af177a3106bcfa4a522902b44a… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: A cmk/post_rename_site/plugins/actions/compute_api_spec.py A cmk/update_config/plugins/actions/compute_api_spec.py M omd/packages/check_mk/check_mk.make A omd/packages/check_mk/post-create/02_cmk-compute-api-spec M tests/unit/cmk/post_rename_site/test_load_plugins.py Log Message: ----------- Execute the API spec generation in various situations This change registers the new cmk-compose-api-spec command into various events happening during the lifecycle of a site. - post-create hook: Create the initial spec after a site has been created - post rename action: Update the spec after a site has been copied, restored or renamed - update-config action: Update the spec after Checkmk has been updated It might be necessary to run the command in some other situations. We'll have to find out about those along the way. CMK-15981 Change-Id: I29eeffd60ec11a797f55d103a8a86fd74fdf3935 Commit: b6624c2eef52ccdddd5a1cad0e3b56efe09fad19 https://github.com/Checkmk/checkmk/commit/b6624c2eef52ccdddd5a1cad0e3b56efe… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: A .werks/15724.md Log Message: ----------- 15724 FIX Change API specification computation The specification of the REST API defines the structure of the API. It is computed automatically from the implementation in Checkmk. Previously the specification was computed during runtime when something requested access to the specification. This could be a user opening ReDoc or the Swagger UI. The specification was then computed ad-hoc and cached in the memory of the apache process. This caused several issues: * After spawning a new apache, the specification needed to be recomputed for every process. This caused a delay in the first request hitting an apache process asking for it. * It was held in memory by every process consuming a few MB. * The invalidation of the cache and computation of new specification could not be triggered manually. With this change the specification is now stored in the site and made available to all apache processes from there. With the dedicated command `cmk-compute-api-spec` the computation can now be triggered in specific situations automatically or manually for debugging. The specification is now updated in these situations: * post-create hook: Create the initial spec after a site has been created * post rename action: Update the spec after a site has been copied, restored or renamed * update-config action: Update the spec after the site has been updated CMK-15981 Change-Id: Ic524bf8146640c66919fd49653b805c981f47771 Commit: 4d48980ecd07c1c7e0681275c8e6c50fd4cedd86 https://github.com/Checkmk/checkmk/commit/4d48980ecd07c1c7e0681275c8e6c50fd… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/wsgi/applications/rest_api.py Log Message: ----------- Improve spec serving performance by using a cache The final spec is now cached in the tmpfs of the site. This way the API UIs can be rendered even faster. The cache is invalidated once a new specification is available. CMK-15981 Change-Id: I71c4aa43a0e631f6a2022ec2e7905993941db47d Commit: 5e00598b9367018ec18e5ab0a301193401c910de https://github.com/Checkmk/checkmk/commit/5e00598b9367018ec18e5ab0a30119340… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/wato/pages/tags.py Log Message: ----------- Consistently use shared helper for editing tags CMK-15981 Change-Id: I066da0f9d5de5942a21386a45d96bb49035d4c5b Commit: 378e74ff4c28d17f083dfe42f840e77f9777365d https://github.com/Checkmk/checkmk/commit/378e74ff4c28d17f083dfe42f840e77f9… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/registration.py A cmk/gui/openapi/spec_generator_job.py M cmk/gui/plugins/main_modules/registration.py M cmk/gui/watolib/tags.py M tests/unit/cmk/gui/test_background_job.py Log Message: ----------- Background job for spec creation triggered by UI actions We need to trigger an update of the spec to reflect changes made by previous config steps in the API specification. Since the computation takes some time, this must not be done synchronously. We need to be able to track the outcome of the job and be able to analyze the progress of the job through the UI. So it makes sense to realize this as background job. The job shall be triggered by every change to the configuration which has an effect on the API specification. The first place we trigger this job is the configuration of tags. Change-Id: Ie595552cd79eb3c38abdfa594b91ac7c99ac9330 Commit: 464c39957c29336b35eb8b9c349209e6da5406a8 https://github.com/Checkmk/checkmk/commit/464c39957c29336b35eb8b9c349209e6d… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/spec_generator.py M tests/unit/cmk/gui/openapi/test_spec_files.py Log Message: ----------- Execute spec endpoint tests with dummy spec Previously these tests were meant to ensure the endpoints work and whether the API spec can be computed. This lead to long execution times. Since the spec is now generated separately, we can also execute the tests with a dummy spec and can remove the slow marker. For the computation of the spec in the site environment we have an integration test in place. Of course, this is not ideal for the development workflow. So we need to find a way to test the spec computation in a subsequent step. Cleanup `validate_spec` related warnings along the way. CMK-15981 Change-Id: Id8a642fa4b5aaa55beff9a87df76a708e7b14308 Commit: 54210e93ab46dcaa917e01291b58949a5cb42585 https://github.com/Checkmk/checkmk/commit/54210e93ab46dcaa917e01291b58949a5… Author: Lars Michelsen <lm(a)checkmk.com> Date: 2024-02-21 (Wed, 21 Feb 2024) Changed paths: M cmk/gui/openapi/spec_generator.py Log Message: ----------- Try to get rid of the deepcopy It is unclear whether it is still needed. So let's try to get rid of it. CMK-15981 Change-Id: I005cbea7b8b8078a0c07c8c7cffca12ceb659a3d Compare: https://github.com/Checkmk/checkmk/compare/4f3b00e8e35b...54210e93ab46 To unsubscribe from these emails, change your notification settings at https://github.com/Checkmk/checkmk/settings/notifications