leaving sitecustomize.pyc)
Message-ID: <5afd7146.5TWHjjFuWE7zdHkT%lm(a)mathias-kettner.de>
User-Agent: Heirloom mailx 12.5 6/20/10
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Module: check_mk
Branch: master
Commit: 62f538266514779ee6dd1db2086e28b1c5c7dd3f
URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=62f53826651477…
Author: Lars Michelsen <lm(a)mathias-kettner.de>
Date: Thu May 17 14:05:43 2018 +0200
6101 FIX Fixed partial uninstall of Check_MK 1.5 versions (leaving sitecustomize.pyc)
The python delivered with Check_MK was not precompiling the lib/python2.7/sitecustomize.pyc
during packaging. It was then created during runtime on the target system. Since this file
is not under control of the RPM/DEB packages the file was left on the filesystem after
uninstall. The version was still visible after removal in "omd version" output until one
removed the /omd/versions/[version] directory.
Change-Id: Ie7eb6e4b68589f598dfd7c654f2de87c7cc8ff15
---
.werks/6101 | 15 +++++++++++++++
omd/packages/python/.gitignore | 1 +
omd/packages/python/Makefile | 11 +++++++++--
3 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/.werks/6101 b/.werks/6101
new file mode 100644
index 0000000..30f2f99
--- /dev/null
+++ b/.werks/6101
@@ -0,0 +1,15 @@
+Title: Fixed partial uninstall of Check_MK 1.5 versions (leaving sitecustomize.pyc)
+Level: 1
+Component: packages
+Class: fix
+Compatible: compat
+Edition: cre
+State: unknown
+Version: 1.6.0i1
+Date: 1526558604
+
+The python delivered with Check_MK was not precompiling the lib/python2.7/sitecustomize.pyc
+during packaging. It was then created during runtime on the target system. Since this file
+is not under control of the RPM/DEB packages the file was left on the filesystem after
+uninstall. The version was still visible after removal in "omd version" output until one
+removed the /omd/versions/[version] directory.
diff --git a/omd/packages/python/.gitignore b/omd/packages/python/.gitignore
index b5d449b..2020b77 100644
--- a/omd/packages/python/.gitignore
+++ b/omd/packages/python/.gitignore
@@ -2,3 +2,4 @@ Python-2.7.1?
bin/
build/
destdir/
+sitecustomize.pyc
diff --git a/omd/packages/python/Makefile b/omd/packages/python/Makefile
index 8e05038..7297e3b 100644
--- a/omd/packages/python/Makefile
+++ b/omd/packages/python/Makefile
@@ -40,6 +40,13 @@ build: bin/gcc bin/g++
# python-modules, ...) during compilation and install targets.
# NOTE: -j1 seems to be necessary when --enable-optimizations is used
PATH="$(abspath bin):$$PATH" ; $(MAKE) -j1 -C $(DIR) DESTDIR=$(PACKAGE_PYTHON_DESTDIR) install
+ $(MAKE) sitecustomize.pyc
+
+sitecustomize.pyc: sitecustomize.py
+ export PYTHONPATH="$$PYTHONPATH:$(PACKAGE_PYTHON_PYTHONPATH)" ; \
+ export LDFLAGS="$(PACKAGE_PYTHON_LDFLAGS)" ; \
+ export LD_LIBRARY_PATH="$(PACKAGE_PYTHON_LD_LIBRARY_PATH)" ; \
+ $(PACKAGE_PYTHON_EXECUTABLE) -m py_compile sitecustomize.py
# The compiler detection code below is basically what part of AC_PROC_CXX does.
bin/gcc:
@@ -87,12 +94,12 @@ install: bin/gcc bin/g++
$(DESTDIR)$(OMD_ROOT)/bin/pydoc \
$(DESTDIR)$(OMD_ROOT)/bin/python2.7-config \
$(DESTDIR)$(OMD_ROOT)/bin/2to3
- install -m 644 sitecustomize.py $(DESTDIR)$(OMD_ROOT)/lib/python2.7/
+ install -m 644 sitecustomize.py sitecustomize.pyc $(DESTDIR)$(OMD_ROOT)/lib/python2.7/
skel:
clean:
- rm -rf $(DIR) bin build $(PACKAGE_PYTHON_DESTDIR)
+ rm -rf $(DIR) bin build $(PACKAGE_PYTHON_DESTDIR) sitecutomize.pyc
upstream:
git rm Python-*.tgz
Module: check_mk
Branch: master
Commit: 96e91b87d88ea5aa9a495a28dca1418cbd7703d8
URL: http://git.mathias-kettner.de/git/?p=check_mk.git;a=commit;h=96e91b87d88ea5…
Author: Andreas Boesl <ab(a)mathias-kettner.de>
Date: Thu May 17 11:57:34 2018 +0200
5812 FIX Fixed Check_MK check helper crashes related to loading of precompiled checks
The latest versions no longer read checks as plain text python files (Werk 6046).
The operation which generates the precompilated file was not atomic, so a file could
be partially written, while being read by another process.
This caused various exceptions in multiple checks.
Change-Id: I5ef5e0359d3aeec7954e0d08fea857a1f40e1fd4
---
.werks/5812 | 13 ++++++++
omd/packages/python/Makefile | 6 ++++
.../python/patches/pycompile_compile_atomic.dif | 35 ++++++++++++++++++++++
3 files changed, 54 insertions(+)
diff --git a/.werks/5812 b/.werks/5812
new file mode 100644
index 0000000..07efb11
--- /dev/null
+++ b/.werks/5812
@@ -0,0 +1,13 @@
+Title: Fixed Check_MK check helper crashes related to loading of precompiled checks
+Level: 2
+Component: checks
+Compatible: compat
+Edition: cre
+Version: 1.6.0i1
+Date: 1526550541
+Class: fix
+
+The latest versions no longer read checks as plain text python files (Werk 6046).
+The operation which generates the precompilated file was not atomic, so a file could
+be partially written, while being read by another process.
+This caused various exceptions in multiple checks.
diff --git a/omd/packages/python/Makefile b/omd/packages/python/Makefile
index 503535e..8e05038 100644
--- a/omd/packages/python/Makefile
+++ b/omd/packages/python/Makefile
@@ -8,6 +8,12 @@ DIR = $(NAME)-$(VERSION)
build: bin/gcc bin/g++
tar xzf $(DIR).tgz
+
+ set -e ; for p in patches/*.dif ; do \
+ echo "applying $$p..." ; \
+ patch -p1 -b -d $(DIR) < $$p ; \
+ done
+
# Things are a bit tricky here: For PGO/LTO we need a rather recent compiler,
# but we don't want to bake paths to our build system into _sysconfigdata and
# friends. Workaround: Find a recent compiler to be used for building and make a
diff --git a/omd/packages/python/patches/pycompile_compile_atomic.dif b/omd/packages/python/patches/pycompile_compile_atomic.dif
new file mode 100644
index 0000000..a7e844f
--- /dev/null
+++ b/omd/packages/python/patches/pycompile_compile_atomic.dif
@@ -0,0 +1,35 @@
+diff -r 8edfe539d4c6 Lib/py_compile.py
+--- a/Lib/py_compile.py Mon May 20 10:34:58 2013 -0400
++++ b/Lib/py_compile.py Mon May 20 17:46:50 2013 -0400
+@@ -120,13 +120,24 @@
+ return
+ if cfile is None:
+ cfile = file + (__debug__ and 'c' or 'o')
+- with open(cfile, 'wb') as fc:
+- fc.write('\0\0\0\0')
+- wr_long(fc, timestamp)
+- marshal.dump(codeobject, fc)
+- fc.flush()
+- fc.seek(0, 0)
+- fc.write(MAGIC)
++ # Atomically write the pyc/pyo file. Issue #13146.
++ # id() is used to generate a pseudo-random filename.
++ path_tmp = '{}.{}'.format(cfile, id(cfile))
++ try:
++ with open(path_tmp, 'wb') as fc:
++ fc.write('\0\0\0\0')
++ wr_long(fc, timestamp)
++ marshal.dump(codeobject, fc)
++ fc.flush()
++ fc.seek(0, 0)
++ fc.write(MAGIC)
++ os.rename(path_tmp, cfile)
++ except OSError:
++ try:
++ os.unlink(path_tmp)
++ except OSError:
++ pass
++ raise
+
+ def main(args=None):
+ """Compile several source files.