1. 14 Dec, 2023 5 commits
  2. 13 Dec, 2023 11 commits
  3. 12 Dec, 2023 10 commits
    • Kirill Smelkov's avatar
      Unbreak wendelin.core and pygolang tests · de03e816
      Kirill Smelkov authored
      Hello up there.
      
      I've noticed that wendelin.core tests became failing to build and
      automatically disabled some time ago. Not good. I've tried to fix build
      failures. Please see individual patches for details.
      
      /cc @levin.zimmermann
      /reviewed-by @jerome
      /reviewed-on nexedi/slapos!1492
      de03e816
    • Kirill Smelkov's avatar
      pygolang: Unbreak tests · 9ba67430
      Kirill Smelkov authored
      After probably 02fad6a8 (component/{numpy,scipy}: Version up for Python3.)
      pygolang test build started to fail in numpy compilation as shown in the
      appendix. The failure is correct as pygolang never explicitly depended on
      ${numpy:egg}. We did not noticed before probably because in earlier versions
      numpy did not required Cython to be present at install time.
      
      -> Fix it by explicitly adding ${numpy:egg} to pygolang[all_test] dependencies.
      
      Apendix. Log of test build failure
      
          networkcache: Trying to download pypi:numpy=1.22.0 from network cache...
          Getting distribution for 'numpy==1.22.0'.
          WARNING: The easy_install command is deprecated and will be removed in a future version.
          Running from numpy source directory.
          Processing numpy/random/_bounded_integers.pxd.in
          Processing numpy/random/_pcg64.pyx
          Traceback (most recent call last):
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 53, in process_pyx
              import Cython
          ModuleNotFoundError: No module named 'Cython'
      
          The above exception was the direct cause of the following exception:
      
          Traceback (most recent call last):
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 234, in <module>
              main()
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 230, in main
              find_process_files(root_dir)
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 221, in find_process_files
              process(root_dir, fromfile, tofile, function, hash_db)
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 187, in process
              processor_function(fromfile, tofile)
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/tools/cythonize.py", line 60, in process_pyx
              raise OSError(msg) from e
          OSError: Cython needs to be installed in Python as a module
          Traceback (most recent call last):
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 154, in save_modules
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 195, in setup_context
            File "<string>", line 7, in setup_context
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 250, in run_setup
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 45, in _execfile
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 450, in <module>
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 432, in setup_package
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 237, in generate_cython
          RuntimeError: Running cythonize failed!
      
          During handling of the above exception, another exception occurred:
      
          Traceback (most recent call last):
            File "<string>", line 10, in <module>
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 2317, in main
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/__init__.py", line 162, in setup
            File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/distutils/core.py", line 148, in setup
              dist.run_commands()
            File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/distutils/dist.py", line 966, in run_commands
              self.run_command(cmd)
            File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/distutils/dist.py", line 985, in run_command
              cmd_obj.run()
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 424, in run
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 666, in easy_install
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 711, in install_item
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 896, in install_eggs
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 1164, in build_and_install
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/command/easy_install.py", line 1150, in run_setup
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 253, in run_setup
            File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/contextlib.py", line 137, in __exit__
              self.gen.throw(typ, value, traceback)
            File "<string>", line 7, in setup_context
            File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/contextlib.py", line 137, in __exit__
              self.gen.throw(typ, value, traceback)
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 195, in setup_context
            File "/srv/slapgrid/slappart49/srv/runner/shared/python3/a066243faa0ae8c256b7daa6207cc9a6/lib/python3.9/contextlib.py", line 137, in __exit__
              self.gen.throw(typ, value, traceback)
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 166, in save_modules
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 141, in resume
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/_vendor/six.py", line 685, in reraise
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 154, in save_modules
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 195, in setup_context
            File "<string>", line 7, in setup_context
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 250, in run_setup
            File "/srv/slapgrid/slappart49/srv/runner/software/0919a77f026d6fc6870b3d6ec703b69d/eggs/setuptools-44.1.1-py3.9.egg/setuptools/sandbox.py", line 45, in _execfile
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 450, in <module>
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 432, in setup_package
            File "/tmp/easy_install-z2of107_/numpy-1.22.0/setup.py", line 237, in generate_cython
          RuntimeError: Running cythonize failed!
          Cythonizing sources
          An error occurred when trying to install numpy 1.22.0. Look above this message for any errors that were output by easy_install.
          While:
            Installing gpython.
            Base installation request: 'pygolang[all_test]'
              Requirement of pygolang[all_test]: wheel
              Requirement of pygolang[all_test]: setuptools_dso>=2.8
              Requirement of pygolang[all_test]: setuptools
              Requirement of pygolang[all_test]: pytest
              Requirement of pygolang[all_test]: numpy
              Requirement of pygolang[all_test]: ipython
              Requirement of pygolang[all_test]: cython<3
              Requirement of pygolang[all_test]: geventmp
              Requirement of pygolang[all_test]: setuptools_dso>=2.8
              Requirement of pygolang[all_test]: decorator
              Requirement of pygolang[all_test]: six
              Requirement of pygolang[all_test]: gevent
              Requirement of setuptools_dso>=2.8: setuptools
              Requirement of pytest: wcwidth
              Requirement of pytest: six>=1.10.0
              Requirement of pytest: py>=1.5.0
              Requirement of pytest: pluggy<1.0,>=0.12
              Requirement of pytest: packaging
              Requirement of pytest: more-itertools>=4.0.0
              Requirement of pytest: attrs>=17.4.0
              Requirement of pytest: atomicwrites>=1.0
            Getting distribution for 'numpy==1.22.0'.
          Error: Couldn't install: numpy 1.22.0
      9ba67430
    • Kirill Smelkov's avatar
      ZEO: Fix ZEO4-wc2 installation, take2 · f276d26f
      Kirill Smelkov authored
      I already fixed ZEO4-wc2 installation once in 97832d95 (ZEO: Fix
      ZEO4-wc2 installation). That worked but, as it turned out, not fully:
      Even though correct ZEO4-wc2-repository is cloned, and correct ZEO4-wc2
      develop-egg is installed, we don't tell buildout which egg-version of
      ZEO to use. This leads to the situation when buildout picks up _latest_
      ZEO egg, when another egg, which depends on ZEO, is installed.
      
      For example wendelin.core/test-zodb4-wc2 was failing as
      
          Updating ZEO4-wc2-repository.
          Installing ZEO4-wc2.
          warning: no previously-included files matching '*.pyc' found anywhere in distribution
          Installing ZEO.
          Installing zodbtools.
          While:
            Installing zodbtools.
            Base installation request: 'zodbtools'
              Requirement of zodbtools==0.0.0.dev8: dateparser
              Requirement of zodbtools==0.0.0.dev8: six
              Requirement of zodbtools==0.0.0.dev8: pygolang>=0.0.0.dev6
              Requirement of zodbtools==0.0.0.dev8: zope.interface
              Requirement of zodbtools==0.0.0.dev8: zodburi
              Requirement of zodbtools==0.0.0.dev8: ZODB
              Requirement of dateparser: tzlocal
              Requirement of dateparser: regex!=2019.02.19
              Requirement of dateparser: pytz
              Requirement of dateparser: python-dateutil
              Requirement of pygolang>=0.0.0.dev6: Importing
              Requirement of pygolang>=0.0.0.dev6: decorator
              Requirement of pygolang>=0.0.0.dev6: six
              Requirement of pygolang>=0.0.0.dev6: gevent
              Requirement of zope.interface: setuptools
              Requirement of zodburi: ZEO
              Requirement of zodburi: ZConfig
              Requirement of zodburi: ZODB
              Requirement of ZODB: zodbpickle>=0.6.0
              Requirement of ZODB: zope.interface
              Requirement of ZODB: zc.lockfile
              Requirement of ZODB: six
              Requirement of ZODB: transaction>=1.5.0
              Requirement of ZODB: ZConfig
              Requirement of ZODB: BTrees>=4.2.0
              Requirement of ZODB: persistent>=4.2.0
              Requirement of tzlocal: pytz
              Requirement of python-dateutil: six>=1.5
              Requirement of gevent: greenlet>=0.4.17
              Requirement of gevent: setuptools
              Requirement of gevent: zope.interface
              Requirement of gevent: zope.event
              Requirement of ZEO: trollius
              Requirement of ZEO: futures
              Requirement of ZEO: zope.interface
              Requirement of ZEO: zdaemon
              Requirement of ZEO: ZConfig
              Requirement of ZEO: zc.lockfile
              Requirement of ZEO: persistent>=4.1.0
              Requirement of ZEO: transaction>=2.0.3
              Requirement of ZEO: six
              Requirement of ZEO: ZODB>=5.1.1
              Requirement of zodbpickle>=0.6.0: setuptools
              Requirement of zc.lockfile: setuptools
              Requirement of transaction>=1.5.0: zope.interface
              Requirement of BTrees>=4.2.0: zope.interface>=5.0.0
              Requirement of BTrees>=4.2.0: persistent>=4.1.0
              Requirement of persistent>=4.2.0: cffi
              Requirement of persistent>=4.2.0: zope.interface
              Requirement of zope.event: setuptools
            Getting distribution for 'trollius'.
          Error: Picked: trollius = 2.2.1
      
      -> Fix that by specifying which egg-version ZEO4/ZEO4-wc2 should be using.
      
      For the reference for ZODB we already do the same - specify egg-version for all
      ZODB5, ZODB4-wc2 and ZODB4:
      
      https://lab.nexedi.com/nexedi/slapos/blob/6765c349/component/ZODB/buildout.cfg#L52-72
      f276d26f
    • Kirill Smelkov's avatar
      wendelin.core: tests: Pin ZopeUndo · b8148370
      Kirill Smelkov authored
      ZEO[test] requires ZopeUndo, but we now never pin that anywhere in e.g. stack/slapos.cfg .
      Previously it was working without explicit pinning because Zope2, contrary to Zope4, pins ZopeUndo to 2.12.0.
      
      Build of test-zodb5.cfg was failing as
      
          Installing wendelin.core-python.
          While:
            Installing wendelin.core-python.
            Base installation request: 'wendelin.core[test]', 'pygolang[pyx.build]', 'neoppod[tests]', 'ZEO[test]'
              Requirement of ZEO[test]==5.4.0: zope.testrunner
              Requirement of ZEO[test]==5.4.0: zdaemon
              Requirement of ZEO[test]==5.4.0: msgpack<1
              Requirement of ZEO[test]==5.4.0: mock
              Requirement of ZEO[test]==5.4.0: transaction
              Requirement of ZEO[test]==5.4.0: manuel
              Requirement of ZEO[test]==5.4.0: zope.testing
              Requirement of ZEO[test]==5.4.0: ZopeUndo
              Requirement of ZEO[test]==5.4.0: ZODB>=5.5.1
              Requirement of ZEO[test]==5.4.0: ZConfig
              Requirement of ZEO[test]==5.4.0: trollius
              Requirement of ZEO[test]==5.4.0: futures
              Requirement of ZEO[test]==5.4.0: zope.interface
              Requirement of ZEO[test]==5.4.0: zdaemon
              Requirement of ZEO[test]==5.4.0: ZConfig
              Requirement of ZEO[test]==5.4.0: zc.lockfile
              Requirement of ZEO[test]==5.4.0: persistent>=4.1.0
              Requirement of ZEO[test]==5.4.0: transaction>=2.0.3
              Requirement of ZEO[test]==5.4.0: six
              Requirement of ZEO[test]==5.4.0: ZODB>=5.1.1
              Requirement of neoppod[tests]==1.12.0: neoppod[admin,client,ctl,master,storage-importer,storage-mysqldb,storage-pymysql,storage-sqlite]
              Requirement of neoppod[tests]==1.12.0: psutil>=2
              Requirement of neoppod[tests]==1.12.0: zope.testing
              Requirement of neoppod[tests]==1.12.0: coverage
              Requirement of neoppod[tests]==1.12.0: python-dateutil
              Requirement of neoppod[tests]==1.12.0: msgpack>=0.5.6
              Requirement of pygolang[pyx.build]==0.1: wheel
              Requirement of pygolang[pyx.build]==0.1: setuptools_dso>=1.7
              Requirement of pygolang[pyx.build]==0.1: setuptools
              Requirement of pygolang[pyx.build]==0.1: cython
              Requirement of pygolang[pyx.build]==0.1: Importing
              Requirement of pygolang[pyx.build]==0.1: decorator
              Requirement of pygolang[pyx.build]==0.1: six
              Requirement of pygolang[pyx.build]==0.1: gevent
              Requirement of wendelin.core[test]: ZEO
              Requirement of wendelin.core[test]: neoppod
              Requirement of wendelin.core[test]: scipy
              Requirement of wendelin.core[test]: pytest
              Requirement of wendelin.core[test]: psutil
              Requirement of wendelin.core[test]: six
              Requirement of wendelin.core[test]: pygolang>=0.1
              Requirement of wendelin.core[test]: zodbtools>=0.0.0.dev8
              Requirement of wendelin.core[test]: ZODB>=4
              Requirement of wendelin.core[test]: numpy
              Requirement of zope.testrunner: zope.interface
              Requirement of zope.testrunner: zope.exceptions
              Requirement of zope.testrunner: six
              Requirement of zope.testrunner: setuptools
              Requirement of zdaemon: setuptools
              Requirement of zdaemon: ZConfig
              Requirement of mock: funcsigs>=1
              Requirement of mock: six
              Requirement of transaction: zope.interface
              Requirement of manuel: six
              Requirement of manuel: setuptools
              Requirement of zope.testing: setuptools
            Getting distribution for 'ZopeUndo'.
          Error: Picked: ZopeUndo = 6.0
      
      -> pin ZopeUndo to 5.0 - the last version that supports both py2 and py3.
      b8148370
    • Kirill Smelkov's avatar
      wendelin.core: Unbreak tests · 7f7ff9c0
      Kirill Smelkov authored
      After d8409763 (NEO: default ZODB was changed to 5) wendelin.core tests
      started to fail to build because files, that the tests extend from, were
      renamed on neoppod side. Example failure:
      
          An internal error occurred due to a bug in either zc.buildout or in a
          recipe being used:
          Traceback (most recent call last):
            File "/opt/slapos/eggs/zc.buildout-2.7.1+slapos019-py3.7.egg/zc/buildout/buildout.py", line 2359, in main
              user_defaults, command, args)
            File "/opt/slapos/eggs/zc.buildout-2.7.1+slapos019-py3.7.egg/zc/buildout/buildout.py", line 312, in __init__
              data['buildout'], override))
            File "/opt/slapos/eggs/zc.buildout-2.7.1+slapos019-py3.7.egg/zc/buildout/buildout.py", line 1974, in _open
              seen, processing))
            File "/opt/slapos/eggs/zc.buildout-2.7.1+slapos019-py3.7.egg/zc/buildout/buildout.py", line 1974, in _open
              seen, processing))
            File "/opt/slapos/eggs/zc.buildout-2.7.1+slapos019-py3.7.egg/zc/buildout/buildout.py", line 1974, in _open
              seen, processing))
            File "/opt/slapos/eggs/zc.buildout-2.7.1+slapos019-py3.7.egg/zc/buildout/buildout.py", line 1940, in _open
              with open(downloaded_filename) as fp:
          FileNotFoundError: [Errno 2] No such file or directory: '/srv/slapgrid/slappart49/srv/project/slapos/software/neoppod/software-zodb5.cfg'
      
      -> Fix it by adjusting wendelin.core tests correspondingly.
      
      Similarly to 339490eb (components/zodbtools: reorganize test profiles)
      make each test profile to explicitly indicate which configuration it
      tests against.
      
      Drop support for testing pristine ZODB4 - now only ZODB5 and ZODB4-wc2 remain.
      
      We can drop test coverage for ZODB4 because slapos mainline switched to
      ZODB5 to be used by default in 1c51c4cd (stack/erp5: version up all zope
      stack (Zope 4.8.7), and because wendelin.core 2 works only with ZODB5
      and ZODB4-wc2, but not with plain ZODB4.
      7f7ff9c0
    • Joanne Hugé's avatar
      software/ors-amarisoft: update changelog · 6765c349
      Joanne Hugé authored
      6765c349
    • Jérome Perrin's avatar
    • Levin Zimmermann's avatar
      software/wendelin: Set with-max-rlimit-nofile default to true · 67a38887
      Levin Zimmermann authored
      In Wendelin we mostly always want to run our zopes with the soft limit
      set to the hard limit.
      
      /reviewed-by @jerome, @jm, @rafael, @vpelletier
      /reviewed-on !1465
      
      ---
      
      This patch was already added in !1451, but
      removed (by force push), due to issues discussed in !1451 (comment 193296).
      67a38887
    • Levin Zimmermann's avatar
      software/erp5: Add with-max-rlimit-nofile option · f55f0c0c
      Levin Zimmermann authored
      nexedi/erp5@39369169 added the functionality to ERP5 to
      set its soft limit of allowed open file descriptors to the system wide hard limit.
      This parameter is useful for Wendelin based instances where the 1024
      limit is easily reached.
      
      With this patch, this parameter can also be set via SlapOS, which
      simplifies usage of the Wendelin SR.
      
      /reviewed-by @jerome, @jm, @rafael, @vpelletier
      /reviewed-on nexedi/slapos!1465
      
      ---
      
      This patch was already added in nexedi/slapos!1451, but
      removed (by force push), due to issues discussed in nexedi/slapos!1451 (comment 193296).
      f55f0c0c
    • Thomas Gambier's avatar
      software/cloudooo: fix input schema JSON · 8103c4f9
      Thomas Gambier authored
      8103c4f9
  4. 11 Dec, 2023 1 commit
  5. 08 Dec, 2023 4 commits
  6. 07 Dec, 2023 1 commit
  7. 06 Dec, 2023 4 commits
  8. 05 Dec, 2023 2 commits
  9. 04 Dec, 2023 2 commits