nexedi / slapos

python-2.7 was depending on test suite, it just runs the test suite to collect profiling data and run a profile guided build ( see https://github.com/python/cpython#profile-guided-optimization ). With recent version up on openssl, python-2.7 does not seem to pass the test suite.

Also, this MR tries to enable tests for openssl itself, they don't seem to pass for openssl 1.0 either - but they pass when just running make test interactively. I have not investigated this.

mentioned in merge request !442

For python 2.7, this adds 7 minutes of build time.

For OpenSSL 1.0 , about 1 minute.

I made this commit was just to make test pass, perhaps it should not be in this MR

mentioned in merge request !442

I looked at the build history on test node and regarding python 2.7 tests, it seems that:

• before and after openssl version up, test_distutils sometimes fail. For exemple it failed on rapidspace-testnode-000-3Nodes-ERP5PROJECT8 on https://nexedi.erp5.net/test_result_module/20181129-6B6E77/TestResult_viewNodeList - but it passed on rapidspace-testnode-002-3Nodes-ERP5PROJECT4 on https://nexedi.erp5.net/test_result_module/20181202-24E22B83/TestResult_viewNodeList
  
• test_ssl sometimes fail; for example it failed on rapidspace-testnode-002-3Nodes-ERP5PROJECT4 on https://nexedi.erp5.net/test_result_module/20181202-24E22B83/TestResult_viewNodeList - but did not fail on rapidspace-testnode-000-3Nodes-ERP5PROJECT8 on https://nexedi.erp5.net/test_result_module/20181129-6B6E77/TestResult_viewNodeList )

edit: this is by looking at the test run to collect profiling data, which does not run the same tests that when we run make test

My only concern here is increase too much the compilation time.

For python 2.7, this adds 7 minutes of build time.
For OpenSSL 1.0 , about 1 minute.

I think this is very much acceptable for our core components.

@luke , with your patch of openssl 1.1.0 python2.7 tests suite seem to pass, at least on my environment ( https://lab.nexedi.com/jerome/slapos/commits/e6f451cb3ad752c972962561028ab3e3148a40b3 )

/cc @tomo

I'm still not familiar with the shacache system. When the cache is used for a component, it means we don't run at all the recipe, right ? So in theory, we have to pay the 7 + 1 minutes increase time only in test nodes ? The users will have the components from the cache ?

I'm OK to have the time to run tests increasing but for users I'd like to keep the build time as fast as possible.

If the cache doesn't work this way, maybe we should let the choice to run or not the test during software installatio and only testnodes would chose to run the tests (that's their job after all).

@luke , with your patch of openssl 1.1.0 python2.7 tests suite seem to pass, at least on my environment ( https://lab.nexedi.com/jerome/slapos/commits/e6f451cb3ad752c972962561028ab3e3148a40b3 )

Thanks. Converted it to MR !459.

@tomo put it very quickly, shacache has currently two usages:

• cache URLs used to download code ( exemple here) so that if URL is no longer available, slapos uses the cached version and can still build
• cache full software release after it's built. This is referred as "binary cache". It works by using a big archive of the full installation directory of a software ( ex. /opt/slapgrid/b0bcd9831b23f334bc85e4a193c748a0 ).

With this change, every software installation would takes approx. 8 more minutes, unless binary cache is used. Also it will fail if something is wrong, so there will also be cases where software will not install at all. The good side is that it fails very early when something will be wrong. I think we prefer having errors building software while we are installing machines rather than realizing that something does not work once the system is in production.

At this point we don't have "data" at all , we don't really know, maybe it cause a lot of false positives and will prevent building on some machines just because of flaky tests ( I noticed this morning that apparently nix disable make check for python 2.7 because "expensive, and fails". If I understood correctly the meaning of doCheck in their build system ). So if we start depending on tests, maybe we need a way to ignore test status. At this point, I don't know, it would be interesting to try on more machines.

Also, about cache, there's also shared option of slapos.recipe.cmmi ( slapos.recipe.cmmi!6 ) which makes that each component is installed only once per slapos (and maybe already installed by parent slapos in case of recursive slapos), but this is not really used yet. In any case, this is on our plan the solution for all our "installing software takes time" problems.

mentioned in merge request !459