• Kirill Smelkov's avatar
    Fix build for zodbtools/pygolang/nxdtest/... tests · 3d6db516
    Kirill Smelkov authored
    Commit 7d1ea024 (version up: zc.buildout 2.7.1+slapos013) switched buildout to
    not use egg wheels, even for wheels that are source-only without any binaries.
    As the result some eggs stopped to install because they setup_require other
    eggs and neither buildout handles setup_requires automatically, nor there is
    explicit setup_require declaration in corresponding components.
    
    An example of such egg is python-dateutil: it setup_requires setuptools-scm,
    and without switching python-dateutil to zc.recipe.egg:custom with
    setup_requires the build fails.
    
    The way 7d1ea024 used to workaround this is to mark all such eggs with :whl
    suffix marking them that "it is ok to use wheel for this particular egg". The
    complete list of eggs for which such markings was done in 7d1ea024 is:
    
        ipykernel
        plantuml
        mock
        PasteScript
        backports.functools-lru-cache
        cliff
        stevedore
        CacheControl
        pytest-runner
        jsonschema
        lockfile
        python-dateutil
    
    However this list is not really complete because after 7d1ea024 the build
    started to fail for e.g. zodbtools test:
    
            Traceback (most recent call last):
              File "<string>", line 10, in <module>
              File "/srv/slapgrid/slappart16/srv/testnode/dfp/soft/c670f74c8de08964f63c4b9f7d48c3d1/eggs/setuptools-44.1.1-py2.7.egg/setuptools/command/easy_install.py", line 2321, in main
              ...
              File "/srv/slapgrid/slappart16/srv/testnode/dfp/soft/c670f74c8de08964f63c4b9f7d48c3d1/eggs/setuptools-44.1.1-py2.7.egg/pkg_resources/__init__.py", line 786, in resolve
                raise DistributionNotFound(req, requirers)
            pkg_resources.DistributionNotFound: The 'setuptools-scm' distribution was not found and is required by the application
            An error occurred when trying to install pytest 4.6.11. Look above this message for any errors that were output by easy_install.
            While:
              Installing zodbtools-python.
              Base installation request: 'zodbtools[test]'
                Requirement of zodbtools[test]: mock
                Requirement of zodbtools[test]: pytz
                Requirement of zodbtools[test]: freezegun
                Requirement of zodbtools[test]: pytest
                Requirement of zodbtools[test]: dateparser
                Requirement of zodbtools[test]: six
                Requirement of zodbtools[test]: pygolang>=0.0.0.dev6
                Requirement of zodbtools[test]: zope.interface
                Requirement of zodbtools[test]: zodburi
                Requirement of zodbtools[test]: ZODB
                Requirement of mock: funcsigs>=1
                Requirement of mock: six
                Requirement of freezegun: python-dateutil>=2.7
              Getting distribution for 'pytest==4.6.11'.
            Error: Couldn't install: pytest 4.6.11
    
    and similarly for pygolang and nxdtest tests.
    
    I would be more comfortable if the fix would be to teach buildout to either
    handle setup_requires automatically, or to automatically install wheels if the
    wheel is source-only. However, given the way - that 7d1ea024 started to use
    to handle such situations - let's fix this via explicitly marking pytest, and
    other related eggs that setup_require something and are source-only, as :whl.
    
    NOTE we did not notice the problem earlier because on testnodes, after the
    switch to new buildout, pytest egg was already installed and incremental
    rebuild did not need to reinstall pytest egg. So most of pygolang and zodbtools
    test results remained in "ok" state, and only zodbtools-kirr, that was recently
    reconfigured or changed testnode, indicated this issue. That's why the problem
    remained unnoticed for a long time.
    
    /reviewed-by @jerome
    /reviewed-on !1026
    3d6db516
Name
Last commit
Last update
..
6tunnel Loading commit data...
PyWavelets Loading commit data...
ZEO Loading commit data...
ZODB Loading commit data...
accords Loading commit data...
alsa Loading commit data...
apache Loading commit data...
apache-php Loading commit data...
argon2 Loading commit data...
aspell Loading commit data...
attr Loading commit data...
autoconf Loading commit data...
automake Loading commit data...
babeld Loading commit data...
bash Loading commit data...
bash-completion Loading commit data...
bazel Loading commit data...
bcrypt Loading commit data...
binutils Loading commit data...
bison Loading commit data...
boa Loading commit data...
boost-lib Loading commit data...
busybox Loading commit data...
bzip2 Loading commit data...
ca-certificates Loading commit data...
caddy Loading commit data...
ccache Loading commit data...
cclient Loading commit data...
chainer Loading commit data...
chromedriver Loading commit data...
chromium Loading commit data...
chrpath Loading commit data...
cloud9 Loading commit data...
cloudooo Loading commit data...
cmake Loading commit data...
condor Loading commit data...
consul Loading commit data...
coreutils Loading commit data...
corosync Loading commit data...
coturn Loading commit data...
couchdb Loading commit data...
cpio Loading commit data...
cryptopp Loading commit data...
cups Loading commit data...
curl Loading commit data...
cyrus-sasl Loading commit data...
cython Loading commit data...
cython-zstd Loading commit data...
cythonplus Loading commit data...
dash Loading commit data...
dbus Loading commit data...
dcron Loading commit data...
depot_tools Loading commit data...
dropbear Loading commit data...
dtach Loading commit data...
egg-patch Loading commit data...
embulk Loading commit data...
erlang Loading commit data...
ethtool Loading commit data...
f2c Loading commit data...
faketime Loading commit data...
fastjar Loading commit data...
ffmpeg Loading commit data...
fftw3 Loading commit data...
file Loading commit data...
findutils Loading commit data...
firefox Loading commit data...
firewalld Loading commit data...
fish-shell Loading commit data...
flex Loading commit data...
fluentd Loading commit data...
fmtlib Loading commit data...
fontconfig Loading commit data...
fonts Loading commit data...
freetype Loading commit data...
garbage-collector Loading commit data...
gateone Loading commit data...
gcc Loading commit data...
gdal Loading commit data...
gdb Loading commit data...
gdbm Loading commit data...
geoip2 Loading commit data...
geos Loading commit data...
gettext Loading commit data...
ghostscript Loading commit data...
giflib Loading commit data...
git Loading commit data...
glib Loading commit data...
glibmm Loading commit data...
glpk Loading commit data...
gmp Loading commit data...
gnu-config Loading commit data...
gnupg Loading commit data...
gnutls Loading commit data...
golang Loading commit data...
gperf Loading commit data...
graphviz Loading commit data...
grep Loading commit data...
groff Loading commit data...
groonga Loading commit data...
gtk-2 Loading commit data...
gtk-3 Loading commit data...
gtkmm Loading commit data...
gzip Loading commit data...
h2o Loading commit data...
h5py Loading commit data...
haproxy Loading commit data...
hdf5 Loading commit data...
headless-chromium Loading commit data...
helloweb Loading commit data...
hwloc Loading commit data...
icu Loading commit data...
imagemagick Loading commit data...
inkscape Loading commit data...
intltool Loading commit data...
ioping Loading commit data...
ipython Loading commit data...
jasper Loading commit data...
java Loading commit data...
java-jdk Loading commit data...
jbigkit Loading commit data...
jemalloc Loading commit data...
jpegoptim Loading commit data...
jq Loading commit data...
jsl Loading commit data...
json-c Loading commit data...
jupyter Loading commit data...
jupyter-py2 Loading commit data...
keras Loading commit data...
kerberos Loading commit data...
kumo Loading commit data...
lcms Loading commit data...
leptonica Loading commit data...
libaio Loading commit data...
libatlas Loading commit data...
libcap Loading commit data...
libcap-ng Loading commit data...
libconfig Loading commit data...
libdb Loading commit data...
libdmtx Loading commit data...
libepoxy Loading commit data...
libestr Loading commit data...
libevent Loading commit data...
libexpat Loading commit data...
libfastjson Loading commit data...
libffi Loading commit data...
libiconv Loading commit data...
libjpeg Loading commit data...
libmemcached Loading commit data...
libmicrohttpd Loading commit data...
libpng Loading commit data...
libreoffice-bin Loading commit data...
librsvg Loading commit data...
librsync Loading commit data...
libsigc Loading commit data...
libsodium Loading commit data...
libstemmer Loading commit data...
libtasn1 Loading commit data...
libtiff Loading commit data...
libtool Loading commit data...
liburing Loading commit data...
libuuid Loading commit data...
libxml2 Loading commit data...
libxslt Loading commit data...
libyaml Loading commit data...
libzip Loading commit data...
libzmq Loading commit data...
lksctp-tools Loading commit data...
llvm Loading commit data...
lmbench Loading commit data...
lmsensors Loading commit data...
logrotate Loading commit data...
lua Loading commit data...
lunzip Loading commit data...
lxc Loading commit data...
lxml-python Loading commit data...
lynx Loading commit data...
lz4 Loading commit data...
lzo Loading commit data...
m2crypto Loading commit data...
m4 Loading commit data...
make Loading commit data...
manpy Loading commit data...
mariadb Loading commit data...
matplotlib Loading commit data...
mbedtls Loading commit data...
membase Loading commit data...
memcached Loading commit data...
memstrike Loading commit data...
mesa Loading commit data...
meson Loading commit data...
messagepack Loading commit data...
mosh Loading commit data...
msgpack-numpy Loading commit data...
msgpack-python Loading commit data...
mysql-5.1 Loading commit data...
mysql-tritonn-5.0 Loading commit data...
nano Loading commit data...
ncurses Loading commit data...
neon Loading commit data...
netcat Loading commit data...
nettle Loading commit data...
nftables Loading commit data...
nghttp2 Loading commit data...
nginx Loading commit data...
ninja Loading commit data...
nltk Loading commit data...
noVNC Loading commit data...
nodejs Loading commit data...
nspr Loading commit data...
nss Loading commit data...
nullmailer Loading commit data...
numpy Loading commit data...
nxdtest Loading commit data...
ocropus Loading commit data...
ocropy Loading commit data...
oniguruma Loading commit data...
onlyoffice-core Loading commit data...
oood Loading commit data...
openblas Loading commit data...
openldap Loading commit data...
openoffice Loading commit data...
openoffice-bin Loading commit data...
openssh Loading commit data...
openssl Loading commit data...
openvpn Loading commit data...
optipng Loading commit data...
p11-kit Loading commit data...
p7zip Loading commit data...
packer Loading commit data...
pandas Loading commit data...
patch Loading commit data...
patchelf Loading commit data...
pcre Loading commit data...
percona-toolkit Loading commit data...
perl Loading commit data...
perl-DBD-MySQL Loading commit data...
perl-DBI Loading commit data...
perl-Data-Dump Loading commit data...
perl-Devel-CheckLib Loading commit data...
perl-Encode-Locale Loading commit data...
perl-File-Listing Loading commit data...
perl-HTML-Parser Loading commit data...
perl-HTML-Tagset Loading commit data...
perl-HTTP-Cookies Loading commit data...
perl-HTTP-Daemon Loading commit data...
perl-HTTP-Date Loading commit data...
perl-HTTP-Message Loading commit data...
perl-HTTP-Negotiate Loading commit data...
perl-IO-HTML Loading commit data...
perl-Image-ExifTool Loading commit data...
perl-Image-Magick Loading commit data...
perl-LWP-MediaTypes Loading commit data...
perl-Net-HTTP Loading commit data...
perl-Term-ReadLine-Gnu Loading commit data...
perl-URI Loading commit data...
perl-WWW-RobotRules Loading commit data...
perl-XML-LibXML Loading commit data...
perl-XML-NamespaceSupport Loading commit data...
perl-XML-Parser Loading commit data...
perl-XML-SAX Loading commit data...
perl-XML-SAX-Base Loading commit data...
perl-libwww-perl Loading commit data...
phantomjs Loading commit data...
pil-python Loading commit data...
pillow Loading commit data...
pkgconfig Loading commit data...
pole Loading commit data...
poppler Loading commit data...
popt Loading commit data...
postfix Loading commit data...
postgresql Loading commit data...
powerdns Loading commit data...
proftpd Loading commit data...
proj4 Loading commit data...
protobuf Loading commit data...
protobuf-python Loading commit data...
proxysql Loading commit data...
pulp Loading commit data...
pure-ftpd Loading commit data...
pwgen Loading commit data...
pycrypto-python Loading commit data...
pycuda Loading commit data...
pycurl Loading commit data...
pygolang Loading commit data...
pylint Loading commit data...
pypy Loading commit data...
pysvn-python Loading commit data...
pytest Loading commit data...
python-2.7 Loading commit data...
python-PyYAML Loading commit data...
python-backports-lzma Loading commit data...
python-cffi Loading commit data...
python-cocoapi Loading commit data...
python-cryptography Loading commit data...
python-kerberos Loading commit data...
python-ldap-python Loading commit data...
python-manuel Loading commit data...
python-mysqlclient Loading commit data...
python-pynacl Loading commit data...
python-pyzmq Loading commit data...
python-slip Loading commit data...
python3 Loading commit data...
pytorch Loading commit data...
qemu-kvm Loading commit data...
qt Loading commit data...
quic_client-bin Loading commit data...
quickjs Loading commit data...
r-language Loading commit data...
ragel Loading commit data...
randomsleep Loading commit data...
rdiff-backup Loading commit data...
re6stnet Loading commit data...
readline Loading commit data...
redis Loading commit data...
replication-manager Loading commit data...
restic Loading commit data...
rina-tools Loading commit data...
rpm2cpio Loading commit data...
rpy2 Loading commit data...
rrdtools Loading commit data...
rsync Loading commit data...
rsyslogd Loading commit data...
ruby Loading commit data...
rust Loading commit data...
scikit-image Loading commit data...
scikit-learn Loading commit data...
scipy Loading commit data...
screen Loading commit data...
seaborn Loading commit data...
sed Loading commit data...
serf Loading commit data...
sheepstrike Loading commit data...
shellinabox Loading commit data...
slapos Loading commit data...
smartmontools Loading commit data...
snappy Loading commit data...
socat Loading commit data...
sphinx Loading commit data...
spidermonkey Loading commit data...
sqlite3 Loading commit data...
squid Loading commit data...
srslte Loading commit data...
statsmodels Loading commit data...
stunnel Loading commit data...
subversion Loading commit data...
swig Loading commit data...
sysbench Loading commit data...
systemd Loading commit data...
tar Loading commit data...
tcl Loading commit data...
tempstorage Loading commit data...
tensorboard Loading commit data...
tensorflow Loading commit data...
tesseract Loading commit data...
texinfo Loading commit data...
tig Loading commit data...
tmux Loading commit data...
tokyocabinet Loading commit data...
tomcat Loading commit data...
trafficserver Loading commit data...
unixodbc Loading commit data...
unstable Loading commit data...
unzip Loading commit data...
userhosts Loading commit data...
util-linux Loading commit data...
varnish Loading commit data...
vim Loading commit data...
vm-img Loading commit data...
vnu Loading commit data...
w3m Loading commit data...
webp Loading commit data...
wendelin.core Loading commit data...
wget Loading commit data...
wkhtmltopdf Loading commit data...
xapian Loading commit data...
xorg Loading commit data...
xpdf Loading commit data...
xtrabackup Loading commit data...
xz-utils Loading commit data...
yaml-cpp Loading commit data...
yarn Loading commit data...
zabbix Loading commit data...
zbar Loading commit data...
zbarlight Loading commit data...
zeromq Loading commit data...
zip Loading commit data...
zlib Loading commit data...
zodbtools Loading commit data...
zstd Loading commit data...
defaults.cfg Loading commit data...