Skip to content

G
gevent
Commit 5d2bcb8d authored by Jason Madden's avatar Jason Madden Committed by GitHub
Browse files
Options

Merge pull request #1050 from gevent/pypy-appveyor 

Use setuptools to build libuv, and enable it on Windows (especially for PyPy)
parents fc34e2ff 8fba2912
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 2240 additions and 2397 deletions
CHANGES.rst
View file @ 5d2bcb8d
......@@ -101,8 +101,7 @@
- Add initial *experimental* support for using libuv as a backend
instead of libev, controlled by setting the environment variable
``GEVENT_CORE_CFFI_ONLY=libuv`` before importing gevent. This only
works on POSIX systems and it still suffers a number of limitations
compared to libev, notably:
suffers a number of limitations compared to libev, notably:
- Timers (such as ``gevent.sleep`` and ``gevent.Timeout``) only
support a resolution of 1ms. Attempting to use something smaller
......@@ -133,8 +132,9 @@
than either of the libev implementations (cython or CFFI), so
pressure on the garbage collector will be higher.
- The build system may not be correctly producing embedded static
libraries, at least on OS X. libuv 1.18 or higher is required.
- The build system does not support using a system libuv; the
embedded copy must be used. Using setuptools to compile libuv was
the most portable method found.
- If anything unexpected happens, libuv likes to ``abort()`` the
entire process instead of reporting an error. For example, closing
......@@ -142,7 +142,7 @@
process to be exited.
Feedback and pull requests are welcome, especially to address the
issues mentioned above and for Windows support.
issues mentioned above.
Again, this is extremely experimental and all of it is subject to
change.
......
Makefile
View file @ 5d2bcb8d
......@@ -117,8 +117,8 @@ PY27=$(BUILD_RUNTIMES)/snakepit/python2.7.14
PY34=$(BUILD_RUNTIMES)/snakepit/python3.4.7
PY35=$(BUILD_RUNTIMES)/snakepit/python3.5.4
PY36=$(BUILD_RUNTIMES)/snakepit/python3.6.2
PYPY=$(BUILD_RUNTIMES)/snakepit/pypy580
PYPY3=$(BUILD_RUNTIMES)/snakepit/pypy3.5_580
PYPY=$(BUILD_RUNTIMES)/snakepit/pypy590
PYPY3=$(BUILD_RUNTIMES)/snakepit/pypy3.5_590
TOOLS=$(BUILD_RUNTIMES)/tools
......@@ -151,20 +151,18 @@ $(PYPY):
$(PYPY3):
scripts/install.sh pypy3
PIP?=$(BUILD_RUNTIMES)/versions/$(PYTHON)/bin/pip
develop:
ls -l $(BUILD_RUNTIMES)/snakepit/
echo pip is at `which $(PIP)`
echo python is at `which $(PYTHON)`
# First install a newer pip so that it can use the wheel cache
# (only needed until travis upgrades pip to 7.x; note that the 3.5
# environment uses pip 7.1 by default)
${PIP} install -U pip setuptools
python -m pip install -U pip setuptools
# Then start installing our deps so they can be cached. Note that use of --build-options / --global-options / --install-options
# disables the cache.
# We need wheel>=0.26 on Python 3.5. See previous revisions.
${PIP} install -U -r dev-requirements.txt
python -m pip install -U -r dev-requirements.txt
lint-py27: $(PY27)
PYTHON=python2.7.14 PATH=$(BUILD_RUNTIMES)/versions/python2.7.14/bin:$(PATH) make develop travis_test_linters
......@@ -177,22 +175,23 @@ test-py278: $(PY278)
PYTHON=python2.7.8 PATH=$(BUILD_RUNTIMES)/versions/python2.7.8/bin:$(PATH) make develop toxtest
test-py34: $(PY34)
PYTHON=python3.4.7 PIP=pip PATH=$(BUILD_RUNTIMES)/versions/python3.4.7/bin:$(PATH) make develop toxtest
PYTHON=python3.4.7 PATH=$(BUILD_RUNTIMES)/versions/python3.4.7/bin:$(PATH) make develop toxtest
test-py35: $(PY35)
PYTHON=python3.5.4 PIP=pip PATH=$(BUILD_RUNTIMES)/versions/python3.5.4/bin:$(PATH) make develop fulltoxtest
PYTHON=python3.5.4 PATH=$(BUILD_RUNTIMES)/versions/python3.5.4/bin:$(PATH) make develop fulltoxtest
test-py36: $(PY36)
PYTHON=python3.6.2 PIP=pip PATH=$(BUILD_RUNTIMES)/versions/python3.6.2/bin:$(PATH) make develop toxtest
PYTHON=python3.6.2 PATH=$(BUILD_RUNTIMES)/versions/python3.6.2/bin:$(PATH) make develop toxtest
test-py36-libuv: $(PY36)
GEVENT_CORE_CFFI_ONLY=libuv make test-py36
test-pypy: $(PYPY)
PYTHON=$(PYPY) PIP=pip PATH=$(BUILD_RUNTIMES)/versions/pypy580/bin:$(PATH) make develop toxtest
ls $(BUILD_RUNTIMES)/versions/pypy590/bin/
PYTHON=$(PYPY) PATH=$(BUILD_RUNTIMES)/versions/pypy590/bin:$(PATH) make develop toxtest
test-pypy3: $(PYPY3)
PYTHON=$(PYPY3) PIP=pip PATH=$(BUILD_RUNTIMES)/versions/pypy3.5_580/bin:$(PATH) make develop toxtest
PYTHON=$(PYPY3) PATH=$(BUILD_RUNTIMES)/versions/pypy3.5_590/bin:$(PATH) make develop toxtest
test-py27-cffi: $(PY27)
GEVENT_CORE_CFFI_ONLY=1 PYTHON=python2.7.14 PATH=$(BUILD_RUNTIMES)/versions/python2.7.14/bin:$(PATH) make develop toxtest
......
_setuplibuv.py deleted 100644 → 0
View file @ fc34e2ff
# -*- coding: utf-8 -*-
"""
libuv build utilities.
"""
from __future__ import print_function, absolute_import, division
import ast
import os
import platform
import subprocess
import sys
from _setuputils import WIN
from _setuputils import DEFINE_MACROS
from _setuputils import Extension
from _setuputils import dep_abspath
from _setuputils import system
from _setuputils import glob_many
from _setuputils import should_embed
from _setuputils import _parse_environ
from distutils import log # pylint:disable=no-name-in-module
from distutils.errors import DistutilsError # pylint: disable=no-name-in-module,import-error
# Inspired by code from https://github.com/saghul/pyuv
LIBUV_EMBED = should_embed('libuv')
if WIN and not LIBUV_EMBED:
raise DistutilsError('using a system provided libuv is unsupported on Windows')
LIBUV_INCLUDE_DIR = dep_abspath('libuv', 'include')
# Set this to force dynamic linking of libuv, even when building the
# embedded copy. This is most useful when upgrading/changing libuv
# in place.
LIBUV_DYNAMIC_EMBED = _parse_environ("GEVENT_LIBUV_DYNAMIC_EMBED")
if LIBUV_DYNAMIC_EMBED is None:
# Not set in the environment. Are we developing?
# This is convenient for my workflow
# XXX Is there a better way to get this?
# This probably doesn't work for 'pip install -e .'
# or dev-requirments.txt
if 'develop' in sys.argv:
LIBUV_DYNAMIC_EMBED = LIBUV_EMBED
LIBUV_LIBRARIES = []
if sys.platform.startswith('linux'):
LIBUV_LIBRARIES.append('rt')
elif WIN:
LIBUV_LIBRARIES.append('advapi32')
LIBUV_LIBRARIES.append('iphlpapi')
LIBUV_LIBRARIES.append('psapi')
LIBUV_LIBRARIES.append('shell32')
LIBUV_LIBRARIES.append('userenv')
LIBUV_LIBRARIES.append('ws2_32')
elif sys.platform.startswith('freebsd'):
LIBUV_LIBRARIES.append('kvm')
if not LIBUV_EMBED or LIBUV_DYNAMIC_EMBED:
LIBUV_LIBRARIES.append('uv')
def prepare_windows_env(env):
env.pop('VS140COMNTOOLS', None)
env.pop('VS120COMNTOOLS', None)
env.pop('VS110COMNTOOLS', None)
if sys.version_info < (3, 3):
env.pop('VS100COMNTOOLS', None)
env['GYP_MSVS_VERSION'] = '2008'
else:
env['GYP_MSVS_VERSION'] = '2010'
if not env.get('PYTHON', '').endswith('.exe'):
env.pop('PYTHON', None)
if env.get('PYTHON'):
log.info("Using python from env %s", env['PYTHON'])
return # Already manually set by user.
if sys.version_info[:2] == (2, 7):
env['PYTHON'] = sys.executable
return # The current executable is fine.
# Try if `python` on PATH is the right one. If we would execute
# `python` directly the current executable might be used so we
# delegate this to cmd.
cmd = ['cmd.exe', '/C', 'python', '-c', 'import sys; '
'v = str(sys.version_info[:2]); sys.stdout.write(v); '
'sys.stdout.flush()']
try:
sub = subprocess.Popen(cmd, stdout=subprocess.PIPE)
stdout, _ = sub.communicate()
version = ast.literal_eval(stdout.decode(sys.stdout.encoding).strip()) # pylint:disable=no-member
if version == (2, 7):
return # Python on PATH is fine
except OSError:
pass
# Check default install locations
path = os.path.join('%SYSTEMDRIVE%', 'Python27', 'python.exe')
path = os.path.expandvars(path)
if os.path.isfile(path):
log.info('Using "%s" to build libuv...' % path)
env['PYTHON'] = path
# Things needed for run_with_env.cmd
# What we're building for, not what we're building with
# (because it's just the C library)
env['PYTHON_VERSION'] = str(sys.version_info[0]) + '.' + str(sys.version_info[1]) + ".x"
# XXX: Just guessing here. Is PYTHON_ARCH correct?
if 'PYTHON_ARCH' not in env:
env['PYTHON_ARCH'] = '64' if platform.architecture()[0] == '64bit' else '32'
from distutils.msvc9compiler import query_vcvarsall # pylint:disable=import-error,no-name-in-module
if sys.version_info[:2] >= (3, 5):
version = 14
else:
version = 9 # for 2.7, but probably not right for 3.4?
env.update(query_vcvarsall(version))
else:
raise DistutilsError('No appropriate Python version found. An '
'installation of 2.7 is required to '
'build libuv. You can set the environment '
'variable "PYTHON" to point to a custom '
'installation location.')
# This is a dummy extension that serves to let us hook into
# when we need to compile libuv
LIBUV = Extension(name='gevent.libuv._libuv',
sources=['src/gevent/libuv/_libuv.c'],
include_dirs=[LIBUV_INCLUDE_DIR],
libraries=LIBUV_LIBRARIES,
define_macros=list(DEFINE_MACROS),
depends=glob_many('deps/libuv/src/*.[ch]'))
if LIBUV_EMBED:
libuv_dir = dep_abspath('libuv')
if WIN:
libuv_library_dir = os.path.join(libuv_dir, 'Release', 'lib')
libuv_lib = os.path.join(libuv_library_dir, 'libuv.lib')
LIBUV.extra_link_args.extend(['/NODEFAULTLIB:libcmt', '/LTCG'])
LIBUV.extra_objects.append(libuv_lib)
else:
libuv_library_dir = os.path.join(libuv_dir, '.libs')
libuv_lib = os.path.join(libuv_library_dir, 'libuv.a')
if not LIBUV_DYNAMIC_EMBED:
LIBUV.extra_objects.append(libuv_lib)
else:
LIBUV.library_dirs.append(libuv_library_dir)
LIBUV.extra_link_args.extend(["-Wl,-rpath", libuv_library_dir])
def configure_libuv(_bext, _ext):
def build_libuv():
cflags = '-fPIC'
env = os.environ.copy()
env['CFLAGS'] = ' '.join(x for x in (cflags,
env.get('CFLAGS', None),
env.get('ARCHFLAGS', None))
if x)
# Since we're building a static library, if link-time-optimization is requested, it
# results in failure to properly create the library archive. This goes unnoticed on
# OS X until import time because of '-undefined dynamic_lookup'. On the raspberry
# pi, it causes the linker to crash
if '-flto' in env['CFLAGS']:
log.info("Removing LTO")
env['CFLAGS'] = env['CFLAGS'].replace('-flto', '')
log.info('Building libuv with cflags %s', env['CFLAGS'])
if WIN:
prepare_windows_env(env)
libuv_arch = {'32bit': 'x86', '64bit': 'x64'}[platform.architecture()[0]]
system(["cmd", "/E:ON", "/V:ON", "/C", "..\\..\\appveyor\\run_with_env.cmd",
'vcbuild.bat', libuv_arch, 'release'],
cwd=libuv_dir,
env=env,
shell=False)
else:
# autogen: requires automake and libtool installed
system(['./autogen.sh'],
cwd=libuv_dir,
env=env)
# On OS X, the linker will link to the full path
# of the library libuv *as encoded in the dylib it finds*.
# So the libdir is important and must match the actual location
# of the dynamic library if we want to dynamically link to it.
# Otherwise, we wind up linking to /usr/local/lib/libuv.dylib by
# default, which can't be found. `otool -D libuv.dylib` will show
# this name, and `otool -L src/gevent/libuv/_corecffi.so` will show
# what got linked. Note that this approach results in libuv.dylib
# apparently linking to *itself*, which is weird, but not harmful
system(['./configure', '--libdir=' + libuv_library_dir],
cwd=libuv_dir,
env=env)
system(['make'],
cwd=libuv_dir,
env=env)
if not os.path.exists(libuv_lib):
log.info('libuv needs to be compiled.')
build_libuv()
else:
log.info('No need to build libuv.')
if LIBUV_EMBED:
LIBUV.configure = configure_libuv
appveyor.yml
View file @ 5d2bcb8d
......@@ -10,12 +10,11 @@ environment:
# Pre-installed Python versions, which Appveyor may upgrade to
# a later point release.
# XXX: Sadly, PyPy won't link CFFI on Win32
# - PYTHON: "C:\\pypy-4.0.1-win32"
# PYTHON_ID: "pypy"
# PYTHON_EXE: pypy
# PYTHON_VERSION: "2.7.x"
# PYTHON_ARCH: "32"
- PYTHON: "C:\\pypy2-v5.9.0-win32"
PYTHON_ID: "pypy"
PYTHON_EXE: pypy
PYTHON_VERSION: "2.7.x"
PYTHON_ARCH: "32"
- PYTHON: "C:\\Python36-x64"
PYTHON_VERSION: "3.6.x" # currently 3.6.0
......@@ -66,6 +65,10 @@ environment:
# PYTHON_ARCH: "32"
# PYTHON_EXE: python
matrix:
allow_failures:
- PYTHON_ID: "pypy"
install:
# If there is a newer build queued for the same PR, cancel this one.
# The AppVeyor 'rollout builds' option is supposed to serve the same
......@@ -91,14 +94,11 @@ install:
New-Item -ItemType directory -Path "$env:PYTMP" | Out-Null;
}
if ("${env:PYTHON_ID}" -eq "pypy") {
if (!(Test-Path "${env:PYTMP}\pypy-4.0.1-win32.zip")) {
(New-Object Net.WebClient).DownloadFile('https://bitbucket.org/pypy/pypy/downloads/pypy-4.0.1-win32.zip', "${env:PYTMP}\pypy-4.0.1-win32.zip");
if (!(Test-Path "${env:PYTMP}\pypy2-v5.9.0-win32.zip")) {
(New-Object Net.WebClient).DownloadFile('https://bitbucket.org/pypy/pypy/downloads/pypy2-v5.9.0-win32.zip', "${env:PYTMP}\pypy2-v5.9.0-win32.zip");
}
7z x -y "${env:PYTMP}\pypy-4.0.1-win32.zip" -oC:\ | Out-Null;
if (!(Test-Path "${env:PYTMP}\get-pip.py")) {
(New-Object Net.WebClient).DownloadFile('https://bootstrap.pypa.io/get-pip.py', "${env:PYTMP}\get-pip.py");
}
& "${env:PYTHON}\pypy.exe" "${env:PYTMP}\get-pip.py";
7z x -y "${env:PYTMP}\pypy2-v5.9.0-win32.zip" -oC:\ | Out-Null;
& "${env:PYTHON}\pypy.exe" "-mensurepip";
}
elseif (-not(Test-Path($env:PYTHON))) {
......@@ -123,9 +123,13 @@ install:
# compiled extensions and are not provided as pre-built wheel packages,
# pip will build them from source using the MSVC compiler matching the
# target Python version and architecture
# NOTE: psutil won't install under PyPy.
- "%CMD_IN_ENV% pip install -U wheel cython greenlet psutil"
# Note that psutil won't build under PyPy on Windows.
- "%CMD_IN_ENV% pip install -U setuptools wheel cython greenlet cffi"
- ps:
if ("${env:PYTHON_ID}" -ne "pypy") {
pip install psutil | Out-Null;
}
- ps: "if(Test-Path(\"${env:PYTHON}\\bin\")) {ls ${env:PYTHON}\\bin;}"
- ps: "if(Test-Path(\"${env:PYTHON}\\Scripts\")) {ls ${env:PYTHON}\\Scripts;}"
......
deps/libuv/vcbuild.bat
View file @ 5d2bcb8d
......@@ -157,7 +157,9 @@ goto run
@rem Build the sln with msbuild.
:msbuild-found
msbuild uv.sln /t:%target% /p:Configuration=%config% /p:Platform="%msbuild_platform%" /clp:NoSummary;NoItemAndPropertyList;Verbosity=minimal /nologo
msbuild /ver
where msbuild
msbuild uv.sln /t:%target% /p:Configuration=%config% /p:Platform="%msbuild_platform%" /clp:NoSummary;NoItemAndPropertyList;Verbosity=minimal
if errorlevel 1 exit /b 1
:run
......
dev-requirements.txt
View file @ 5d2bcb8d
......@@ -8,8 +8,7 @@ coverage>=4.0
coveralls>=1.0
cffi
futures
# Makes tests faster, but causes issues on the old
# linux version Travis CI uses. We have a workaround.
# Makes tests faster
psutil
# For viewing README.rst (restview --long-description),
# CONTRIBUTING.rst, etc.
......
examples/processes.py
View file @ 5d2bcb8d
......@@ -3,19 +3,23 @@ from __future__ import print_function
import gevent
from gevent import subprocess
import sys
# run 2 jobs in parallel
p1 = subprocess.Popen(['uname'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['ls'], stdout=subprocess.PIPE)
if sys.platform.startswith("win"):
print("Unable to run on windows")
else:
# run 2 jobs in parallel
p1 = subprocess.Popen(['uname'], stdout=subprocess.PIPE)
p2 = subprocess.Popen(['ls'], stdout=subprocess.PIPE)
gevent.wait([p1, p2], timeout=2)
gevent.wait([p1, p2], timeout=2)
# print the results (if available)
if p1.poll() is not None:
print('uname: %r' % p1.stdout.read())
else:
print('uname: job is still running')
if p2.poll() is not None:
print('ls: %r' % p2.stdout.read())
else:
print('ls: job is still running')
# print the results (if available)
if p1.poll() is not None:
print('uname: %r' % p1.stdout.read())
else:
print('uname: job is still running')
if p2.poll() is not None:
print('ls: %r' % p2.stdout.read())
else:
print('ls: job is still running')
scripts/install.sh
View file @ 5d2bcb8d
......@@ -97,10 +97,10 @@ for var in "$@"; do
install 3.6.2 python3.6.2
;;
pypy)
install pypy2.7-5.8.0 pypy580
install pypy2.7-5.9.0 pypy590
;;
pypy3)
install pypy3.5-5.8.0 pypy3.5_580
install pypy3.5-5.9.0 pypy3.5_590
;;
esac
done
setup.py
View file @ 5d2bcb8d
......@@ -7,7 +7,7 @@ import os
from _setuputils import read
from _setuputils import read_version
from _setuputils import system
from _setuputils import PYPY, WIN, CFFI_WIN_BUILD_ANYWAY
from _setuputils import PYPY, WIN
from _setuputils import IGNORE_CFFI
from _setuputils import ConfiguringBuildExt
from _setuputils import MakeSdist
......@@ -19,15 +19,6 @@ from _setuputils import BuildFailed
from setuptools import Extension, setup
from setuptools import find_packages
if PYPY and WIN and not CFFI_WIN_BUILD_ANYWAY:
# We can't properly handle (hah!) file-descriptors and
# handle mapping on Windows/CFFI, because the file needed,
# libev_vfd.h, can't be included, linked, and used: it uses
# Python API functions, and you're not supposed to do that from
# CFFI code. Plus I could never get the libraries= line to ffi.compile()
# correct to make linking work.
raise Exception("Unable to install on PyPy/Windows")
if WIN:
# Make sure the env vars that make.cmd needs are set
if not os.environ.get('PYTHON_EXE'):
......@@ -52,7 +43,6 @@ from _setuplibev import CORE
from _setupares import ARES
from _setuplibuv import LIBUV
SEMAPHORE = Extension(name="gevent._semaphore",
sources=["src/gevent/gevent._semaphore.c"])
......@@ -68,13 +58,23 @@ EXT_MODULES = [
LOCAL,
]
cffi_modules = [
'src/gevent/libev/_corecffi_build.py:ffi',
]
LIBEV_CFFI_MODULE = 'src/gevent/libev/_corecffi_build.py:ffi'
LIBUV_CFFI_MODULE = 'src/gevent/libuv/_corecffi_build.py:ffi'
cffi_modules = []
if not WIN:
EXT_MODULES.append(LIBUV)
cffi_modules.append('src/gevent/libuv/_corecffi_build.py:ffi')
# We can't properly handle (hah!) file-descriptors and
# handle mapping on Windows/CFFI with libev, because the file needed,
# libev_vfd.h, can't be included, linked, and used: it uses
# Python API functions, and you're not supposed to do that from
# CFFI code. Plus I could never get the libraries= line to ffi.compile()
# correct to make linking work.
cffi_modules.append(
LIBEV_CFFI_MODULE
)
cffi_modules.append(LIBUV_CFFI_MODULE)
if PYPY:
install_requires = []
......@@ -124,7 +124,7 @@ if ((len(sys.argv) >= 2
'--version',
'clean',
'--long-description')))
or __name__ != '__main__'):
or __name__ != '__main__'):
_BUILDING = False
......@@ -135,7 +135,8 @@ def run_setup(ext_modules, run_make):
# to build the CFFI module. We need to configure libev
# because the CORE Extension won't.
# TODO: Generalize this.
system(libev_configure_command)
if LIBEV_CFFI_MODULE in cffi_modules and not WIN:
system(libev_configure_command)
MakeSdist.make()
......
src/gevent/_compat.py
View file @ 5d2bcb8d
......@@ -11,6 +11,7 @@ import sys
PY2 = sys.version_info[0] == 2
PY3 = sys.version_info[0] >= 3
PYPY = hasattr(sys, 'pypy_version_info')
WIN = sys.platform.startswith("win")
## Types
......@@ -21,7 +22,7 @@ if PY3:
else:
import __builtin__ # pylint:disable=import-error
string_types = __builtin__.basestring,
string_types = (__builtin__.basestring,)
text_type = __builtin__.unicode
integer_types = (int, __builtin__.long)
......
src/gevent/_ffi/loop.py
View file @ 5d2bcb8d
......@@ -177,7 +177,7 @@ if sys.version_info[0] >= 3:
integer_types = (int,)
else:
import __builtin__ # pylint:disable=import-error
basestring = __builtin__.basestring,
basestring = (__builtin__.basestring,)
integer_types = (int, __builtin__.long)
......
src/gevent/core.py
View file @ 5d2bcb8d
......@@ -19,8 +19,10 @@ except ImportError:
if lib == 'libuv':
from gevent.libuv import loop as _core
else:
from gevent.libev import corecffi as _core
try:
from gevent.libev import corecffi as _core
except ImportError:
from gevent.libuv import loop as _core
copy_globals(_core, globals())
......
src/gevent/libev/corecffi.py
View file @ 5d2bcb8d
......@@ -124,7 +124,7 @@ if sys.version_info[0] >= 3:
integer_types = (int,)
else:
import __builtin__ # pylint:disable=import-error
basestring = __builtin__.basestring,
basestring = (__builtin__.basestring,)
integer_types = (int, __builtin__.long)
......
src/gevent/libuv/_corecffi_build.py
View file @ 5d2bcb8d
......@@ -45,41 +45,202 @@ _cdef = _cdef.replace('#define GEVENT_UV_OS_SOCK_T int', '')
_cdef = _cdef.replace('GEVENT_ST_NLINK_T', st_nlink_type())
_cdef = _cdef.replace("GEVENT_STRUCT_DONE _;", '...;')
_cdef = _cdef.replace("GEVENT_UV_OS_SOCK_T", 'int' if not WIN else 'SOCKET')
# if sys.platform.startswith('win'):
# # We must have the vfd_open, etc, functions on
# # Windows. But on other platforms, going through
# # CFFI to just return the file-descriptor is slower
# # than just doing it in Python, so we check for and
# # workaround their absence in corecffi.py
# _cdef += """
# typedef int... vfd_socket_t;
# int vfd_open(vfd_socket_t);
# vfd_socket_t vfd_get(int);
# void vfd_free(int);
# """
# uv_os_sock_t is int on POSIX and SOCKET on Win32, but socket is
# just another name for handle, which is just another name for 'void*'
# which we will treat as an 'unsigned long' or 'unsigned long long'
# since it comes through 'fileno()' where it has been cast as an int.
_void_pointer_as_integer = 'unsigned long' if system_bits() == 32 else 'unsigned long long'
_cdef = _cdef.replace("GEVENT_UV_OS_SOCK_T", 'int' if not WIN else _void_pointer_as_integer)
setup_py_dir = os.path.abspath(os.path.join(thisdir, '..', '..', '..'))
libuv_dir = os.path.abspath(os.path.join(setup_py_dir, 'deps', 'libuv'))
sys.path.append(setup_py_dir)
include_dirs = [
LIBUV_INCLUDE_DIRS = [
thisdir, # libev_vfd.h
os.path.join(libuv_dir, 'include'),
os.path.join(libuv_dir, 'src'),
]
# Initially based on https://github.com/saghul/pyuv/blob/v1.x/setup_libuv.py
def _libuv_source(rel_path):
# Certain versions of setuptools, notably on windows, are *very*
# picky about what we feed to sources= "setup() arguments must
# *always* be /-separated paths relative to the setup.py
# directory, *never* absolute paths." POSIX doesn't have that issue.
path = os.path.join('deps', 'libuv', 'src', rel_path)
return path
LIBUV_SOURCES = [
_libuv_source('fs-poll.c'),
_libuv_source('inet.c'),
_libuv_source('threadpool.c'),
_libuv_source('uv-common.c'),
_libuv_source('version.c'),
]
from _setuplibuv import LIBUV_LIBRARIES # pylint:disable=import-error
from _setuplibuv import LIBUV # pylint:disable=import-error
if WIN:
LIBUV_SOURCES += [
_libuv_source('win/async.c'),
_libuv_source('win/core.c'),
_libuv_source('win/detect-wakeup.c'),
_libuv_source('win/dl.c'),
_libuv_source('win/error.c'),
_libuv_source('win/fs-event.c'),
_libuv_source('win/fs.c'),
# getaddrinfo.c refers to ConvertInterfaceIndexToLuid
# and ConvertInterfaceLuidToNameA, which are supposedly in iphlpapi.h
# and iphlpapi.lib/dll. But on Windows 10 with Python 3.5 and VC 14 (Visual Studio 2015),
# I get an undefined warning from the compiler for those functions and
# a link error from the linker, so this file can't be included.
# This is possibly because the functions are defined for Windows Vista, and
# Python 3.5 builds with at earlier SDK?
# Fortunately we don't use those functions.
#_libuv_source('win/getaddrinfo.c'),
# getnameinfo.c refers to uv__getaddrinfo_translate_error from
# getaddrinfo.c, which we don't have.
#_libuv_source('win/getnameinfo.c'),
_libuv_source('win/handle.c'),
_libuv_source('win/loop-watcher.c'),
_libuv_source('win/pipe.c'),
_libuv_source('win/poll.c'),
_libuv_source('win/process-stdio.c'),
_libuv_source('win/process.c'),
_libuv_source('win/req.c'),
_libuv_source('win/signal.c'),
_libuv_source('win/snprintf.c'),
_libuv_source('win/stream.c'),
_libuv_source('win/tcp.c'),
_libuv_source('win/thread.c'),
_libuv_source('win/timer.c'),
_libuv_source('win/tty.c'),
_libuv_source('win/udp.c'),
_libuv_source('win/util.c'),
_libuv_source('win/winapi.c'),
_libuv_source('win/winsock.c'),
]
else:
LIBUV_SOURCES += [
_libuv_source('unix/async.c'),
_libuv_source('unix/core.c'),
_libuv_source('unix/dl.c'),
_libuv_source('unix/fs.c'),
_libuv_source('unix/getaddrinfo.c'),
_libuv_source('unix/getnameinfo.c'),
_libuv_source('unix/loop-watcher.c'),
_libuv_source('unix/loop.c'),
_libuv_source('unix/pipe.c'),
_libuv_source('unix/poll.c'),
_libuv_source('unix/process.c'),
_libuv_source('unix/signal.c'),
_libuv_source('unix/stream.c'),
_libuv_source('unix/tcp.c'),
_libuv_source('unix/thread.c'),
_libuv_source('unix/timer.c'),
_libuv_source('unix/tty.c'),
_libuv_source('unix/udp.c'),
]
if sys.platform.startswith('linux'):
LIBUV_SOURCES += [
_libuv_source('unix/linux-core.c'),
_libuv_source('unix/linux-inotify.c'),
_libuv_source('unix/linux-syscalls.c'),
_libuv_source('unix/procfs-exepath.c'),
_libuv_source('unix/proctitle.c'),
_libuv_source('unix/sysinfo-loadavg.c'),
_libuv_source('unix/sysinfo-memory.c'),
]
elif sys.platform == 'darwin':
LIBUV_SOURCES += [
_libuv_source('unix/bsd-ifaddrs.c'),
_libuv_source('unix/darwin.c'),
_libuv_source('unix/darwin-proctitle.c'),
_libuv_source('unix/fsevents.c'),
_libuv_source('unix/kqueue.c'),
_libuv_source('unix/proctitle.c'),
]
elif sys.platform.startswith(('freebsd', 'dragonfly')):
LIBUV_SOURCES += [
_libuv_source('unix/bsd-ifaddrs.c'),
_libuv_source('unix/freebsd.c'),
_libuv_source('unix/kqueue.c'),
_libuv_source('unix/posix-hrtime.c'),
]
elif sys.platform.startswith('openbsd'):
LIBUV_SOURCES += [
_libuv_source('unix/bsd-ifaddrs.c'),
_libuv_source('unix/kqueue.c'),
_libuv_source('unix/openbsd.c'),
_libuv_source('unix/posix-hrtime.c'),
]
elif sys.platform.startswith('netbsd'):
LIBUV_SOURCES += [
_libuv_source('unix/bsd-ifaddrs.c'),
_libuv_source('unix/kqueue.c'),
_libuv_source('unix/netbsd.c'),
_libuv_source('unix/posix-hrtime.c'),
]
elif sys.platform.startswith('sunos'):
LIBUV_SOURCES += [
_libuv_source('unix/no-proctitle.c'),
_libuv_source('unix/sunos.c'),
]
LIBUV_MACROS = []
def _define_macro(name, value):
LIBUV_MACROS.append((name, value))
LIBUV_LIBRARIES = []
def _add_library(name):
LIBUV_LIBRARIES.append(name)
if sys.platform != 'win32':
_define_macro('_LARGEFILE_SOURCE', 1)
_define_macro('_FILE_OFFSET_BITS', 64)
if sys.platform.startswith('linux'):
_add_library('dl')
_add_library('rt')
elif sys.platform == 'darwin':
_define_macro('_DARWIN_USE_64_BIT_INODE', 1)
_define_macro('_DARWIN_UNLIMITED_SELECT', 1)
elif sys.platform.startswith('netbsd'):
_add_library('kvm')
elif sys.platform.startswith('sunos'):
_define_macro('__EXTENSIONS__', 1)
_define_macro('_XOPEN_SOURCE', 500)
_add_library('kstat')
_add_library('nsl')
_add_library('sendfile')
_add_library('socket')
elif WIN:
_define_macro('_GNU_SOURCE', 1)
_define_macro('WIN32', 1)
_define_macro('_CRT_SECURE_NO_DEPRECATE', 1)
_define_macro('_CRT_NONSTDC_NO_DEPRECATE', 1)
_define_macro('_WIN32_WINNT', '0x0600')
_add_library('advapi32')
_add_library('iphlpapi')
_add_library('psapi')
_add_library('shell32')
_add_library('user32')
_add_library('userenv')
_add_library('ws2_32')
ffi.cdef(_cdef)
ffi.set_source('gevent.libuv._corecffi', _source,
include_dirs=include_dirs,
library_dirs=LIBUV.library_dirs,
extra_objects=list(LIBUV.extra_objects),
extra_link_args=list(LIBUV.extra_link_args),
ffi.set_source('gevent.libuv._corecffi',
_source,
sources=LIBUV_SOURCES,
depends=LIBUV_SOURCES,
include_dirs=LIBUV_INCLUDE_DIRS,
libraries=list(LIBUV_LIBRARIES))
if __name__ == '__main__':
......
src/gevent/libuv/_corecffi_cdef.c
View file @ 5d2bcb8d
......@@ -163,6 +163,11 @@ typedef struct uv_fs_poll_s uv_fs_poll_t;
// callbacks with the same signature
// XXX: Note that these, and all callbacks, are defined to take
// a void* or handle* instead of the more specific, correct,
// value. This allows us to use the same gevent_generic_callback
// without having to do a bunch of casts everywhere. This does produce
// minor warnings when compiling the CFFI extension, though.
typedef void (*uv_close_cb)(uv_handle_t *handle);
typedef void (*uv_idle_cb)(void *handle);
typedef void (*uv_timer_cb)(void *handle);
......@@ -295,10 +300,16 @@ int uv_signal_stop(uv_signal_t *handle);
// Unix any file descriptor that would be accepted by poll(2) can be
// used.
int uv_poll_init(uv_loop_t *loop, uv_poll_t *handle, int fd);
// Initialize the handle using a socket descriptor. On Unix this is identical to uv_poll_init(). On windows it takes a SOCKET handle.
// The socket is set to non-blocking mode.
// On Windows, how to get the SOCKET type defined?
//int uv_poll_init_socket(uv_loop_t* loop, uv_poll_t* handle, GEVENT_UV_OS_SOCK_T socket);
// Initialize the handle using a socket descriptor. On Unix this is
// identical to uv_poll_init(). On windows it takes a SOCKET handle;
// SOCKET handles are another name for HANDLE objects in win32, and
// those are defined as PVOID, even though they are not actually
// pointers (they're small integers). CPython and PyPy both return
// the SOCKET (as cast to an int) from the socket.fileno() method.
// libuv uses ``uv_os_sock_t`` for this type, which is defined as an
// int on unix.
int uv_poll_init_socket(uv_loop_t* loop, uv_poll_t* handle, GEVENT_UV_OS_SOCK_T socket);
int uv_poll_start(uv_poll_t *handle, int events, uv_poll_cb cb);
int uv_poll_stop(uv_poll_t *handle);
......
src/gevent/libuv/_libuv.c deleted 100644 → 0
View file @ fc34e2ff
This diff is collapsed.
src/gevent/libuv/_libuv.py deleted 100644 → 0
View file @ fc34e2ff
# Empty module, exists to build libuv library
src/gevent/libuv/loop.py
View file @ 5d2bcb8d
......@@ -185,6 +185,13 @@ class loop(AbstractLoop):
"""
Return all the handles that are open and their ref status.
"""
# XXX: Disabled because, at least on Windows, the times this
# gets called often produce `SystemError: ffi.from_handle():
# dead or bogus handle object`, and sometimes that crashes the process.
return []
def _really_debug(self):
handle_state = namedtuple("HandleState",
['handle',
'watcher',
......
src/gevent/libuv/watcher.py
View file @ 5d2bcb8d
......@@ -3,6 +3,7 @@
from __future__ import absolute_import, print_function
import functools
import sys
import weakref
import gevent.libuv._corecffi as _corecffi # pylint:disable=no-name-in-module,import-error
......@@ -10,7 +11,6 @@ import gevent.libuv._corecffi as _corecffi # pylint:disable=no-name-in-module,im
ffi = _corecffi.ffi
libuv = _corecffi.lib
from gevent._ffi import watcher as _base
_closing_handles = set()
......@@ -28,11 +28,10 @@ def _dbg(*args, **kwargs):
def _pid_dbg(*args, **kwargs):
import os
import sys
kwargs['file'] = sys.stderr
print(os.getpid(), *args, **kwargs)
# _dbg = _pid_dbg
#_dbg = _pid_dbg
_events = [(libuv.UV_READABLE, "READ"),
(libuv.UV_WRITABLE, "WRITE")]
......@@ -224,14 +223,21 @@ class io(_base.IoMixin, watcher):
def _watcher_ffi_start(self):
self._watcher_start(self._watcher, self._events, self._watcher_callback)
if sys.platform.startswith('win32'):
# We can only handle sockets. We smuggle the SOCKET through disguised
# as a fileno
_watcher_init = watcher._LIB.uv_poll_init_socket
class _multiplexwatcher(object):
callback = None
args = ()
pass_events = False
ref = True
def __init__(self, events, watcher):
self.events = events
self.callback = None
self.args = ()
self.pass_events = False
self.ref = True
# References:
# These objects keep the original IO object alive;
......@@ -242,6 +248,7 @@ class io(_base.IoMixin, watcher):
self._watcher_ref = watcher
def start(self, callback, *args, **kwargs):
_dbg("Starting IO multiplex watcher for", self.fd, callback)
self.pass_events = kwargs.get("pass_events")
self.callback = callback
self.args = args
......@@ -251,6 +258,7 @@ class io(_base.IoMixin, watcher):
watcher._io_start()
def stop(self):
_dbg("Stopping IO multiplex watcher for", self.fd, self.callback)
self.callback = None
self.pass_events = None
self.args = None
......@@ -310,14 +318,22 @@ class io(_base.IoMixin, watcher):
# the reader, we get a LoopExit. So we can't return here and arguably shouldn't print it
# either. The negative events mask will match the watcher's mask.
# See test__fileobject.py:Test.test_newlines for an example.
# On Windows (at least with PyPy), we can get ENOTSOCK (socket operation on non-socket)
# if a socket gets closed. If we don't pass the events on, we hang.
# See test__makefile_ref.TestSSL for examples.
# return
#_dbg("Callback event for watcher", self._fd, "event", events)
for watcher_ref in self._multiplex_watchers:
watcher = watcher_ref()
if not watcher or not watcher.callback:
continue
if events & watcher.events:
#_dbg("Event for watcher", self._fd, events, watcher.events, events & watcher.events)
send_event = (events & watcher.events) or events < 0
if send_event:
if not watcher.pass_events:
watcher.callback(*watcher.args)
else:
......
src/greentest/2.7pypy/allsans.pem 0 → 100644
View file @ 5d2bcb8d
-----BEGIN PRIVATE KEY-----
MIICdgIBADANBgkqhkiG9w0BAQEFAASCAmAwggJcAgEAAoGBAOoy7/QOtTjQ0niE
6uDcTwtkC0R2Tvy1AjVnXohCntZfdzbTGDoYTgXSOLsP8A697jUiJ8VCePGH50xG
Z4DKnAF3a9O3a9nr2pLXb0iY3XOMv+YEBii7CfI+3oxFYgCl0sMgHzDD2ZTVYAsm
DWgLUVsE2gHEccRwrM2tPf2EgR+FAgMBAAECgYEA3qyfyYVSeTrTYxO93x6ZaVMu
A2IZp9zSxMQL9bKiI2GRj+cV2ebSCGbg2btFnD6qBor7FWsmYz+8g6FNN/9sY4az
61rMqMtQvLBe+7L8w70FeTze4qQ4Y1oQri0qD6tBWhDVlpnbI5Py9bkZKD67yVUk
elcEA/5x4PrYXkuqsAECQQD80NjT0mDvaY0JOOaQFSEpMv6QiUA8GGX8Xli7IoKb
tAolPG8rQBa+qSpcWfDMTrWw/aWHuMEEQoP/bVDH9W4FAkEA7SYQbBAKnojZ5A3G
kOHdV7aeivRQxQk/JN8Fb8oKB9Csvpv/BsuGxPKXHdhFa6CBTTsNRtHQw/szPo4l
xMIjgQJAPoMxqibR+0EBM6+TKzteSL6oPXsCnBl4Vk/J5vPgkbmR7KUl4+7j8N8J
b2554TrxKEN/w7CGYZRE6UrRd7ATNQJAWD7Yz41sli+wfPdPU2xo1BHljyl4wMk/
EPZYbI/PCbdyAH/F935WyQTIjNeEhZc1Zkq6FwdOWw8ns3hrv3rKgQJAHXv1BqUa
czGPIFxX2TNoqtcl6/En4vrxVB1wzsfzkkDAg98kBl7qsF+S3qujSzKikjeaVbI2
/CyWR2P3yLtOmA==
-----END PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIIDcjCCAtugAwIBAgIJAN5dc9TOWjB7MA0GCSqGSIb3DQEBCwUAMF0xCzAJBgNV
BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u
IFNvZnR3YXJlIEZvdW5kYXRpb24xEDAOBgNVBAMMB2FsbHNhbnMwHhcNMTYwODA1
MTAyMTExWhcNMjYwODAzMTAyMTExWjBdMQswCQYDVQQGEwJYWTEXMBUGA1UEBwwO
Q2FzdGxlIEFudGhyYXgxIzAhBgNVBAoMGlB5dGhvbiBTb2Z0d2FyZSBGb3VuZGF0
aW9uMRAwDgYDVQQDDAdhbGxzYW5zMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB
gQDqMu/0DrU40NJ4hOrg3E8LZAtEdk78tQI1Z16IQp7WX3c20xg6GE4F0ji7D/AO
ve41IifFQnjxh+dMRmeAypwBd2vTt2vZ69qS129ImN1zjL/mBAYouwnyPt6MRWIA
pdLDIB8ww9mU1WALJg1oC1FbBNoBxHHEcKzNrT39hIEfhQIDAQABo4IBODCCATQw
ggEwBgNVHREEggEnMIIBI4IHYWxsc2Fuc6AeBgMqAwSgFwwVc29tZSBvdGhlciBp
ZGVudGlmaWVyoDUGBisGAQUCAqArMCmgEBsOS0VSQkVST1MuUkVBTE2hFTAToAMC
AQGhDDAKGwh1c2VybmFtZYEQdXNlckBleGFtcGxlLm9yZ4IPd3d3LmV4YW1wbGUu
b3JnpGcwZTELMAkGA1UEBhMCWFkxFzAVBgNVBAcMDkNhc3RsZSBBbnRocmF4MSMw
IQYDVQQKDBpQeXRob24gU29mdHdhcmUgRm91bmRhdGlvbjEYMBYGA1UEAwwPZGly
bmFtZSBleGFtcGxlhhdodHRwczovL3d3dy5weXRob24ub3JnL4cEfwAAAYcQAAAA
AAAAAAAAAAAAAAAAAYgEKgMEBTANBgkqhkiG9w0BAQsFAAOBgQAy16h+F+nOmeiT
VWR0fc8F/j6FcadbLseAUaogcC15OGxCl4UYpLV88HBkABOoGCpP155qwWTwOrdG
iYPGJSusf1OnJEbvzFejZf6u078bPd9/ZL4VWLjv+FPGkjd+N+/OaqMvgj8Lu99f
3Y/C4S7YbHxxwff6C6l2Xli+q6gnuQ==
-----END CERTIFICATE-----
src/greentest/2.7pypy/keycert2.pem 0 → 100644
View file @ 5d2bcb8d
-----BEGIN PRIVATE KEY-----
MIICdQIBADANBgkqhkiG9w0BAQEFAASCAl8wggJbAgEAAoGBANcLaMB7T/Wi9DBc
PltGzgt8cxsv55m7PQPHMZvn6Ke8xmNqcmEzib8opRwKGrCV6TltKeFlNSg8dwQK
Tl4ktyTkGCVweRQJ37AkBayvEBml5s+QD4vlhqkJPsL/Nsd+fnqngOGc5+59+C6r
s3XpiLlF5ah/z8q92Mnw54nypw1JAgMBAAECgYBE3t2Mj7GbDLZB6rj5yKJioVfI
BD6bSJEQ7bGgqdQkLFwpKMU7BiN+ekjuwvmrRkesYZ7BFgXBPiQrwhU5J28Tpj5B
EOMYSIOHfzdalhxDGM1q2oK9LDFiCotTaSdEzMYadel5rmKXJ0zcK2Jho0PCuECf
tf/ghRxK+h1Hm0tKgQJBAO6MdGDSmGKYX6/5kPDje7we/lSLorSDkYmV0tmVShsc
JxgaGaapazceA/sHL3Myx7Eenkip+yPYDXEDFvAKNDECQQDmxsT9NOp6mo7ISvky
GFr2vVHsJ745BMWoma4rFjPBVnS8RkgK+b2EpDCdZSrQ9zw2r8sKTgrEyrDiGTEg
wJyZAkA8OOc0flYMJg2aHnYR6kwVjPmGHI5h5gk648EMPx0rROs1sXkiUwkHLCOz
HvhCq+Iv+9vX2lnVjbiu/CmxRdIxAkA1YEfzoKeTD+hyXxTgB04Sv5sRGegfXAEz
i8gC4zG5R/vcCA1lrHmvEiLEZL/QcT6WD3bQvVg0SAU9ZkI8pxARAkA7yqMSvP1l
gJXy44R+rzpLYb1/PtiLkIkaKG3x9TUfPnfD2jY09fPkZlfsRU3/uS09IkhSwimV
d5rWoljEfdou
-----END PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIICXTCCAcagAwIBAgIJALVQzebTtrXFMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNV
BAYTAlhZMRcwFQYDVQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9u
IFNvZnR3YXJlIEZvdW5kYXRpb24xFTATBgNVBAMMDGZha2Vob3N0bmFtZTAeFw0x
NDExMjMxNzAwMDdaFw0yNDExMjAxNzAwMDdaMGIxCzAJBgNVBAYTAlhZMRcwFQYD
VQQHDA5DYXN0bGUgQW50aHJheDEjMCEGA1UECgwaUHl0aG9uIFNvZnR3YXJlIEZv
dW5kYXRpb24xFTATBgNVBAMMDGZha2Vob3N0bmFtZTCBnzANBgkqhkiG9w0BAQEF
AAOBjQAwgYkCgYEA1wtowHtP9aL0MFw+W0bOC3xzGy/nmbs9A8cxm+fop7zGY2py
YTOJvyilHAoasJXpOW0p4WU1KDx3BApOXiS3JOQYJXB5FAnfsCQFrK8QGaXmz5AP
i+WGqQk+wv82x35+eqeA4Zzn7n34LquzdemIuUXlqH/Pyr3YyfDnifKnDUkCAwEA
AaMbMBkwFwYDVR0RBBAwDoIMZmFrZWhvc3RuYW1lMA0GCSqGSIb3DQEBBQUAA4GB
AKuay3vDKfWzt5+ch/HHBsert84ISot4fUjzXDA/oOgTOEjVcSShHxqNShMOW1oA
QYBpBB/5Kx5RkD/w6imhucxt2WQPRgjX4x4bwMipVH/HvFDp03mG51/Cpi1TyZ74
El7qa/Pd4lHhOLzMKBA6503fpeYSFUIBxZbGLqylqRK7
-----END CERTIFICATE-----
src/greentest/2.7pypy/test_ftplib.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/2.7pypy/test_httplib.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/2.7pypy/test_httpservers.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/2.7pypy/test_select.py
View file @ 5d2bcb8d
......@@ -29,9 +29,9 @@ class SelectTestCase(unittest.TestCase):
self.assertIsNot(w, x)
def test_select(self):
cmd = 'for i in 0 1 2 3 4 5 6 7 8 9; do echo testing...; sleep 0.1; done'
cmd = 'for i in 0 1 2 3 4 5 6 7 8 9; do echo testing...; sleep 1; done'
p = os.popen(cmd, 'r')
for tout in (0, 0.1, 0.2, 0.4, 0.8, 1.6) + (None,)*10:
for tout in (0, 1, 2, 4, 8, 16) + (None,)*10:
if test_support.verbose:
print 'timeout =', tout
rfd, wfd, xfd = select.select([p], [], [], tout)
......
src/greentest/2.7pypy/test_smtplib.py
View file @ 5d2bcb8d
......@@ -179,31 +179,31 @@ class DebuggingServerTests(unittest.TestCase):
def testBasic(self):
# connect
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3)
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15)
smtp.quit()
def testNOOP(self):
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3)
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15)
expected = (250, 'Ok')
self.assertEqual(smtp.noop(), expected)
smtp.quit()
def testRSET(self):
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3)
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15)
expected = (250, 'Ok')
self.assertEqual(smtp.rset(), expected)
smtp.quit()
def testNotImplemented(self):
# EHLO isn't implemented in DebuggingServer
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3)
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15)
expected = (502, 'Error: command "EHLO" not implemented')
self.assertEqual(smtp.ehlo(), expected)
smtp.quit()
def testVRFY(self):
# VRFY isn't implemented in DebuggingServer
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3)
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15)
expected = (502, 'Error: command "VRFY" not implemented')
self.assertEqual(smtp.vrfy('nobody@nowhere.com'), expected)
self.assertEqual(smtp.verify('nobody@nowhere.com'), expected)
......@@ -212,21 +212,21 @@ class DebuggingServerTests(unittest.TestCase):
def testSecondHELO(self):
# check that a second HELO returns a message that it's a duplicate
# (this behavior is specific to smtpd.SMTPChannel)
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3)
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15)
smtp.helo()
expected = (503, 'Duplicate HELO/EHLO')
self.assertEqual(smtp.helo(), expected)
smtp.quit()
def testHELP(self):
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3)
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15)
self.assertEqual(smtp.help(), 'Error: command "HELP" not implemented')
smtp.quit()
def testSend(self):
# connect and send mail
m = 'A test message'
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=3)
smtp = smtplib.SMTP(HOST, self.port, local_hostname='localhost', timeout=15)
smtp.sendmail('John', 'Sally', m)
# XXX(nnorwitz): this test is flaky and dies with a bad file descriptor
# in asyncore. This sleep might help, but should really be fixed
......@@ -292,6 +292,33 @@ class BadHELOServerTests(unittest.TestCase):
HOST, self.port, 'localhost', 3)
@unittest.skipUnless(threading, 'Threading required for this test.')
class TooLongLineTests(unittest.TestCase):
respdata = '250 OK' + ('.' * smtplib._MAXLINE * 2) + '\n'
def setUp(self):
self.old_stdout = sys.stdout
self.output = StringIO.StringIO()
sys.stdout = self.output
self.evt = threading.Event()
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.settimeout(15)
self.port = test_support.bind_port(self.sock)
servargs = (self.evt, self.respdata, self.sock)
threading.Thread(target=server, args=servargs).start()
self.evt.wait()
self.evt.clear()
def tearDown(self):
self.evt.wait()
sys.stdout = self.old_stdout
def testLineTooLong(self):
self.assertRaises(smtplib.SMTPResponseException, smtplib.SMTP,
HOST, self.port, 'localhost', 3)
sim_users = {'Mr.A@somewhere.com':'John A',
'Ms.B@somewhere.com':'Sally B',
'Mrs.C@somewhereesle.com':'Ruth C',
......@@ -507,11 +534,27 @@ class SMTPSimTests(unittest.TestCase):
#TODO: add tests for correct AUTH method fallback now that the
#test infrastructure can support it.
def test_quit_resets_greeting(self):
smtp = smtplib.SMTP(HOST, self.port,
local_hostname='localhost',
timeout=15)
code, message = smtp.ehlo()
self.assertEqual(code, 250)
self.assertIn('size', smtp.esmtp_features)
smtp.quit()
self.assertNotIn('size', smtp.esmtp_features)
smtp.connect(HOST, self.port)
self.assertNotIn('size', smtp.esmtp_features)
smtp.ehlo_or_helo_if_needed()
self.assertIn('size', smtp.esmtp_features)
smtp.quit()
def test_main(verbose=None):
test_support.run_unittest(GeneralTests, DebuggingServerTests,
NonConnectingTests,
BadHELOServerTests, SMTPSimTests)
BadHELOServerTests, SMTPSimTests,
TooLongLineTests)
if __name__ == '__main__':
test_main()
src/greentest/2.7pypy/test_socket.py
View file @ 5d2bcb8d
......@@ -2,6 +2,7 @@ import unittest
from test import test_support
import errno
import itertools
import socket
import select
import time
......@@ -11,9 +12,14 @@ import sys
import os
import array
import contextlib
from weakref import proxy
import signal
import math
import weakref
try:
import _socket
except ImportError:
_socket = None
def try_address(host, port=0, family=socket.AF_INET):
"""Try to bind a socket on the given host:port and return True
......@@ -80,7 +86,7 @@ class ThreadableTest:
clientTearDown ()
Any new test functions within the class must then define
tests in pairs, where the test name is preceeded with a
tests in pairs, where the test name is preceded with a
'_' to indicate the client portion of the test. Ex:
def testFoo(self):
......@@ -243,9 +249,22 @@ class SocketPairTest(unittest.TestCase, ThreadableTest):
class GeneralModuleTests(unittest.TestCase):
@unittest.skipUnless(_socket is not None, 'need _socket module')
def test_csocket_repr(self):
s = _socket.socket(_socket.AF_INET, _socket.SOCK_STREAM)
try:
expected = ('<socket object, fd=%s, family=%s, type=%s, protocol=%s>'
% (s.fileno(), s.family, s.type, s.proto))
self.assertEqual(repr(s), expected)
finally:
s.close()
expected = ('<socket object, fd=-1, family=%s, type=%s, protocol=%s>'
% (s.family, s.type, s.proto))
self.assertEqual(repr(s), expected)
def test_weakref(self):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
p = proxy(s)
p = weakref.proxy(s)
self.assertEqual(p.fileno(), s.fileno())
s.close()
s = None
......@@ -257,6 +276,14 @@ class GeneralModuleTests(unittest.TestCase):
else:
self.fail('Socket proxy still exists')
def test_weakref__sock(self):
s = socket.socket()._sock
w = weakref.ref(s)
self.assertIs(w(), s)
del s
test_support.gc_collect()
self.assertIsNone(w())
def testSocketError(self):
# Testing socket module exceptions
def raise_error(*args, **kwargs):
......@@ -273,7 +300,7 @@ class GeneralModuleTests(unittest.TestCase):
"Error raising socket exception.")
def testSendtoErrors(self):
# Testing that sendto doens't masks failures. See #10169.
# Testing that sendto doesn't mask failures. See #10169.
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self.addCleanup(s.close)
s.bind(('', 0))
......@@ -603,17 +630,24 @@ class GeneralModuleTests(unittest.TestCase):
sock.close()
def test_getsockaddrarg(self):
host = '0.0.0.0'
port = self._get_unused_port(bind_address=host)
sock = socket.socket()
self.addCleanup(sock.close)
port = test_support.find_unused_port()
big_port = port + 65536
neg_port = port - 65536
sock = socket.socket()
try:
self.assertRaises((OverflowError, ValueError), sock.bind, (host, big_port))
self.assertRaises((OverflowError, ValueError), sock.bind, (host, neg_port))
sock.bind((host, port))
finally:
sock.close()
self.assertRaises((OverflowError, ValueError), sock.bind, (HOST, big_port))
self.assertRaises((OverflowError, ValueError), sock.bind, (HOST, neg_port))
# Since find_unused_port() is inherently subject to race conditions, we
# call it a couple times if necessary.
for i in itertools.count():
port = test_support.find_unused_port()
try:
sock.bind((HOST, port))
except OSError as e:
if e.errno != errno.EADDRINUSE or i == 5:
raise
else:
break
@unittest.skipUnless(os.name == "nt", "Windows specific")
def test_sock_ioctl(self):
......@@ -677,7 +711,7 @@ class GeneralModuleTests(unittest.TestCase):
pass
def check_sendall_interrupted(self, with_timeout):
# socketpair() is not stricly required, but it makes things easier.
# socketpair() is not strictly required, but it makes things easier.
if not hasattr(signal, 'alarm') or not hasattr(socket, 'socketpair'):
self.skipTest("signal.alarm and socket.socketpair required for this test")
# Our signal handlers clobber the C errno by calling a math function
......@@ -706,7 +740,6 @@ class GeneralModuleTests(unittest.TestCase):
c.close()
s.close()
@unittest.skip("Needs fix in CFFI; hangs forever")
def test_sendall_interrupted(self):
self.check_sendall_interrupted(False)
......@@ -797,7 +830,7 @@ class BasicTCPTest(SocketConnectedTest):
self.serv_conn.sendall(big_chunk)
@unittest.skipUnless(hasattr(socket, 'fromfd'),
'socket.fromfd not availble')
'socket.fromfd not available')
def testFromFd(self):
# Testing fromfd()
fd = self.cli_conn.fileno()
......@@ -1292,15 +1325,11 @@ class NetworkConnectionNoServer(unittest.TestCase):
def mocked_socket_module(self):
"""Return a socket which times out on connect"""
old_socket = socket.socket
import gevent.socket
old_g_socket = gevent.socket.socket
socket.socket = self.MockSocket
gevent.socket.socket = self.MockSocket
try:
yield
finally:
socket.socket = old_socket
gevent.socket.socket = old_g_socket
def test_connect(self):
port = test_support.find_unused_port()
......@@ -1734,7 +1763,7 @@ class TIPCThreadableTest(unittest.TestCase, ThreadableTest):
self.conn, self.connaddr = self.srv.accept()
def clientSetUp(self):
# The is a hittable race between serverExplicitReady() and the
# There is a hittable race between serverExplicitReady() and the
# accept() call; sleep a little while to avoid it, otherwise
# we could get an exception
time.sleep(0.1)
......
src/greentest/2.7pypy/test_socketserver.py
View file @ 5d2bcb8d
......@@ -158,6 +158,8 @@ class SocketServerTest(unittest.TestCase):
if verbose: print "waiting for server"
server.shutdown()
t.join()
server.server_close()
self.assertRaises(socket.error, server.socket.fileno)
if verbose: print "done"
def stream_examine(self, proto, addr):
......@@ -173,6 +175,8 @@ class SocketServerTest(unittest.TestCase):
def dgram_examine(self, proto, addr):
s = socket.socket(proto, socket.SOCK_DGRAM)
if HAVE_UNIX_SOCKETS and proto == socket.AF_UNIX:
s.bind(self.pickaddr(proto))
s.sendto(TEST_STR, addr)
buf = data = receive(s, 100)
while data and '\n' not in buf:
......@@ -267,27 +271,24 @@ class SocketServerTest(unittest.TestCase):
# Make sure select was called again:
self.assertGreater(mock_select.called, 1)
# Alas, on Linux (at least) recvfrom() doesn't return a meaningful
# client address so this cannot work:
# @requires_unix_sockets
# def test_UnixDatagramServer(self):
# self.run_server(SocketServer.UnixDatagramServer,
# SocketServer.DatagramRequestHandler,
# self.dgram_examine)
#
# @requires_unix_sockets
# def test_ThreadingUnixDatagramServer(self):
# self.run_server(SocketServer.ThreadingUnixDatagramServer,
# SocketServer.DatagramRequestHandler,
# self.dgram_examine)
#
# @requires_unix_sockets
# @requires_forking
# def test_ForkingUnixDatagramServer(self):
# self.run_server(SocketServer.ForkingUnixDatagramServer,
# SocketServer.DatagramRequestHandler,
# self.dgram_examine)
@requires_unix_sockets
def test_UnixDatagramServer(self):
self.run_server(SocketServer.UnixDatagramServer,
SocketServer.DatagramRequestHandler,
self.dgram_examine)
@requires_unix_sockets
def test_ThreadingUnixDatagramServer(self):
self.run_server(SocketServer.ThreadingUnixDatagramServer,
SocketServer.DatagramRequestHandler,
self.dgram_examine)
@requires_unix_sockets
@requires_forking
def test_ForkingUnixDatagramServer(self):
self.run_server(ForkingUnixDatagramServer,
SocketServer.DatagramRequestHandler,
self.dgram_examine)
@reap_threads
def test_shutdown(self):
......@@ -314,6 +315,40 @@ class SocketServerTest(unittest.TestCase):
for t, s in threads:
t.join()
def test_tcpserver_bind_leak(self):
# Issue #22435: the server socket wouldn't be closed if bind()/listen()
# failed.
# Create many servers for which bind() will fail, to see if this result
# in FD exhaustion.
for i in range(1024):
with self.assertRaises(OverflowError):
SocketServer.TCPServer((HOST, -1),
SocketServer.StreamRequestHandler)
class MiscTestCase(unittest.TestCase):
def test_shutdown_request_called_if_verify_request_false(self):
# Issue #26309: BaseServer should call shutdown_request even if
# verify_request is False
class MyServer(SocketServer.TCPServer):
def verify_request(self, request, client_address):
return False
shutdown_called = 0
def shutdown_request(self, request):
self.shutdown_called += 1
SocketServer.TCPServer.shutdown_request(self, request)
server = MyServer((HOST, 0), SocketServer.StreamRequestHandler)
s = socket.socket(server.address_family, socket.SOCK_STREAM)
s.connect(server.server_address)
s.close()
server.handle_request()
self.assertEqual(server.shutdown_called, 1)
server.server_close()
def test_main():
if imp.lock_held():
......
src/greentest/2.7pypy/test_ssl.py
View file @ 5d2bcb8d
......@@ -26,6 +26,9 @@ ssl = support.import_module("ssl")
PROTOCOLS = sorted(ssl._PROTOCOL_NAMES)
HOST = support.HOST
IS_LIBRESSL = ssl.OPENSSL_VERSION.startswith('LibreSSL')
IS_OPENSSL_1_1 = not IS_LIBRESSL and ssl.OPENSSL_VERSION_INFO >= (1, 1, 0)
def data_file(*name):
return os.path.join(os.path.dirname(__file__), *name)
......@@ -57,6 +60,8 @@ CRLFILE = data_file("revocation.crl")
SIGNED_CERTFILE = data_file("keycert3.pem")
SIGNED_CERTFILE2 = data_file("keycert4.pem")
SIGNING_CA = data_file("pycacert.pem")
# cert with all kinds of subject alt names
ALLSANFILE = data_file("allsans.pem")
REMOTE_HOST = "self-signed.pythontest.net"
REMOTE_ROOT_CERT = data_file("selfsigned_pythontestdotnet.pem")
......@@ -164,7 +169,6 @@ class BasicSocketTests(unittest.TestCase):
self.assertIn(ssl.HAS_SNI, {True, False})
self.assertIn(ssl.HAS_ECDH, {True, False})
def test_random(self):
v = ssl.RAND_status()
if support.verbose:
......@@ -245,6 +249,27 @@ class BasicSocketTests(unittest.TestCase):
self.assertEqual(p['subjectAltName'], san)
def test_parse_all_sans(self):
p = ssl._ssl._test_decode_cert(ALLSANFILE)
self.assertEqual(p['subjectAltName'],
(
('DNS', 'allsans'),
('othername', '<unsupported>'),
('othername', '<unsupported>'),
('email', 'user@example.org'),
('DNS', 'www.example.org'),
('DirName',
((('countryName', 'XY'),),
(('localityName', 'Castle Anthrax'),),
(('organizationName', 'Python Software Foundation'),),
(('commonName', 'dirname example'),))),
('URI', 'https://www.python.org/'),
('IP Address', '127.0.0.1'),
('IP Address', '0:0:0:0:0:0:0:1\n'),
('Registered ID', '1.2.3.4.5')
)
)
def test_DER_to_PEM(self):
with open(CAFILE_CACERT, 'r') as f:
pem = f.read()
......@@ -281,9 +306,9 @@ class BasicSocketTests(unittest.TestCase):
self.assertGreaterEqual(status, 0)
self.assertLessEqual(status, 15)
# Version string as returned by {Open,Libre}SSL, the format might change
if "LibreSSL" in s:
self.assertTrue(s.startswith("LibreSSL {:d}.{:d}".format(major, minor)),
(s, t))
if IS_LIBRESSL:
self.assertTrue(s.startswith("LibreSSL {:d}".format(major)),
(s, t, hex(n)))
else:
self.assertTrue(s.startswith("OpenSSL {:d}.{:d}.{:d}".format(major, minor, fix)),
(s, t))
......@@ -742,15 +767,15 @@ class ContextTests(unittest.TestCase):
def test_options(self):
ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
# OP_ALL | OP_NO_SSLv2 | OP_NO_SSLv3 is the default value
self.assertEqual(ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3,
ctx.options)
default = (ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3)
if not IS_LIBRESSL and ssl.OPENSSL_VERSION_INFO >= (1, 1, 0):
default |= ssl.OP_NO_COMPRESSION
self.assertEqual(default, ctx.options)
ctx.options |= ssl.OP_NO_TLSv1
self.assertEqual(ssl.OP_ALL | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3 | ssl.OP_NO_TLSv1,
ctx.options)
self.assertEqual(default | ssl.OP_NO_TLSv1, ctx.options)
if can_clear_options():
ctx.options = (ctx.options & ~ssl.OP_NO_SSLv2) | ssl.OP_NO_TLSv1
self.assertEqual(ssl.OP_ALL | ssl.OP_NO_TLSv1 | ssl.OP_NO_SSLv3,
ctx.options)
ctx.options = (ctx.options & ~ssl.OP_NO_TLSv1)
self.assertEqual(default, ctx.options)
ctx.options = 0
self.assertEqual(0, ctx.options)
else:
......@@ -1088,6 +1113,7 @@ class ContextTests(unittest.TestCase):
self.assertRaises(TypeError, ctx.load_default_certs, 'SERVER_AUTH')
@unittest.skipIf(sys.platform == "win32", "not-Windows specific")
@unittest.skipIf(IS_LIBRESSL, "LibreSSL doesn't support env vars")
def test_load_default_certs_env(self):
ctx = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
with support.EnvironmentVarGuard() as env:
......@@ -1534,7 +1560,6 @@ class NetworkedTests(unittest.TestCase):
sys.stdout.write("%s\n" % x)
else:
self.fail("Got server certificate %s for %s:%s!" % (pem, host, port))
pem = ssl.get_server_certificate((host, port),
ca_certs=cert)
if not pem:
......@@ -2489,8 +2514,6 @@ else:
def test_asyncore_server(self):
"""Check the example asyncore integration."""
indata = "TEST MESSAGE of mixed case\n"
if support.verbose:
sys.stdout.write("\n")
......@@ -2783,7 +2806,7 @@ else:
with closing(context.wrap_socket(socket.socket())) as s:
self.assertIs(s.version(), None)
s.connect((HOST, server.port))
self.assertEqual(s.version(), "TLSv1")
self.assertEqual(s.version(), 'TLSv1')
self.assertIs(s.version(), None)
@unittest.skipUnless(ssl.HAS_ECDH, "test requires ECDH-enabled OpenSSL")
......@@ -2925,24 +2948,36 @@ else:
(['http/3.0', 'http/4.0'], None)
]
for client_protocols, expected in protocol_tests:
server_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
server_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
server_context.load_cert_chain(CERTFILE)
server_context.set_alpn_protocols(server_protocols)
client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
client_context.load_cert_chain(CERTFILE)
client_context.set_alpn_protocols(client_protocols)
stats = server_params_test(client_context, server_context,
chatty=True, connectionchatty=True)
msg = "failed trying %s (s) and %s (c).\n" \
"was expecting %s, but got %%s from the %%s" \
% (str(server_protocols), str(client_protocols),
str(expected))
client_result = stats['client_alpn_protocol']
self.assertEqual(client_result, expected, msg % (client_result, "client"))
server_result = stats['server_alpn_protocols'][-1] \
if len(stats['server_alpn_protocols']) else 'nothing'
self.assertEqual(server_result, expected, msg % (server_result, "server"))
try:
stats = server_params_test(client_context,
server_context,
chatty=True,
connectionchatty=True)
except ssl.SSLError as e:
stats = e
if expected is None and IS_OPENSSL_1_1:
# OpenSSL 1.1.0 raises handshake error
self.assertIsInstance(stats, ssl.SSLError)
else:
msg = "failed trying %s (s) and %s (c).\n" \
"was expecting %s, but got %%s from the %%s" \
% (str(server_protocols), str(client_protocols),
str(expected))
client_result = stats['client_alpn_protocol']
self.assertEqual(client_result, expected,
msg % (client_result, "client"))
server_result = stats['server_alpn_protocols'][-1] \
if len(stats['server_alpn_protocols']) else 'nothing'
self.assertEqual(server_result, expected,
msg % (server_result, "server"))
def test_selected_npn_protocol(self):
# selected_npn_protocol() is None unless NPN is used
......
src/greentest/2.7pypy/test_subprocess.py
View file @ 5d2bcb8d
......@@ -20,7 +20,6 @@ except ImportError:
threading = None
mswindows = (sys.platform == "win32")
PYPY = hasattr(sys, 'pypy_version_info')
#
# Depends on the following external programs: Python
......@@ -33,16 +32,6 @@ PYPY = hasattr(sys, 'pypy_version_info')
# SETBINARY = ''
try:
mkstemp = tempfile.mkstemp
except AttributeError:
# tempfile.mkstemp is not available
def mkstemp():
"""Replacement for mkstemp, calling mktemp."""
fname = tempfile.mktemp()
return os.open(fname, os.O_RDWR|os.O_CREAT), fname
class BaseTestCase(unittest.TestCase):
def setUp(self):
# Try to minimize the number of children we have so this test
......@@ -194,8 +183,8 @@ class ProcessTestCase(BaseTestCase):
p.wait()
self.assertEqual(p.returncode, 47)
@unittest.skipIf(sysconfig.is_python_build() or PYPY,
"need an installed Python. See #7774. Also fails to get sys.prefix on stock PyPy")
@unittest.skipIf(sysconfig.is_python_build(),
"need an installed Python. See #7774")
def test_executable_without_cwd(self):
# For a normal installation, it should work without 'cwd'
# argument. For test runs in the build directory, see #7774.
......@@ -296,6 +285,27 @@ class ProcessTestCase(BaseTestCase):
tf.seek(0)
self.assertStderrEqual(tf.read(), "strawberry")
def test_stderr_redirect_with_no_stdout_redirect(self):
# test stderr=STDOUT while stdout=None (not set)
# - grandchild prints to stderr
# - child redirects grandchild's stderr to its stdout
# - the parent should get grandchild's stderr in child's stdout
p = subprocess.Popen([sys.executable, "-c",
'import sys, subprocess;'
'rc = subprocess.call([sys.executable, "-c",'
' "import sys;"'
' "sys.stderr.write(\'42\')"],'
' stderr=subprocess.STDOUT);'
'sys.exit(rc)'],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
stdout, stderr = p.communicate()
#NOTE: stdout should get stderr from grandchild
self.assertStderrEqual(stdout, b'42')
self.assertStderrEqual(stderr, b'') # should be empty
self.assertEqual(p.returncode, 0)
def test_stdout_stderr_pipe(self):
# capture stdout and stderr to the same pipe
p = subprocess.Popen([sys.executable, "-c",
......@@ -416,7 +426,7 @@ class ProcessTestCase(BaseTestCase):
def test_communicate_pipe_fd_leak(self):
fd_directory = '/proc/%d/fd' % os.getpid()
num_fds_before_popen = len(os.listdir(fd_directory))
p = subprocess.Popen([sys.executable, "-c", "print()"],
p = subprocess.Popen([sys.executable, "-c", "print('')"],
stdout=subprocess.PIPE)
p.communicate()
num_fds_after_communicate = len(os.listdir(fd_directory))
......@@ -669,9 +679,9 @@ class ProcessTestCase(BaseTestCase):
def test_handles_closed_on_exception(self):
# If CreateProcess exits with an error, ensure the
# duplicate output handles are released
ifhandle, ifname = mkstemp()
ofhandle, ofname = mkstemp()
efhandle, efname = mkstemp()
ifhandle, ifname = tempfile.mkstemp()
ofhandle, ofname = tempfile.mkstemp()
efhandle, efname = tempfile.mkstemp()
try:
subprocess.Popen (["*"], stdin=ifhandle, stdout=ofhandle,
stderr=efhandle)
......@@ -861,7 +871,7 @@ class POSIXProcessTestCase(BaseTestCase):
def test_args_string(self):
# args is a string
f, fname = mkstemp()
f, fname = tempfile.mkstemp()
os.write(f, "#!/bin/sh\n")
os.write(f, "exec '%s' -c 'import sys; sys.exit(47)'\n" %
sys.executable)
......@@ -905,7 +915,7 @@ class POSIXProcessTestCase(BaseTestCase):
def test_call_string(self):
# call() function with string argument on UNIX
f, fname = mkstemp()
f, fname = tempfile.mkstemp()
os.write(f, "#!/bin/sh\n")
os.write(f, "exec '%s' -c 'import sys; sys.exit(47)'\n" %
sys.executable)
......@@ -1061,7 +1071,7 @@ class POSIXProcessTestCase(BaseTestCase):
def check_swap_fds(self, stdin_no, stdout_no, stderr_no):
# open up some temporary files
temps = [mkstemp() for i in range(3)]
temps = [tempfile.mkstemp() for i in range(3)]
temp_fds = [fd for fd, fname in temps]
try:
# unlink the files -- we won't need to reopen them
......@@ -1384,7 +1394,7 @@ class CommandsWithSpaces (BaseTestCase):
def setUp(self):
super(CommandsWithSpaces, self).setUp()
f, fname = mkstemp(".py", "te st")
f, fname = tempfile.mkstemp(".py", "te st")
self.fname = fname.lower ()
os.write(f, b"import sys;"
b"sys.stdout.write('%d %s' % (len(sys.argv), [a.lower () for a in sys.argv]))"
......
src/greentest/2.7pypy/test_telnetlib.py
View file @ 5d2bcb8d
......@@ -248,8 +248,8 @@ class ReadTests(TestCase):
func = getattr(telnet, func_name)
self.assertRaises(EOFError, func)
# read_eager and read_very_eager make the same gaurantees
# (they behave differently but we only test the gaurantees)
# read_eager and read_very_eager make the same guarantees
# (they behave differently but we only test the guarantees)
def test_read_very_eager_A(self):
self._test_read_any_eager_A('read_very_eager')
def test_read_very_eager_B(self):
......
src/greentest/2.7pypy/test_thread.py
View file @ 5d2bcb8d
......@@ -234,7 +234,12 @@ class TestForkInThread(unittest.TestCase):
if pid == 0: # child
os.close(self.read_fd)
os.write(self.write_fd, "OK")
sys.exit(0)
# Exiting the thread normally in the child process can leave
# any additional threads (such as the one started by
# importing _tkinter) still running, and this can prevent
# the half-zombie child process from being cleaned up. See
# Issue #26456.
os._exit(0)
else: # parent
os.close(self.write_fd)
......
src/greentest/2.7pypy/test_threading.py
View file @ 5d2bcb8d
......@@ -51,7 +51,7 @@ class TestThread(threading.Thread):
self.nrunning.inc()
if verbose:
print self.nrunning.get(), 'tasks are running'
self.testcase.assertTrue(self.nrunning.get() <= 3)
self.testcase.assertLessEqual(self.nrunning.get(), 3)
time.sleep(delay)
if verbose:
......@@ -59,7 +59,7 @@ class TestThread(threading.Thread):
with self.mutex:
self.nrunning.dec()
self.testcase.assertTrue(self.nrunning.get() >= 0)
self.testcase.assertGreaterEqual(self.nrunning.get(), 0)
if verbose:
print '%s is finished. %d tasks are running' % (
self.name, self.nrunning.get())
......@@ -92,25 +92,25 @@ class ThreadTests(BaseTestCase):
for i in range(NUMTASKS):
t = TestThread("<thread %d>"%i, self, sema, mutex, numrunning)
threads.append(t)
self.assertEqual(t.ident, None)
self.assertTrue(re.match('<TestThread\(.*, initial\)>', repr(t)))
self.assertIsNone(t.ident)
self.assertRegexpMatches(repr(t), r'^<TestThread\(.*, initial\)>$')
t.start()
if verbose:
print 'waiting for all tasks to complete'
for t in threads:
t.join(NUMTASKS)
self.assertTrue(not t.is_alive())
self.assertFalse(t.is_alive())
self.assertNotEqual(t.ident, 0)
self.assertFalse(t.ident is None)
self.assertTrue(re.match('<TestThread\(.*, \w+ -?\d+\)>', repr(t)))
self.assertIsNotNone(t.ident)
self.assertRegexpMatches(repr(t), r'^<TestThread\(.*, \w+ -?\d+\)>$')
if verbose:
print 'all tasks done'
self.assertEqual(numrunning.get(), 0)
def test_ident_of_no_threading_threads(self):
# The ident still must work for the main thread and dummy threads.
self.assertFalse(threading.currentThread().ident is None)
self.assertIsNotNone(threading.currentThread().ident)
def f():
ident.append(threading.currentThread().ident)
done.set()
......@@ -118,7 +118,7 @@ class ThreadTests(BaseTestCase):
ident = []
thread.start_new_thread(f, ())
done.wait()
self.assertFalse(ident[0] is None)
self.assertIsNotNone(ident[0])
# Kill the "immortal" _DummyThread
del threading._active[ident[0]]
......@@ -237,7 +237,7 @@ class ThreadTests(BaseTestCase):
self.assertTrue(ret)
if verbose:
print " verifying worker hasn't exited"
self.assertTrue(not t.finished)
self.assertFalse(t.finished)
if verbose:
print " attempting to raise asynch exception in worker"
result = set_async_exc(ctypes.c_long(t.id), exception)
......@@ -706,52 +706,6 @@ class ThreadJoinOnShutdown(BaseTestCase):
output = "end of worker thread\nend of main thread\n"
self.assertScriptHasOutput(script, output)
@unittest.skipIf(sys.platform in platforms_to_skip, "due to known OS bug")
@unittest.skipIf(sys.pypy_version_info[:2] >= (2,6),
"This test was removed in CPython 2.7.9, and fails under PyPy 2.6 "
"with 'RuntimeError: stream lock is not held' in random_io")
def test_6_daemon_threads(self):
# Check that a daemon thread cannot crash the interpreter on shutdown
# by manipulating internal structures that are being disposed of in
# the main thread.
script = """if True:
import os
import random
import sys
import time
import threading
thread_has_run = set()
def random_io():
'''Loop for a while sleeping random tiny amounts and doing some I/O.'''
while True:
in_f = open(os.__file__, 'rb')
stuff = in_f.read(200)
null_f = open(os.devnull, 'wb')
null_f.write(stuff)
time.sleep(random.random() / 1995)
null_f.close()
in_f.close()
thread_has_run.add(threading.current_thread())
def main():
count = 0
for _ in range(40):
new_thread = threading.Thread(target=random_io)
new_thread.daemon = True
new_thread.start()
count += 1
while len(thread_has_run) < count:
time.sleep(0.001)
# Trigger process shutdown
sys.exit(0)
main()
"""
rc, out, err = assert_python_ok('-c', script)
self.assertFalse(err)
@unittest.skipUnless(hasattr(os, 'fork'), "needs os.fork()")
@unittest.skipIf(sys.platform in platforms_to_skip, "due to known OS bug")
def test_reinit_tls_after_fork(self):
......@@ -791,7 +745,7 @@ class ThreadJoinOnShutdown(BaseTestCase):
def generator():
while 1:
yield "genereator"
yield "generator"
def callback():
if callback.gen is None:
......@@ -838,6 +792,85 @@ class ThreadingExceptionTests(BaseTestCase):
thread.start()
self.assertRaises(RuntimeError, setattr, thread, "daemon", True)
def test_print_exception(self):
script = r"""if 1:
import threading
import time
running = False
def run():
global running
running = True
while running:
time.sleep(0.01)
1.0/0.0
t = threading.Thread(target=run)
t.start()
while not running:
time.sleep(0.01)
running = False
t.join()
"""
rc, out, err = assert_python_ok("-c", script)
self.assertEqual(out, '')
self.assertIn("Exception in thread", err)
self.assertIn("Traceback (most recent call last):", err)
self.assertIn("ZeroDivisionError", err)
self.assertNotIn("Unhandled exception", err)
def test_print_exception_stderr_is_none_1(self):
script = r"""if 1:
import sys
import threading
import time
running = False
def run():
global running
running = True
while running:
time.sleep(0.01)
1.0/0.0
t = threading.Thread(target=run)
t.start()
while not running:
time.sleep(0.01)
sys.stderr = None
running = False
t.join()
"""
rc, out, err = assert_python_ok("-c", script)
self.assertEqual(out, '')
self.assertIn("Exception in thread", err)
self.assertIn("Traceback (most recent call last):", err)
self.assertIn("ZeroDivisionError", err)
self.assertNotIn("Unhandled exception", err)
def test_print_exception_stderr_is_none_2(self):
script = r"""if 1:
import sys
import threading
import time
running = False
def run():
global running
running = True
while running:
time.sleep(0.01)
1.0/0.0
sys.stderr = None
t = threading.Thread(target=run)
t.start()
while not running:
time.sleep(0.01)
running = False
t.join()
"""
rc, out, err = assert_python_ok("-c", script)
self.assertEqual(out, '')
self.assertNotIn("Unhandled exception", err)
class LockTests(lock_tests.LockTests):
locktype = staticmethod(threading.Lock)
......@@ -849,7 +882,7 @@ class EventTests(lock_tests.EventTests):
eventtype = staticmethod(threading.Event)
class ConditionAsRLockTests(lock_tests.RLockTests):
# An Condition uses an RLock by default and exports its API.
# Condition uses an RLock by default and exports its API.
locktype = staticmethod(threading.Condition)
class ConditionTests(lock_tests.ConditionTests):
......
src/greentest/2.7pypy/test_threading_local.py
View file @ 5d2bcb8d
import unittest
from doctest import DocTestSuite
from test import test_support
from test import test_support as support
import weakref
import gc
# Modules under test
_thread = test_support.import_module('thread')
threading = test_support.import_module('threading')
_thread = support.import_module('thread')
threading = support.import_module('threading')
import _threading_local
......@@ -35,7 +35,6 @@ class BaseLocalTest:
del t
gc.collect()
gc.collect() # needed when run through gevent; why?
self.assertEqual(len(weaklist), n)
# XXX _threading_local keeps the local of the last stopped thread alive.
......@@ -64,14 +63,9 @@ class BaseLocalTest:
# Simply check that the variable is correctly set
self.assertEqual(local.x, i)
threads= []
for i in range(10):
t = threading.Thread(target=f, args=(i,))
t.start()
threads.append(t)
for t in threads:
t.join()
with support.start_threads(threading.Thread(target=f, args=(i,))
for i in range(10)):
pass
def test_derived_cycle_dealloc(self):
# http://bugs.python.org/issue6990
......@@ -174,7 +168,7 @@ class BaseLocalTest:
obj = cls()
obj.x = 5
self.assertEqual(obj.__dict__, {'x': 5})
if test_support.check_impl_detail():
if support.check_impl_detail():
with self.assertRaises(AttributeError):
obj.__dict__ = {}
with self.assertRaises(AttributeError):
......@@ -203,7 +197,7 @@ class ThreadLocalTest(unittest.TestCase, BaseLocalTest):
wr = weakref.ref(x)
del x
gc.collect()
self.assertIs(wr(), None)
self.assertIsNone(wr())
class PyThreadingLocalTest(unittest.TestCase, BaseLocalTest):
_local = _threading_local.local
......@@ -230,7 +224,7 @@ def test_main():
setUp=setUp, tearDown=tearDown)
)
test_support.run_unittest(suite)
support.run_unittest(suite)
if __name__ == '__main__':
test_main()
src/greentest/2.7pypy/test_urllib.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/2.7pypy/test_urllib2.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/2.7pypy/test_urllib2net.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/2.7pypy/test_wsgiref.py
View file @ 5d2bcb8d
from __future__ import nested_scopes # Backward compat for 2.1
from unittest import TestCase
from wsgiref.util import setup_testing_defaults
from wsgiref.headers import Headers
from wsgiref.handlers import BaseHandler, BaseCGIHandler
from wsgiref import util
from wsgiref.validate import validator
from wsgiref.simple_server import WSGIServer, WSGIRequestHandler, demo_app
from wsgiref.simple_server import WSGIServer, WSGIRequestHandler
from wsgiref.simple_server import make_server
from StringIO import StringIO
from SocketServer import BaseServer
import os
import re
import sys
......@@ -113,6 +113,11 @@ class IntegrationTests(TestCase):
out, err = run_amock()
self.check_hello(out)
def test_request_length(self):
out, err = run_amock(data="GET " + ("x" * 65537) + " HTTP/1.0\n\n")
self.assertEqual(out.splitlines()[0],
"HTTP/1.0 414 Request-URI Too Long")
def test_validated_hello(self):
out, err = run_amock(validator(hello_app))
# the middleware doesn't support len(), so content-length isn't there
......
src/greentest/2.7pypy/version
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/greentest.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/known_failures.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/patched_tests_setup.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/test__core.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/test__core_stat.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/test__core_watcher.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/test__makefile_ref.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/test__os.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/test__socket_close.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/test__threadpool.py
View file @ 5d2bcb8d
This diff is collapsed.
src/greentest/testrunner.py
View file @ 5d2bcb8d
This diff is collapsed.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7