- 30 Jan, 2017 1 commit
-
-
Kirill Smelkov authored
Recently we are starting to get following errors on a production instance: File ".../bin/runzope", line 211, in <module> sys.exit(Zope2.Startup.run.run()) File ".../eggs/Zope2-2.13.24-py2.7.egg/Zope2/Startup/run.py", line 26, in run starter.run() File ".../eggs/Zope2-2.13.24-py2.7.egg/Zope2/Startup/__init__.py", line 111, in run Lifetime.loop() File ".../eggs/Zope2-2.13.24-py2.7.egg/Lifetime/__init__.py", line 43, in loop lifetime_loop() File ".../eggs/Zope2-2.13.24-py2.7.egg/Lifetime/__init__.py", line 53, in lifetime_loop asyncore.poll(timeout, map) File ".../parts/python2.7/lib/python2.7/asyncore.py", line 145, in poll r, w, e = select.select(r, w, e, timeout) ValueError: filedescriptor out of range in select() Initially we thought the reason here is that number of file descriptors this process uses goes beyond allowed limit (65K open files on that instance) because wendelin.core used 1 fd per an array view (opening /dev/shm/ramh.XXXXX) but that turned out to be not strictly that case: The reason here is that select() has limit for how many #fd it can monitor at all. The limit is system-specific but on Linux it is usually 1024 - http://man7.org/linux/man-pages/man2/select.2.html#BUGS : $ cat s.c #include <sys/select.h> #include <stdio.h> int main() { printf("%d\n", FD_SETSIZE); return 0; } $ tcc -run s.c 1024 Also select() can monitor only file descriptors which are by itself are "< FD_SETSIZE", e.g. it cannot monitor fd=1025 even if there are only 10 opened file descriptors because 1025 is not < 1024. As was said above in wendelin.core every array view uses 1 file descriptor, so if we are using not so small amount of arrays, even though #fd does not go beyond process-specific ulimit, because of select() usage the total number of allowed opened file descriptors becomes essentially 1024. So let's switch from select() to poll() which does not have this 1024 #fd limit. Asyncore already support using poll() out of the box - either via passing use_pull=True to asyncore.loop() https://docs.python.org/2/library/asyncore.html#asyncore.loop or by using asyncore.poll2() instead of asyncore.poll() https://github.com/python/cpython/blob/2.7/Lib/asyncore.py#L170 https://github.com/python/cpython/blob/2.7/Lib/asyncore.py#L125 -------- This patch switches asyncore.poll() -> asyncore.poll2() for only 1 place in Zope. There are however many such places in Zope and other software. For this reason, for me, what makes sense to do is not to patch all such places, but instead change via runtime-patching asyncore.poll to be asyncore.poll2 - this way all software will automatically benefit from poll() usage instead of select. P.S. What might also make sense to do in the future is to actually let asyncore.poll to use epoll(), as both select() and poll() are doing all fd registration on _every_ call, so when #fd grows this preparation time grows too. For epoll() file descriptors are registered only once. For this to work asyncore.socket_map has to be patched also, since there are places in code which modify this fd registry directly (e.g. remove fd from there etc)
-
- 18 Jan, 2017 2 commits
-
-
Maurits van Rees authored
In functional doc tests you can apparently have a test case that has no runTest method. Until now the Testing package added a dummy runTest method in that case, and set it to None. But when this dummy runTest method gets called, you get an error: Error in test runTest (Testing.ZopeTestCase.ZopeTestCase.FunctionalTestCase) Traceback (most recent call last): File ".../lib/python2.7/unittest/case.py", line 329, in run testMethod() TypeError: 'NoneType' object is not callable Simply importing Testing.ZopeTestCase.FunctionalTestCase in a new test file may be enough to trigger this. So this has something to do with the order in which tests are found. I fixed it by making the dummy runTest method callable.
-
Tres Seaver authored
Apply plonehotfix 20170717 [2.13]
-
- 17 Jan, 2017 2 commits
-
-
Maurits van Rees authored
This applies PloneHotfix20170117.
-
Maurits van Rees authored
-
- 13 Jan, 2017 7 commits
-
-
Hanno Schlichting authored
-
Hanno Schlichting authored
-
Hanno Schlichting authored
-
Hanno Schlichting authored
-
Hanno Schlichting authored
-
Hanno Schlichting authored
-
Hanno Schlichting authored
-
- 27 Dec, 2016 2 commits
-
-
Maurits van Rees authored
Patch zope.interface to remove docstrings and avoid publishing.
-
Maurits van Rees authored
From Products.PloneHotfix20161129. Signed-off-by: Maurits van Rees <maurits@vanrees.org>
-
- 21 Dec, 2016 3 commits
-
-
Tres Seaver authored
Don't copy items the user is not allowed to view. [2.13]
-
Maurits van Rees authored
-
Maurits van Rees authored
-
- 08 Dec, 2016 1 commit
-
-
Maurits van Rees authored
From Products.PloneHotfix20161129.
-
- 15 Sep, 2016 2 commits
-
-
Tres Seaver authored
Add support for optional 'SameSite' cookie attribute
-
Cédric Le Ninivin authored
-
- 14 Sep, 2016 1 commit
-
-
Cédric Le Ninivin authored
As described in the definition document by the ietf: https://tools.ietf.org/html/draft-west-first-party-cookies-07 "The 'SameSite' attribute allows servers to assert that a cookie ought not to be sent along with cross-site requests. This assertion allows user agents to mitigate the risk of cross-origin information leakage, and provides some protection against cross-site request forgery attacks."
-
- 09 Sep, 2016 1 commit
-
-
Hanno Schlichting authored
Revert "Optimize 'OFS.ObjectManager.__contains__' method"
-
- 08 Sep, 2016 1 commit
-
-
Maurits van Rees authored
It causes problems with ZCatalog indexes. See https://github.com/zopefoundation/Zope/issues/69 This reverts commit 753683e3.
-
- 07 Sep, 2016 3 commits
-
-
Hanno Schlichting authored
Quote variables in manage_tabs and manage_container to avoid XSS [2.13]
-
Maurits van Rees authored
From Products.PloneHotfix20160830.
-
Maurits van Rees authored
-
- 02 Aug, 2016 1 commit
-
-
Hanno Schlichting authored
-
- 01 Aug, 2016 1 commit
-
-
Hanno Schlichting authored
The 3.0 release of the project contains no code. It allows projects to declare a dependency on it for forward compatibility with Zope 4.
-
- 17 Jul, 2016 3 commits
-
-
Hanno Schlichting authored
Apply hotfix 20160419 rebased
-
Maurits van Rees authored
-
Maurits van Rees authored
From Products.PloneHotfix20160419.
-
- 29 Jun, 2016 2 commits
-
-
Tres Seaver authored
Optimize "OFS.ObjectManager.__contains__" method (backport)
-
Malthe Borch authored
Backported from master.
-
- 04 Mar, 2016 2 commits
-
-
Maurits van Rees authored
fix ReST rendering problem in changelog
-
Jens W. Klein authored
-
- 29 Feb, 2016 5 commits
-
-
Tres Seaver authored
[ci skip]
-
Tres Seaver authored
[ci skip]
-
Tres Seaver authored
[ci skip]
-
https://github.com/gweis/ZopeTres Seaver authored
-
-