1. 29 Aug, 2017 2 commits
    • Jason Madden's avatar
      Use a higher pickle protocol for serializing objects on Python 2 (#179) · be5a9d54
      Jason Madden authored
      * Use a higher pickle protocol (2) for serializing objects on Python 2
      
      Previously protocol 1 was used. This is more efficient for new-style
      classes (all persistent objects are new-style), according to the docs,
      at the cost of being very slightly less space efficient for old-style
      classes.
      
      In tests of a persistent object with two trivial numeric attributes,
      the higher protocol was 12 bytes smaller, and serialized and
      deserialized 1us faster. Introducing a reference to another new-style
      class for a more realistic test made the higher protocol twice as fast
      to serialize (20.5 vs 10.3us), almost half the size (215 vs 142
      bytes), and it deserialized 30% faster (6.5 vs 4.6us).
      
      On Python 2, this will now allow open ``file`` objects to be
      pickled (loading the object will result in a closed file); previously
      this would result in a ``TypeError`` (as does under Python 3). We had
      tests that you couldn't do that with a BlobFile so I had to update it
      to still make that true.
      
      I wouldn't recommend serializing arbitrary open files under Python
      2 (for one thing, they can't trivially be deserialized in Python 3),
      but I didn't take any steps to prevent it either. Since this hasn't
      been possible, there shouldn't be code in the wild that is trying to
      do it---and it wouldn't be forward compatible with Python 3 either.
      be5a9d54
    • XeL64's avatar
      fix typo (#180) · b340b651
      XeL64 authored
      b340b651
  2. 25 Jul, 2017 4 commits
  3. 24 Jul, 2017 11 commits
  4. 17 May, 2017 5 commits
  5. 17 Apr, 2017 1 commit
    • Jason Madden's avatar
      Make all classes new-style. (#160) · e714f515
      Jason Madden authored
      * Make all classes new-style.
      
      On PyPy, it's documented (http://pypy.org/performance.html#id3) that
      old-style classes are slow, and classes that derive from both old and
      new are especially slow. I checked with the PyPy devs on IRC today and
      they confirmed that's still true with the latest PyPy2 releases.
      
      Unfortunately, FileStorage was one such mixed class, as was Connection.
      
      Moving to new-style classes seems to have a positive impact in the
      benchmarks. Here's zodbshootout on PyPy2 5.7.1 against a
      FileStorage (running in --threads and with 5 reps to be sure we get
      warmed up). First, current ZODB:
      
      "Transaction",                fs
      "Add 1000 Objects",            31,738
      "Update 1000 Objects",         42,444
      "Read 1000 Cold Objects",      51,894
      "Read 1000 Hot Objects",       53,187
      "Read 1000 Steamin' Objects", 835,877
      
      And with this PR:
      
      "Transaction",                fs
      "Add 1000 Objects",             35,651
      "Update 1000 Objects",          54,906
      "Read 1000 Cold Objects",      103,484
      "Read 1000 Hot Objects",        84,721
      "Read 1000 Steamin' Objects", 2,112,095
      
      The tests that hit the storage extensively are notably faster, as are
      steamin and hot, Connection having been a mixed class.
      
      I ran all tests multiple times. The data files were removed between
      runs. There's some variation, but the new-style classes always seem
      better.
      
      For comparison, here's CPython 2.7.13:
      
      "Transaction",                fs
      "Add 1000 Objects",            19,531
      "Update 1000 Objects",         16,201
      "Read 1000 Cold Objects",      22,111
      "Read 1000 Hot Objects",       21,851
      "Read 1000 Steamin' Objects", 880,582
      
      Locally I've run the tests under 2.7 and they all passed.
      e714f515
  6. 14 Apr, 2017 1 commit
    • Jason Madden's avatar
      Speed up ZODB.blob.BushyLayout.oid_to_path on Python 3 (#161) · 3580ddd8
      Jason Madden authored
      Profiling (https://github.com/zodb/zodbshootout/pull/32/) showed that
      this method was the only blob-related method that showed up in a test
      of creating blobs, other than those that actually performed IO.
      
      With this change its total and cumulative time drops from 0.003/0.004
      to 0.001/0.002 in a small benchmark. Blobs created per second shows a
      small but consistent improvement.
      
      Before:
      
         ncalls  tottime  percall  cumtime  percall filename:lineno(function)
            100    0.005    0.000    0.005    0.000 {built-in method rename}
            100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x1080d3a60}
            200    0.003    0.000    0.004    0.000 blob.py:576(oid_to_path)
            101    0.003    0.000    0.003    0.000 {built-in method mkdir}
            100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
            402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
              1    0.002    0.002    0.034    0.034 Connection.py:553(_store_objects)
            201    0.002    0.000    0.002    0.000 {built-in method stat}
           5633    0.001    0.000    0.002    0.000 {built-in method isinstance}
      
      After:
      
         ncalls  tottime  percall  cumtime  percall filename:lineno(function)
            100    0.005    0.000    0.005    0.000 {built-in method rename}
            101    0.005    0.000    0.005    0.000 {built-in method mkdir}
            100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x10636aa60}
            402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
            100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
              1    0.002    0.002    0.035    0.035 Connection.py:553(_store_objects)
            201    0.002    0.000    0.002    0.000 {built-in method stat}
           4033    0.001    0.000    0.001    0.000 {built-in method isinstance}
         ....
            200    0.001    0.000    0.002    0.000 blob.py:576(oid_to_path)
      3580ddd8
  7. 11 Apr, 2017 8 commits
  8. 09 Apr, 2017 1 commit
  9. 08 Apr, 2017 7 commits