1. 21 May, 2015 1 commit
  2. 19 Jun, 2014 1 commit
    • client: in iterator records, export data serial as stored by NEO · e76af297
      There is simply no way to guess data serials and instead of producing random
      values, the only solution is to retrieve the values from storages.
      
      There are still differences for data serials between FileStorage and NEO:
      - NEO always resolves to original serial, to avoid any indirection
        (which slightly speeds up undo at the expense of a more complex pack code)
      - NEO does not make any difference between object deletion and creation undone
        (data serial always null in storage)
      It has to be decided whether NEO implementation should be changed about this.
      
      Apart from that, conversion database back from NEO should be fixed.
      testExportFileStorageBug passes and there was in fact no FileStorage bug.
      
      Another change is that iterator does not trash the client cache anymore.
      Julien Muchembled committed
  3. 04 Jun, 2014 1 commit
  4. 03 Jun, 2014 1 commit
  5. 07 Jan, 2014 4 commits
  6. 04 Jan, 2014 1 commit
  7. 13 Dec, 2013 1 commit
  8. 26 Aug, 2012 1 commit
  9. 23 Aug, 2012 2 commits
  10. 20 Aug, 2012 1 commit
  11. 23 Jul, 2012 1 commit
  12. 19 Jul, 2012 1 commit
    • client: fix cache invalidation during a load from a storage for the same oid · 04ab2a4c
      This fixes the following an invalidation bug:
      
      ERROR ZODB.Connection Couldn't load state for 0x504e
      Traceback (most recent call last):
        File "ZODB/Connection.py", line 851, in setstate
          self._setstate(obj)
        File "ZODB/Connection.py", line 916, in _setstate
          self._load_before_or_conflict(obj)
        File "ZODB/Connection.py", line 931, in _load_before_or_conflict
          if not self._setstate_noncurrent(obj):
        File "ZODB/Connection.py", line 954, in _setstate_noncurrent
          assert end is not None
      AssertionError
      Julien Muchembled committed
  13. 18 Jul, 2012 1 commit
    • client: always process invalidations in poll thread · bce3bc78
      This fixes an invalidation bug, including the following critical error:
      
      CRITICAL txn.140440071526144 A storage error occurred during the second phase of the two-phase commit.  Resources may be in an inconsistent state.
      ------
      ERROR Zope.SiteErrorLog 1342544345.990.582646288246 /erp5/person_module/Folder_create
      Traceback (innermost last):
        Module ZPublisher.Publish, line 137, in publish
        Module Zope2.App.startup, line 291, in commit
        Module transaction._manager, line 93, in commit
        Module transaction._transaction, line 322, in commit
        Module transaction._transaction, line 424, in _commitResources
        Module neo.client, line 42, in tpc_finish
        Module neo.client.Storage, line 135, in tpc_finish
        Module neo.client.app, line 773, in tpc_finish
        Module neo.client, line 36, in callback
        Module ZODB.DB, line 693, in invalidate
        Module ZODB.DB, line 532, in _connectionMap
        Module ZODB.DB, line 221, in map
        Module transaction.weakset, line 58, in map
        Module ZODB.DB, line 692, in inval
        Module ZODB.Connection, line 350, in invalidate
      AssertionError: invalidations out of order, '\x03\x97\xec;\x19\x86\xc9\xf6' < '\x03\x97\xec;\x19\x87_\xdd'
      Julien Muchembled committed
  14. 13 Jul, 2012 1 commit
  15. 26 Mar, 2012 1 commit
  16. 21 Mar, 2012 1 commit
  17. 20 Mar, 2012 1 commit
    • Review logging to keep all debugging information in RAM and flush only if useful · 1fce5cc4
      The main goal of this patch is to keep all DEBUG logs and packet logger enabled
      without exploding disk usage.
      This is done by keeping the last 16 MB (by default) of debugging information in
      a RAM buffer, and to emit it to an SQLite DB upon RTMIN signal or in case of
      warning and more severe log.
      
      Implementation is also cleaned up for better integration within a framework
      or if run standalone. NEO logger is now a direct child of root handler.
      Only warnings and more severe logs are forwarded to root handler.
      
      A new script 'neolog' is added to pretty-print the contents of the SQLite log.
      
      In unit tests, logging events are not buffered but emitted immediately.
      When a test passes, payloads of all exchanged packets are discarded to reduce
      disk usage on test bots.
      
      This slows down performance tests by about 15 % because even if nothing is
      written to disk, debug and packet log records are now always rendered.
      Julien Muchembled committed
  18. 19 Mar, 2012 4 commits
  19. 16 Mar, 2012 1 commit
  20. 13 Mar, 2012 1 commit
  21. 13 Feb, 2012 1 commit
  22. 10 Feb, 2012 1 commit
  23. 19 Jan, 2012 2 commits
  24. 17 Jan, 2012 1 commit
  25. 10 Jan, 2012 1 commit
  26. 09 Jan, 2012 1 commit
  27. 06 Jan, 2012 1 commit
  28. 02 Jan, 2012 1 commit
  29. 26 Oct, 2011 1 commit
  30. 11 Oct, 2011 1 commit
    • Fix protocol and DB schema so that storages can handle transactions of any size · d5c469be
      - Change protocol to use SHA1 for all checksums:
        - Use SHA1 instead of CRC32 for data checksums.
        - Use SHA1 instead of MD5 for replication.
      
      - Change DatabaseManager API so that backends can store raw data separately from
        object metadata:
        - When processing AskStoreObject, call the backend to store the data
          immediately, instead of keeping it in RAM or in the temporary object table.
          Data is then referenced only by its checksum.
          Without such change, the storage could fail to store the transaction due to
          lack of RAM, or it could make tpc_finish step very slow.
        - Backends have to store data in a separate space, and remove entries as soon
          as they get unreferenced. So they must have an index of checksums in object
          metadata space. A new '_uncommitted_data' backend attribute keeps references
          of uncommitted data.
        - New methods: _pruneData, _storeData, storeData, unlockData
        - MySQL: change vertical partitioning of 'obj' by having data in a separate
          'data' table instead of using a shortened 'obj_short' table.
        - BTree: data is moved from '_obj' to a new '_data' btree.
      
      - Undo is optimized so that backpointers are not required anymore to fetch data:
        - The checksum of an object is None only when creation is undone.
        - Removed DatabaseManager methods: _getObjectData, _getDataTIDFromData
        - DatabaseManager: move some code from _getDataTID to findUndoTID so that
          _getDataTID only has what's specific to backend.
      
      - Removed because already covered by ZODB tests:
        - neo.tests.storage.testStorageDBTests.StorageDBTests.test__getDataTID
        - neo.tests.storage.testStorageDBTests.StorageDBTests.test__getDataTIDFromData
      Julien Muchembled committed
  31. 16 Sep, 2011 1 commit
  32. 10 Jun, 2011 1 commit