Whats new in ZODB 3.8.6 (2010-??-??)
====================================
Bugs Fixed:
- Updating blobs in save points could cause spurious "invalidations
out of order" errors. https://bugs.launchpad.net/zodb/+bug/509801
(Thanks to Christian Zagrodnick for chasing this down.)
- BTree sets and tree sets didn't correctly check values passed to
update or to constructors, causing Python to exit under certain
circumstances.
- Passing keys or values outside the range of 32-bit ints on 64-bit
platforms led to undetected overflow errors. Now these cases cause
Overflow errors to be raised.
https://bugs.launchpad.net/zodb/+bug/143237
- Fixed a serious bug that caused cache failures when run
with Python optimization turned on.
https://bugs.launchpad.net/zodb/+bug/544305
Whats new in ZODB 3.8.5 (2009-12-16)
====================================
Bug Fixed:
- A ZEO threading bug could cause transactions to read inconsistent
data. (This sometimes caused an AssertionError in
Connection._setstate_noncurrent.)
New Feature:
- The standard storages, FileStorage, ClientStorage, DemoStorage, and
MappingStorage now allow the version argument to history and load to
be ommitted. This is to make it easier to write application code
that works with ZODB 3.8 and later versions, which don't support
versions.
Whats new in ZODB 3.8.4 (2009-10-01)
====================================
Bugs Fixed:
- Conflict errors committing blobs caused ZEO servers to stop committing
transactions.
Whats new in ZODB 3.8.3
=======================
New Feature:
- There's a new utility script, strip_versions that strips version
data from storages. This is needed to prepare databases containing
version records for using ZODB 3.9, which no-longer supports
versions.
Bugs Fixed:
- CVE-2009-2701: Fixed a vulnerability in ZEO storage servers when
blobs are available. Someone with write access to a ZEO server
configured to support blobs could read any file on the system
readable by the server process and remove any file removable by the
server process.
- Fixed ``NameError`` in cases where a directory cannot be created,
e.g. when the necessary permissions are missing.
- Fixed a pack test that was not compatible with storages that always
return an object count of 0.
- Calling __setstate__ on a persistent object could under certain
uncommon cause the process to crash.
Whats new in ZODB 3.8.2
=======================
Bugs Fixed:
- Fixed vulnerabilities in the ZEO network protocol that allow:
- CVE-2009-0668 Arbitrary Python code execution in ZODB ZEO storage servers
- CVE-2009-0669 Authentication bypass in ZODB ZEO storage servers
The vulnerabilities only apply if you are using ZEO to share a
database among multiple applications or application instances and if
untrusted clients are able to connect to your ZEO servers.
- Limit the number of object ids that can be allocated at once to
avoid running out of memory.
Whats new in ZODB 3.8.1
=======================
Bugs Fixed:
- (3.8.1) Reset ``_cache`` on a connection's ``_reader`` object when
resetting the cache, to prevent reads from the old cache object, e.g.
during Zope2's auto-refresh of products.
(https://bugs.launchpad.net/zodb/+bug/142667).
- (3.8.1) An exception would be raised when an error occured attempting to
lock a file and logging of said error was enabled.
- (3.8.1) Fixed a bug to allow opening of deep-copied blobs.
- (3.8.1) Fixed bug #189542 by prepending the module to an undefined name.
- (3.8.1) Failures in tpc_finish of client-storages weren't handled
correctly, leaving the client storage in an inconsistent state.
- (3.8.1) If there is a failure while FileStorage is finalizing a transaction,
the file storage is closed because it's internal meta data may be
invalid.
- (3.8.1) FileStorages previously saved indexes after a certain
number of writes. This was done during the last phase of two-phase
commit, which made this critical phase more subject to errors than
it should have been. Also, for large databases, saves were done so
infrequently as to be useless. The feature was removed to reduce
the chance for errors during the last phase of two-phase commit.
- (3.8.1) File storages previously kept an internal object id to
transaction id mapping as an optimization. This mapping caused
excessive memory usage and failures during the last phase of
two-phase commit. This optimization has been removed.
- (3.8.1) Fixed a bug that caused deep copying of blobs to fail.
- (3.8.1) Refactored handling of invalidations on ZEO clients to fix
a possible ordering problem for invalidation messages.
- (3.8.1) An ZEO cache internal data structure can get out of sync
with the data in a way that prevents data from being loaded into the
cache. We don't yet know why, but added an exception handler to
prevent this error from being fatal.
- (3.8.1) Fixed setup.py use of setuptools vs distutils, so .c and .h
files are included in the bdist_egg.
- (3.8.1) On many systems, it was impossible to create more than 32K
blobs. Added a new blob-directory layout to work around this
limitation.
- (3.8.1) Fixed a bug, introduced in an earlier beta, that allowed
clients to connect to out of date servers.
- (3.8.1) Fixed bug that could lead to memory errors due to the use
of a Python dictionary for a mapping that can grow large.
- (3.8.1) Fixed bug #251037: Made packing of blob storages non-blocking.
- (3.8.1) Fixed a bug that could cause InvalidObjectReference errors
for objects that were explicitly added to a database if the object
was modified after a savepoint that added the object.
- (3.8.1) Fixed several bugs that caused ZEO cache corruption when connecting
to servers. These bugs affected both persistent and non-persistent caches.
- (3.8.1) Improved the the ZEO client shutdown support to try to
avoid spurious errors on exit, especially for scripts, such as zeopack.
- (3.8.1) Packing failed for databases containing cross-database references.
- (3.8.1) Cross-database references to databases with empty names
weren't constructed properly.
- (3.8.1) The cache used an excessive amount of memory, causing applications
with large caches to exhaust available memory.
- Fixed a number of bugs in the handling of persistent ZEO caches:
- Cache records are written in several steps. If a process exits
after writing begins and before it is finishes, the cache will be
corrupt on restart. The way records are writted was changed to
make cache record updates atomic.
- There was no lock file to prevent opening a cache multiple times
at once, which would lead to corruption. Persistent caches now
use lock files, in the same way that file storages do.
- A bug in the cache-opening logic led to cache failure in the
unlikely event that a cache has no free blocks.
- When using ZEO Client Storages, Errors occured when trying to store
objects too big to fit in the ZEO cache file.
- Fixed bug in blob filesystem helper: the `isSecure` check was inversed.
- Fixed bug in transaction buffer: a tuple was unpacked incorrectly in
`clear`.
- Fixed bug in Connection.TmpStore: load() would not defer to the
backend storage for loading blobs.
- Fixed bug #190884: Wrong reference to `POSKeyError` caused NameError.
- Completed implementation of ZEO authentication. This fixes issue 220856.
What's new on ZODB 3.8.0
========================
General
-------
- The ZODB Storage APIs have been documented and cleaned up.
- ZODB versions are now officially deprecated and support for them
will be removed in ZODB 3.9. (They have been widely recognized as
deprecated for quite a while.)
- Changed the automatic garbage collection when opening a connection to only
apply the garbage collections on those connections in the pool that are
closed. (This fixed issue 113923.)
- (3.8.0b3) Document conflict resolution (see ZODB/ConflictResolution.txt).
- (3.8.0b3) Bugfix the situation in which comparing persistent objects (for
instance, as members in BTree set or keys of BTree) might cause data
inconsistency during conflict resolution.
- (3.8.0b3) Support multidatabase references in conflict resolution.
- (3.8.0b3) Make it possible to examine oid and (in some situations) database
name of persistent object references during conflict resolution.
- (3.8.0b3) Added missing data attribute for conflict errors.
- (3.8.0b5) Fixed bug 153316: persistent and BTrees gave errors on x86_64
Intel XEON platforms.
ZEO
---
- (3.8.0b6) Bug #98275: Made ZEO cache more tolerant when invalidating current
versions of objects.
- (3.8.0b4, 3.8.0b5) Fixed a serious bug that could cause client I/O to stop
(hang). This was accomonied by a critical log message along the
lines of: "RuntimeError: dictionary changed size during iteration".
(In b4, the bug was only partially fixed.)
- (3.8a1) ZEO's strategoes for avoiding client cache verification were
improved in the case that servers are restarted. Before, if
transactions were committed after the restart, clients that were up
to date or nearly up to date at the time of the restart and then
connected had to verify their caches. Now, it is far more likely
that a client that reconnects soon after a server restart won't have
to verify its cache.
- (3.8a1) Fixed a serious bug that could cause clients that disconnect from and
reconnect to a server to get bad invalidation data if the server
serves multiple storages with active writes.
- (3.8a1) It is now theoretically possible to use a ClientStorage in a storage
server. This might make it possible to offload read load from a
storage server at the cost of increasing write latency. This should
increase write throughput by offloading reads from the final storage
server. This feature is somewhat experimental. It has tests, but
hasn't been used in production.
Transactions
------------
- (3.8a1) Add a doom() and isDoomed() interface to the transaction module.
First step towards the resolution of
http://www.zope.org/Collectors/Zope3-dev/655
A doomed transaction behaves exactly the same way as an active transaction
but raises an error on any attempt to commit it, thus forcing an abort.
Doom is useful in places where abort is unsafe and an exception cannot be
raised. This occurs when the programmer wants the code following the doom to
run but not commit. It is unsafe to abort in these circumstances as a
following get() may implicitly open a new transaction.
Any attempt to commit a doomed transaction will raise a DoomedTransaction
exception.
- (3.8a1) Clean up the ZODB imports in transaction.
Clean up weird import dance with ZODB. This is unnecessary since the
transaction module stopped being imported in ZODB/__init__.py in rev 39622.
- (3.8a1) Support for subtransactions has been removed in favor of
save points.
Blobs
-----
- (3.8b5) Fixed bug #130459: Packing was broken by uncommitted blob data.
- (3.8b4) Fixed bug #127182: Blobs were subclassable which was not desired.
- (3.8b3) Fixed bug #126007: tpc_abort had untested code path that was
broken.
- (3.8b3) Fixed bug #129921: getSize() function in BlobStorage could not
deal with garbage files
- (3.8b1) Updated the Blob implementation in a number of ways. Some
of these are backward incompatible with 3.8a1:
o The Blob class now lives in ZODB.blob
o The blob openDetached method has been replaced by the committed method.
- (3.8a1) Added new blob feature. See the ZODB/Blobs directory for
documentation.
ZODB now handles (reasonably) large binary objects efficiently. Useful to
use from a few kilobytes to at least multiple hundred megabytes.
BTrees
------
- (3.8a1) Added support for 64-bit integer BTrees as separate types.
(For now, we're retaining compile-time support for making the regular
integer BTrees 64-bit.)
- (3.8a1) Normalize names in modules so that BTrees, Buckets, Sets, and
TreeSets can all be accessed with those names in the modules (e.g.,
BTrees.IOBTree.BTree). This is in addition to the older names (e.g.,
BTrees.IOBTree.IOBTree). This allows easier drop-in replacement, which
can especially be simplify code for packages that want to support both
32-bit and 64-bit BTrees.
- (3.8a1) Describe the interfaces for each module and actually declare
the interfaces for each.
- (3.8a1) Fix module references so klass.__module__ points to the Python
wrapper module, not the C extension.
- (3.8a1) introduce module families, to group all 32-bit and all 64-bit
modules.