Commit 643e267f authored by Jim Fulton's avatar Jim Fulton

Made a number of blob changes:

- Unwritten blobs can now be read, and are empty.

- Blobs are considered modified when opened for writing.  This is a
   little bit more conservative than before but fixes a bug that a file
   opened with 'w' actually does modify the file and wasn't considered
   to be a change before.

- Optimistic savepoints now work.

- Fixed bug: could open multiple files for writing.

- Fixed bug: aborting a transaction removed uncommitted data for
   uncommitted blobs.

Todo: 
   Need to remove uncommitted data file if a blob is GCed even when a
   transaction isn't aborted or when it hasn't been added to anything.

- No-longer close files on transaction boundaries.

This allows us to get rid of the transaction-manager dance.
parent 3ccbf36d
This diff is collapsed.
...@@ -26,12 +26,10 @@ A blob implements the IBlob interface: ...@@ -26,12 +26,10 @@ A blob implements the IBlob interface:
>>> IBlob.providedBy(myblob) >>> IBlob.providedBy(myblob)
True True
Opening a new Blob for reading fails: We can open a new blob file for reading, but it won't have any data:
>>> myblob.open("r") >>> myblob.open("r").read()
Traceback (most recent call last): ''
...
BlobError: Blob does not exist.
But we can write data to a new Blob by opening it for writing: But we can write data to a new Blob by opening it for writing:
......
...@@ -35,21 +35,53 @@ Putting a Blob into a Connection works like any other Persistent object:: ...@@ -35,21 +35,53 @@ Putting a Blob into a Connection works like any other Persistent object::
>>> blob1 = Blob() >>> blob1 = Blob()
>>> blob1.open('w').write('this is blob 1') >>> blob1.open('w').write('this is blob 1')
>>> root1['blob1'] = blob1 >>> root1['blob1'] = blob1
>>> transaction.commit() >>> 'blob1' in root1
True
Aborting a blob add leaves the blob unchanged:
Aborting a transaction involving a blob write cleans up uncommitted >>> transaction.abort()
file data:: >>> 'blob1' in root1
False
>>> dead_blob = Blob() >>> blob1._p_oid
>>> dead_blob.open('w').write('this is a dead blob') >>> blob1._p_jar
>>> root1['dead_blob'] = dead_blob >>> blob1.open().read()
>>> fname = dead_blob._p_blob_uncommitted 'this is blob 1'
It doesn't clear the file because there is no previously committed version:
>>> fname = blob1._p_blob_uncommitted
>>> import os >>> import os
>>> os.path.exists(fname) >>> os.path.exists(fname)
True True
Let's put the blob back into the root and commit the change:
>>> root1['blob1'] = blob1
>>> transaction.commit()
Now, if we make a change and abort it, we'll return to the committed
state:
>>> os.path.exists(fname)
False
>>> blob1._p_blob_uncommitted
>>> blob1.open('w').write('this is new blob 1')
>>> blob1.open().read()
'this is new blob 1'
>>> fname = blob1._p_blob_uncommitted
>>> os.path.exists(fname)
True
>>> transaction.abort() >>> transaction.abort()
>>> os.path.exists(fname) >>> os.path.exists(fname)
False False
>>> blob1._p_blob_uncommitted
>>> blob1.open().read()
'this is blob 1'
Opening a blob gives us a filehandle. Getting data out of the Opening a blob gives us a filehandle. Getting data out of the
resulting filehandle is accomplished via the filehandle's read method:: resulting filehandle is accomplished via the filehandle's read method::
...@@ -57,66 +89,44 @@ resulting filehandle is accomplished via the filehandle's read method:: ...@@ -57,66 +89,44 @@ resulting filehandle is accomplished via the filehandle's read method::
>>> connection2 = database.open() >>> connection2 = database.open()
>>> root2 = connection2.root() >>> root2 = connection2.root()
>>> blob1a = root2['blob1'] >>> blob1a = root2['blob1']
>>> blob1a._p_blob_refcounts()
(0, 0)
>>>
>>> blob1afh1 = blob1a.open("r") >>> blob1afh1 = blob1a.open("r")
>>> blob1afh1.read() >>> blob1afh1.read()
'this is blob 1' 'this is blob 1'
>>> # The filehandle keeps a reference to its blob object
>>> blob1afh1.blob._p_blob_refcounts()
(1, 0)
Let's make another filehandle for read only to blob1a, this should bump Let's make another filehandle for read only to blob1a. Aach file
up its refcount by one, and each file handle has a reference to the handle has a reference to the (same) underlying blob::
(same) underlying blob::
>>> blob1afh2 = blob1a.open("r") >>> blob1afh2 = blob1a.open("r")
>>> blob1afh2.blob._p_blob_refcounts()
(2, 0)
>>> blob1afh1.blob._p_blob_refcounts()
(2, 0)
>>> blob1afh2.blob is blob1afh1.blob >>> blob1afh2.blob is blob1afh1.blob
True True
Let's close the first filehandle we got from the blob, this should decrease Let's close the first filehandle we got from the blob::
its refcount by one::
>>> blob1afh1.close() >>> blob1afh1.close()
>>> blob1a._p_blob_refcounts()
(1, 0)
Let's abort this transaction, and ensure that the filehandles that we Let's abort this transaction, and ensure that the filehandles that we
opened are now closed and that the filehandle refcounts on the blob opened are still open::
object are cleared::
>>> transaction.abort() >>> transaction.abort()
>>> blob1afh1.blob._p_blob_refcounts()
(0, 0)
>>> blob1afh2.blob._p_blob_refcounts()
(0, 0)
>>> blob1a._p_blob_refcounts()
(0, 0)
>>> blob1afh2.read() >>> blob1afh2.read()
Traceback (most recent call last): 'this is blob 1'
...
ValueError: I/O operation on closed file
If we open a blob for append, its write refcount should be nonzero. >>> blob1afh2.close()
Additionally, writing any number of bytes to the blobfile should
result in the blob being marked "dirty" in the connection (we just If we open a blob for append, writing any number of bytes to the
aborted above, so the object should be "clean" when we start):: blobfile should result in the blob being marked "dirty" in the
connection (we just aborted above, so the object should be "clean"
when we start)::
>>> bool(blob1a._p_changed) >>> bool(blob1a._p_changed)
False False
>>> blob1a.open('r').read() >>> blob1a.open('r').read()
'this is blob 1' 'this is blob 1'
>>> blob1afh3 = blob1a.open('a') >>> blob1afh3 = blob1a.open('a')
>>> blob1afh3.write('woot!')
>>> blob1a._p_blob_refcounts()
(0, 1)
>>> bool(blob1a._p_changed) >>> bool(blob1a._p_changed)
True True
>>> blob1afh3.write('woot!')
We can open more than one blob object during the course of a single We can open more than one blob object during the course of a single
transaction:: transaction::
...@@ -125,10 +135,6 @@ transaction:: ...@@ -125,10 +135,6 @@ transaction::
>>> blob2.open('w').write('this is blob 3') >>> blob2.open('w').write('this is blob 3')
>>> root2['blob2'] = blob2 >>> root2['blob2'] = blob2
>>> transaction.commit() >>> transaction.commit()
>>> blob2._p_blob_refcounts()
(0, 0)
>>> blob1._p_blob_refcounts()
(0, 0)
Since we committed the current transaction above, the aggregate Since we committed the current transaction above, the aggregate
changes we've made to blob, blob1a (these refer to the same object) and changes we've made to blob, blob1a (these refer to the same object) and
...@@ -200,7 +206,7 @@ int on 64-bit):: ...@@ -200,7 +206,7 @@ int on 64-bit)::
Savepoints and Blobs Savepoints and Blobs
-------------------- --------------------
We do support optimistic savepoints :: We do support optimistic savepoints:
>>> connection5 = database.open() >>> connection5 = database.open()
>>> root5 = connection5.root() >>> root5 = connection5.root()
...@@ -222,17 +228,16 @@ We do support optimistic savepoints :: ...@@ -222,17 +228,16 @@ We do support optimistic savepoints ::
"I'm a happy blob. And I'm singing." "I'm a happy blob. And I'm singing."
>>> transaction.get().commit() >>> transaction.get().commit()
We do not support non-optimistic savepoints:: We support optimistic savepoints too:
>>> blob_fh = root5['blob'].open("a") >>> root5['blob'].open("a").write(" And I'm dancing.")
>>> blob_fh.write(" And the weather is beautiful.")
>>> blob_fh.close()
>>> root5['blob'].open("r").read() >>> root5['blob'].open("r").read()
"I'm a happy blob. And I'm singing. And the weather is beautiful." "I'm a happy blob. And I'm singing. And I'm dancing."
>>> savepoint = transaction.savepoint() # doctest: +ELLIPSIS >>> savepoint = transaction.savepoint()
Traceback (most recent call last): >>> root5['blob'].open("w").write(" And the weather is beautiful.")
... >>> savepoint.rollback()
TypeError: ('Savepoints unsupported', <ZODB.blob.BlobDataManager instance at 0x...>) >>> root5['blob'].open("r").read()
"I'm a happy blob. And I'm singing. And I'm dancing."
>>> transaction.abort() >>> transaction.abort()
Reading Blobs outside of a transaction Reading Blobs outside of a transaction
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment