Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Z
zodb
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Joshua
zodb
Commits
61aeabc6
Commit
61aeabc6
authored
May 18, 2007
by
Jim Fulton
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
consumeFile no-longer works with open files.
Now it also truly consumes files. Binary mode is implicit.
parent
c6679cc2
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
44 additions
and
57 deletions
+44
-57
src/ZODB/Blobs/tests/consume.txt
src/ZODB/Blobs/tests/consume.txt
+22
-37
src/ZODB/Blobs/tests/test_undo.py
src/ZODB/Blobs/tests/test_undo.py
+22
-20
No files found.
src/ZODB/Blobs/tests/consume.txt
View file @
61aeabc6
...
@@ -6,20 +6,28 @@ an O(1) operation we call `consume`::
...
@@ -6,20 +6,28 @@ an O(1) operation we call `consume`::
Let's create a file::
Let's create a file::
>>> import tempfile
>>> to_import = open('to_import', 'wb')
>>> to_import = tempfile.NamedTemporaryFile()
>>> to_import.write("I'm a Blob and I feel fine.")
>>> to_import.write("I'm a Blob and I feel fine.")
>>> to_import.flush()
The file *must* be closed before giving it to consumeFile:
>>> to_import.close()
Now, let's consume this file in a blob by specifying it's name::
Now, let's consume this file in a blob by specifying it's name::
>>> from ZODB.Blobs.Blob import Blob
>>> from ZODB.Blobs.Blob import Blob
>>> blob = Blob()
>>> blob = Blob()
>>> blob.consumeFile(to_import.name)
>>> blob.consumeFile('to_import')
After the consumeFile operation, the original file has been removed:
>>> import os
>>> os.path.exists('to_import')
False
We now can call open on the blob and read and write the data::
We now can call open on the blob and read and write the data::
>>> blob_read = blob.open('r
b
')
>>> blob_read = blob.open('r')
>>> blob_read.read()
>>> blob_read.read()
"I'm a Blob and I feel fine."
"I'm a Blob and I feel fine."
>>> blob_read.close()
>>> blob_read.close()
...
@@ -27,41 +35,24 @@ We now can call open on the blob and read and write the data::
...
@@ -27,41 +35,24 @@ We now can call open on the blob and read and write the data::
>>> blob_write.write('I was changed.')
>>> blob_write.write('I was changed.')
>>> blob_write.close()
>>> blob_write.close()
Please note that the interface for the `consume` method specifies a hard-link
as a part of the contract so your existing file and the blob file will be the
same. If one gets changed the other will reflect those changes as well. This
is especially a known side-effect when consuming a file and then opening the
blob for writing before committing in between::
>>> to_import.seek(0)
>>> to_import.read()
'I was changed.'
(Applications are expected that files for consumption are typically copies of
existing data and that the imported link to the file will be removed after a
successfull import. This can be achieved (as in this test) by using a
NamedTempFile.)
We can not consume a file when there is a reader or writer around for a blob
We can not consume a file when there is a reader or writer around for a blob
already::
already::
>>> to_import2 = tempfile.NamedTemporaryFile()
>>> open('to_import', 'wb').write('I am another blob.')
>>> to_import2.write('I am another blob.')
>>> to_import2.flush()
>>> blob_read = blob.open('r')
>>> blob_read = blob.open('r')
>>> blob.consumeFile(
to_import2.name
)
>>> blob.consumeFile(
'to_import'
)
Traceback (most recent call last):
Traceback (most recent call last):
BlobError: Already opened for reading.
BlobError: Already opened for reading.
>>> blob_read.close()
>>> blob_read.close()
>>> blob_write = blob.open('w')
>>> blob_write = blob.open('w')
>>> blob.consumeFile(
to_import2.name
)
>>> blob.consumeFile(
'to_import'
)
Traceback (most recent call last):
Traceback (most recent call last):
BlobError: Already opened for writing.
BlobError: Already opened for writing.
>>> blob_write.close()
>>> blob_write.close()
Now, after closing all readers and writers we can consume files again::
Now, after closing all readers and writers we can consume files again::
>>> blob.consumeFile(
to_import2.name
)
>>> blob.consumeFile(
'to_import'
)
>>> blob_read = blob.open('r')
>>> blob_read = blob.open('r')
>>> blob_read.read()
>>> blob_read.read()
'I am another blob.'
'I am another blob.'
...
@@ -70,14 +61,13 @@ Now, after closing all readers and writers we can consume files again::
...
@@ -70,14 +61,13 @@ Now, after closing all readers and writers we can consume files again::
Edge cases
Edge cases
==========
==========
There are some edge cases what happens when the link() operation fails. We simulate this in different states:
There are some edge cases what happens when the link() operation
fails. We simulate this in different states:
Case 1: We don't have uncommitted data, but the link operation fails. The
Case 1: We don't have uncommitted data, but the link operation fails. The
exception will be re-raised and the target file will not exist::
exception will be re-raised and the target file will not exist::
>>> input = tempfile.NamedTemporaryFile()
>>> open('to_import', 'wb').write('Some data.')
>>> input.write('Some data')
>>> input.flush()
>>> def failing_link(self, filename):
>>> def failing_link(self, filename):
... raise Exception("I can't link.")
... raise Exception("I can't link.")
...
@@ -88,7 +78,7 @@ exception will be re-raised and the target file will not exist::
...
@@ -88,7 +78,7 @@ exception will be re-raised and the target file will not exist::
BlobError: Blob does not exist.
BlobError: Blob does not exist.
>>> blob._os_link = failing_link
>>> blob._os_link = failing_link
>>> blob.consumeFile(
input.name
)
>>> blob.consumeFile(
'to_import'
)
Traceback (most recent call last):
Traceback (most recent call last):
Exception: I can't link.
Exception: I can't link.
...
@@ -102,18 +92,13 @@ Case 2: We thave uncommitted data, but the link operation fails. The
...
@@ -102,18 +92,13 @@ Case 2: We thave uncommitted data, but the link operation fails. The
exception will be re-raised and the target file will exist with the previous
exception will be re-raised and the target file will exist with the previous
uncomitted data::
uncomitted data::
>>> input = tempfile.NamedTemporaryFile()
>>> input.write('Unimported data')
>>> input.flush()
>>> blob = Blob()
>>> blob = Blob()
>>> blob_writing = blob.open('w')
>>> blob_writing = blob.open('w')
>>> blob_writing.write('Uncommitted data')
>>> blob_writing.write('Uncommitted data')
>>> blob_writing.close()
>>> blob_writing.close()
>>> blob._os_link = failing_link
>>> blob._os_link = failing_link
>>> blob.consumeFile(
input.name
)
>>> blob.consumeFile(
'to_import'
)
Traceback (most recent call last):
Traceback (most recent call last):
Exception: I can't link.
Exception: I can't link.
...
...
src/ZODB/Blobs/tests/test_undo.py
View file @
61aeabc6
...
@@ -28,15 +28,16 @@ from ZODB import utils
...
@@ -28,15 +28,16 @@ from ZODB import utils
class
BlobUndoTests
(
unittest
.
TestCase
):
class
BlobUndoTests
(
unittest
.
TestCase
):
def
setUp
(
self
):
def
setUp
(
self
):
self
.
storagefile
=
tempfile
.
mktemp
()
self
.
test_dir
=
tempfile
.
mkdtemp
()
self
.
blob_dir
=
tempfile
.
mkdtemp
()
self
.
here
=
os
.
getcwd
()
os
.
chdir
(
self
.
test_dir
)
self
.
storagefile
=
'Data.fs'
os
.
mkdir
(
'blobs'
)
self
.
blob_dir
=
'blobs'
def
tearDown
(
self
):
def
tearDown
(
self
):
try
:
os
.
chdir
(
self
.
here
)
os
.
unlink
(
self
.
storagefile
)
shutil
.
rmtree
(
self
.
test_dir
)
except
(
OSError
,
IOError
):
pass
shutil
.
rmtree
(
self
.
blob_dir
)
def
testUndoWithoutPreviousVersion
(
self
):
def
testUndoWithoutPreviousVersion
(
self
):
base_storage
=
FileStorage
(
self
.
storagefile
)
base_storage
=
FileStorage
(
self
.
storagefile
)
...
@@ -60,7 +61,8 @@ class BlobUndoTests(unittest.TestCase):
...
@@ -60,7 +61,8 @@ class BlobUndoTests(unittest.TestCase):
root
=
connection
.
root
()
root
=
connection
.
root
()
# the blob footprint object should exist no longer
# the blob footprint object should exist no longer
self
.
assertRaises
(
KeyError
,
root
.
__getitem__
,
'blob'
)
self
.
assertRaises
(
KeyError
,
root
.
__getitem__
,
'blob'
)
database
.
close
()
def
testUndo
(
self
):
def
testUndo
(
self
):
base_storage
=
FileStorage
(
self
.
storagefile
)
base_storage
=
FileStorage
(
self
.
storagefile
)
blob_storage
=
BlobStorage
(
self
.
blob_dir
,
base_storage
)
blob_storage
=
BlobStorage
(
self
.
blob_dir
,
base_storage
)
...
@@ -93,6 +95,7 @@ class BlobUndoTests(unittest.TestCase):
...
@@ -93,6 +95,7 @@ class BlobUndoTests(unittest.TestCase):
blob
=
root
[
'blob'
]
blob
=
root
[
'blob'
]
self
.
assertEqual
(
blob
.
open
(
'r'
).
read
(),
'this is state 1'
)
self
.
assertEqual
(
blob
.
open
(
'r'
).
read
(),
'this is state 1'
)
transaction
.
abort
()
transaction
.
abort
()
database
.
close
()
def
testUndoAfterConsumption
(
self
):
def
testUndoAfterConsumption
(
self
):
base_storage
=
FileStorage
(
self
.
storagefile
)
base_storage
=
FileStorage
(
self
.
storagefile
)
...
@@ -101,22 +104,16 @@ class BlobUndoTests(unittest.TestCase):
...
@@ -101,22 +104,16 @@ class BlobUndoTests(unittest.TestCase):
connection
=
database
.
open
()
connection
=
database
.
open
()
root
=
connection
.
root
()
root
=
connection
.
root
()
transaction
.
begin
()
transaction
.
begin
()
to_consume
=
tempfile
.
NamedTemporaryFile
()
open
(
'consume1'
,
'w'
).
write
(
'this is state 1'
)
to_consume
.
write
(
'this is state 1'
)
to_consume
.
flush
()
blob
=
Blob
()
blob
=
Blob
()
blob
.
consumeFile
(
to_consume
.
name
)
blob
.
consumeFile
(
'consume1'
)
root
[
'blob'
]
=
blob
root
[
'blob'
]
=
blob
transaction
.
commit
()
transaction
.
commit
()
transaction
.
begin
()
transaction
.
begin
()
blob
=
root
[
'blob'
]
blob
=
root
[
'blob'
]
to_consume
=
tempfile
.
NamedTemporaryFile
()
open
(
'consume2'
,
'w'
).
write
(
'this is state 2'
)
to_consume
.
write
(
'this is state 2'
)
blob
.
consumeFile
(
'consume2'
)
to_consume
.
flush
()
blob
.
consumeFile
(
to_consume
.
name
)
transaction
.
commit
()
transaction
.
commit
()
transaction
.
begin
()
transaction
.
begin
()
...
@@ -135,6 +132,8 @@ class BlobUndoTests(unittest.TestCase):
...
@@ -135,6 +132,8 @@ class BlobUndoTests(unittest.TestCase):
self
.
assertEqual
(
blob
.
open
(
'r'
).
read
(),
'this is state 1'
)
self
.
assertEqual
(
blob
.
open
(
'r'
).
read
(),
'this is state 1'
)
transaction
.
abort
()
transaction
.
abort
()
database
.
close
()
def
testRedo
(
self
):
def
testRedo
(
self
):
base_storage
=
FileStorage
(
self
.
storagefile
)
base_storage
=
FileStorage
(
self
.
storagefile
)
blob_storage
=
BlobStorage
(
self
.
blob_dir
,
base_storage
)
blob_storage
=
BlobStorage
(
self
.
blob_dir
,
base_storage
)
...
@@ -174,7 +173,9 @@ class BlobUndoTests(unittest.TestCase):
...
@@ -174,7 +173,9 @@ class BlobUndoTests(unittest.TestCase):
blob
=
root
[
'blob'
]
blob
=
root
[
'blob'
]
self
.
assertEqual
(
blob
.
open
(
'r'
).
read
(),
'this is state 2'
)
self
.
assertEqual
(
blob
.
open
(
'r'
).
read
(),
'this is state 2'
)
transaction
.
abort
()
transaction
.
abort
()
database
.
close
()
def
testRedoOfCreation
(
self
):
def
testRedoOfCreation
(
self
):
base_storage
=
FileStorage
(
self
.
storagefile
)
base_storage
=
FileStorage
(
self
.
storagefile
)
blob_storage
=
BlobStorage
(
self
.
blob_dir
,
base_storage
)
blob_storage
=
BlobStorage
(
self
.
blob_dir
,
base_storage
)
...
@@ -207,6 +208,7 @@ class BlobUndoTests(unittest.TestCase):
...
@@ -207,6 +208,7 @@ class BlobUndoTests(unittest.TestCase):
self
.
assertEqual
(
blob
.
open
(
'r'
).
read
(),
'this is state 1'
)
self
.
assertEqual
(
blob
.
open
(
'r'
).
read
(),
'this is state 1'
)
transaction
.
abort
()
transaction
.
abort
()
database
.
close
()
def
test_suite
():
def
test_suite
():
suite
=
unittest
.
TestSuite
()
suite
=
unittest
.
TestSuite
()
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment