Skip to content

T
transaction
Commit 3805b636 authored by Tres Seaver's avatar Tres Seaver
Browse files
Options

Merge 'sphinx' branch.

parents 7165a6f9 c0ba11cc
Expand all Show whitespace changes
Inline Side-by-side
Showing with 3823 additions and 2407 deletions
CHANGES.txt
View file @ 3805b636
......@@ -4,6 +4,20 @@ Changes
1.3.1 (unreleased)
------------------
- Refactored existing doctests as Sphinx documentation (snippets are exercised
via 'tox').
- 100% unit test coverage.
- Raise ValueError from ``Transaction.doom`` if the transaction is in a
non-doomable state (rather than using ``assert``).
- Raise ValueError from ``TransactionManager.attempts`` if passed a
non-positive value (rather than using ``assert``).
- Raise ValueError from ``TransactionManager.free`` if passed a foreign
transaction (rather tna using ``assert``).
- Declared support for Python 3.3 in ``setup.py``, and added ``tox`` testing.
- When a non-retryable exception was raised as the result of a call to
......
transaction/tests/convenience.txt → docs/convenience.rst
View file @ 3805b636
......@@ -10,6 +10,8 @@ with support
We can now use the with statement to define transaction boundaries.
.. doctest::
>>> import transaction.tests.savepointsample
>>> dm = transaction.tests.savepointsample.SampleSavepointDataManager()
>>> list(dm.keys())
......@@ -17,6 +19,8 @@ We can now use the with statement to define transaction boundaries.
We can use it with a manager:
.. doctest::
>>> with transaction.manager as t:
... dm['z'] = 3
... t.note('test 3')
......@@ -46,7 +50,9 @@ Retries
Commits can fail for transient reasons, especially conflicts.
Applications will often retry transactions some number of times to
overcome transient failures. This typically looks something like::
overcome transient failures. This typically looks something like:
.. doctest::
for i in range(3):
try:
......@@ -62,6 +68,7 @@ This is rather ugly.
Transaction managers provide a helper for this case. To show this,
we'll use a contrived example:
.. doctest::
>>> ntry = 0
>>> with transaction.manager:
......@@ -96,6 +103,8 @@ a transaction object will be assigned to the variable named.
By default, it tries 3 times. We can tell it how many times to try:
.. doctest::
>>> for attempt in transaction.manager.attempts(2):
... with attempt:
... ntry += 1
......@@ -110,6 +119,8 @@ propagated.
Of course, other errors are propagated directly:
.. doctest::
>>> ntry = 0
>>> for attempt in transaction.manager.attempts():
... with attempt:
......@@ -122,6 +133,8 @@ Of course, other errors are propagated directly:
We can use the default transaction manager:
.. doctest::
>>> for attempt in transaction.attempts():
... with attempt as t:
... t.note('test')
......@@ -140,6 +153,8 @@ control. Data managers can provide a should_retry method that takes
an exception instance and returns True if the transaction should be
attempted again.
.. doctest::
>>> class DM(transaction.tests.savepointsample.SampleSavepointDataManager):
... def should_retry(self, e):
... if 'should retry' in str(e):
......
docs/datamanager.rst 0 → 100644
View file @ 3805b636
Writing a Data Manager
======================
Simple Data Manager
------------------
.. doctest::
>>> from transaction.tests.examples import DataManager
This :class:`transaction.tests.examples.DataManager` class
provides a trivial data-manager implementation and docstrings to illustrate
the the protocol and to provide a tool for writing tests.
Our sample data manager has state that is updated through an inc
method and through transaction operations.
When we create a sample data manager:
.. doctest::
>>> dm = DataManager()
It has two bits of state, state:
.. doctest::
>>> dm.state
0
and delta:
.. doctest::
>>> dm.delta
0
Both of which are initialized to 0. state is meant to model
committed state, while delta represents tentative changes within a
transaction. We change the state by calling inc:
.. doctest::
>>> dm.inc()
which updates delta:
.. doctest::
>>> dm.delta
1
but state isn't changed until we commit the transaction:
.. doctest::
>>> dm.state
0
To commit the changes, we use 2-phase commit. We execute the first
stage by calling prepare. We need to pass a transation. Our
sample data managers don't really use the transactions for much,
so we'll be lazy and use strings for transactions:
.. doctest::
>>> t1 = '1'
>>> dm.prepare(t1)
The sample data manager updates the state when we call prepare:
.. doctest::
>>> dm.state
1
>>> dm.delta
1
This is mainly so we can detect some affect of calling the methods.
Now if we call commit:
.. doctest::
>>> dm.commit(t1)
Our changes are"permanent". The state reflects the changes and the
delta has been reset to 0.
.. doctest::
>>> dm.state
1
>>> dm.delta
0
The :meth:`prepare` Method
----------------------------
Prepare to commit data
.. doctest::
>>> dm = DataManager()
>>> dm.inc()
>>> t1 = '1'
>>> dm.prepare(t1)
>>> dm.commit(t1)
>>> dm.state
1
>>> dm.inc()
>>> t2 = '2'
>>> dm.prepare(t2)
>>> dm.abort(t2)
>>> dm.state
1
It is en error to call prepare more than once without an intervening
commit or abort:
.. doctest::
>>> dm.prepare(t1)
>>> dm.prepare(t1)
Traceback (most recent call last):
...
TypeError: Already prepared
>>> dm.prepare(t2)
Traceback (most recent call last):
...
TypeError: Already prepared
>>> dm.abort(t1)
If there was a preceeding savepoint, the transaction must match:
.. doctest::
>>> rollback = dm.savepoint(t1)
>>> dm.prepare(t2)
Traceback (most recent call last):
,,,
TypeError: ('Transaction missmatch', '2', '1')
>>> dm.prepare(t1)
The :meth:`abort` method
--------------------------
The abort method can be called before two-phase commit to
throw away work done in the transaction:
.. doctest::
>>> dm = DataManager()
>>> dm.inc()
>>> dm.state, dm.delta
(0, 1)
>>> t1 = '1'
>>> dm.abort(t1)
>>> dm.state, dm.delta
(0, 0)
The abort method also throws away work done in savepoints:
.. doctest::
>>> dm.inc()
>>> r = dm.savepoint(t1)
>>> dm.inc()
>>> r = dm.savepoint(t1)
>>> dm.state, dm.delta
(0, 2)
>>> dm.abort(t1)
>>> dm.state, dm.delta
(0, 0)
If savepoints are used, abort must be passed the same
transaction:
.. doctest::
>>> dm.inc()
>>> r = dm.savepoint(t1)
>>> t2 = '2'
>>> dm.abort(t2)
Traceback (most recent call last):
...
TypeError: ('Transaction missmatch', '2', '1')
>>> dm.abort(t1)
The abort method is also used to abort a two-phase commit:
.. doctest::
>>> dm.inc()
>>> dm.state, dm.delta
(0, 1)
>>> dm.prepare(t1)
>>> dm.state, dm.delta
(1, 1)
>>> dm.abort(t1)
>>> dm.state, dm.delta
(0, 0)
Of course, the transactions passed to prepare and abort must
match:
.. doctest::
>>> dm.prepare(t1)
>>> dm.abort(t2)
Traceback (most recent call last):
...
TypeError: ('Transaction missmatch', '2', '1')
>>> dm.abort(t1)
The :meth:`commit` method
---------------------------
Called to omplete two-phase commit
.. doctest::
>>> dm = DataManager()
>>> dm.state
0
>>> dm.inc()
We start two-phase commit by calling prepare:
.. doctest::
>>> t1 = '1'
>>> dm.prepare(t1)
We complete it by calling commit:
.. doctest::
>>> dm.commit(t1)
>>> dm.state
1
It is an error ro call commit without calling prepare first:
.. doctest::
>>> dm.inc()
>>> t2 = '2'
>>> dm.commit(t2)
Traceback (most recent call last):
...
TypeError: Not prepared to commit
>>> dm.prepare(t2)
>>> dm.commit(t2)
If course, the transactions given to prepare and commit must
be the same:
.. doctest::
>>> dm.inc()
>>> t3 = '3'
>>> dm.prepare(t3)
>>> dm.commit(t2)
Traceback (most recent call last):
...
TypeError: ('Transaction missmatch', '2', '3')
The :meth:`savepoint` method
------------------------------
Provide the ability to rollback transaction state
Savepoints provide a way to:
- Save partial transaction work. For some data managers, this
could allow resources to be used more efficiently.
- Provide the ability to revert state to a point in a
transaction without aborting the entire transaction. In
other words, savepoints support partial aborts.
Savepoints don't use two-phase commit. If there are errors in
setting or rolling back to savepoints, the application should
abort the containing transaction. This is *not* the
responsibility of the data manager.
Savepoints are always associated with a transaction. Any work
done in a savepoint's transaction is tentative until the
transaction is committed using two-phase commit.
.. doctest::
>>> dm = DataManager()
>>> dm.inc()
>>> t1 = '1'
>>> r = dm.savepoint(t1)
>>> dm.state, dm.delta
(0, 1)
>>> dm.inc()
>>> dm.state, dm.delta
(0, 2)
>>> r.rollback()
>>> dm.state, dm.delta
(0, 1)
>>> dm.prepare(t1)
>>> dm.commit(t1)
>>> dm.state, dm.delta
(1, 0)
Savepoints must have the same transaction:
.. doctest::
>>> r1 = dm.savepoint(t1)
>>> dm.state, dm.delta
(1, 0)
>>> dm.inc()
>>> dm.state, dm.delta
(1, 1)
>>> t2 = '2'
>>> r2 = dm.savepoint(t2)
Traceback (most recent call last):
...
TypeError: ('Transaction missmatch', '2', '1')
>>> r2 = dm.savepoint(t1)
>>> dm.inc()
>>> dm.state, dm.delta
(1, 2)
If we rollback to an earlier savepoint, we discard all work
done later:
.. doctest::
>>> r1.rollback()
>>> dm.state, dm.delta
(1, 0)
and we can no longer rollback to the later savepoint:
.. doctest::
>>> r2.rollback()
Traceback (most recent call last):
...
TypeError: ('Attempt to roll back to invalid save point', 3, 2)
We can roll back to a savepoint as often as we like:
.. doctest::
>>> r1.rollback()
>>> r1.rollback()
>>> r1.rollback()
>>> dm.state, dm.delta
(1, 0)
>>> dm.inc()
>>> dm.inc()
>>> dm.inc()
>>> dm.state, dm.delta
(1, 3)
>>> r1.rollback()
>>> dm.state, dm.delta
(1, 0)
But we can't rollback to a savepoint after it has been
committed:
.. doctest::
>>> dm.prepare(t1)
>>> dm.commit(t1)
>>> r1.rollback()
Traceback (most recent call last):
...
TypeError: Attempt to rollback stale rollback
transaction/tests/doom.txt → docs/doom.rst
View file @ 3805b636
......@@ -25,6 +25,8 @@ use savepoints and doom() safely.
To see how it works we first need to create a stub data manager:
.. doctest::
>>> from transaction.interfaces import IDataManager
>>> from zope.interface import implementer
>>> @implementer(IDataManager)
......@@ -45,6 +47,8 @@ To see how it works we first need to create a stub data manager:
Start a new transaction:
.. doctest::
>>> import transaction
>>> txn = transaction.begin()
>>> dm = DataManager()
......@@ -56,27 +60,37 @@ sends all outstanding SQL to a relational database for objects changed during
the transaction. This expensive operation is not necessary if the transaction
has been doomed. A non-doomed transaction should return False:
.. doctest::
>>> txn.isDoomed()
False
We can doom a transaction by calling .doom() on it:
.. doctest::
>>> txn.doom()
>>> txn.isDoomed()
True
We can doom it again if we like:
.. doctest::
>>> txn.doom()
The data manager is unchanged at this point:
.. doctest::
>>> dm.total()
0
Attempting to commit a doomed transaction any number of times raises a
DoomedTransaction:
.. doctest::
>>> txn.commit() # doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
DoomedTransaction: transaction doomed, cannot commit
......@@ -86,15 +100,21 @@ DoomedTransaction:
But still leaves the data manager unchanged:
.. doctest::
>>> dm.total()
0
But the doomed transaction can be aborted:
.. doctest::
>>> txn.abort()
Which aborts the data manager:
.. doctest::
>>> dm.total()
1
>>> dm.attr_counter['abort']
......@@ -103,6 +123,8 @@ Which aborts the data manager:
Dooming the current transaction can also be done directly from the transaction
module. We can also begin a new transaction directly after dooming the old one:
.. doctest::
>>> txn = transaction.begin()
>>> transaction.isDoomed()
False
......@@ -115,16 +137,20 @@ After committing a transaction we get an assertion error if we try to doom the
transaction. This could be made more specific, but trying to doom a transaction
after it's been committed is probably a programming error:
.. doctest::
>>> txn = transaction.begin()
>>> txn.commit()
>>> txn.doom()
Traceback (most recent call last):
...
AssertionError
ValueError: non-doomable
A doomed transaction should act the same as an active transaction, so we should
be able to join it:
.. doctest::
>>> txn = transaction.begin()
>>> txn.doom()
>>> dm2 = DataManager()
......@@ -132,5 +158,7 @@ be able to join it:
Clean up:
.. doctest::
>>> txn = transaction.begin()
>>> txn.abort()
docs/hooks.rst 0 → 100644
View file @ 3805b636
Hooking the Transaction Machinery
=================================
The :meth:`addBeforeCommitHook` Method
--------------------------------------
Let's define a hook to call, and a way to see that it was called.
.. doctest::
>>> log = []
>>> def reset_log():
... del log[:]
>>> def hook(arg='no_arg', kw1='no_kw1', kw2='no_kw2'):
... log.append("arg %r kw1 %r kw2 %r" % (arg, kw1, kw2))
Now register the hook with a transaction.
.. doctest::
>>> from transaction import begin
>>> from transaction._compat import func_name
>>> import transaction
>>> t = begin()
>>> t.addBeforeCommitHook(hook, '1')
We can see that the hook is indeed registered.
.. doctest::
>>> [(func_name(hook), args, kws)
... for hook, args, kws in t.getBeforeCommitHooks()]
[('hook', ('1',), {})]
When transaction commit starts, the hook is called, with its
arguments.
.. doctest::
>>> log
[]
>>> t.commit()
>>> log
["arg '1' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()
A hook's registration is consumed whenever the hook is called. Since
the hook above was called, it's no longer registered:
.. doctest::
>>> from transaction import commit
>>> len(list(t.getBeforeCommitHooks()))
0
>>> commit()
>>> log
[]
The hook is only called for a full commit, not for a savepoint.
.. doctest::
>>> t = begin()
>>> t.addBeforeCommitHook(hook, 'A', dict(kw1='B'))
>>> dummy = t.savepoint()
>>> log
[]
>>> t.commit()
>>> log
["arg 'A' kw1 'B' kw2 'no_kw2'"]
>>> reset_log()
If a transaction is aborted, no hook is called.
.. doctest::
>>> from transaction import abort
>>> t = begin()
>>> t.addBeforeCommitHook(hook, ["OOPS!"])
>>> abort()
>>> log
[]
>>> commit()
>>> log
[]
The hook is called before the commit does anything, so even if the
commit fails the hook will have been called. To provoke failures in
commit, we'll add failing resource manager to the transaction.
.. doctest::
>>> class CommitFailure(Exception):
... pass
>>> class FailingDataManager:
... def tpc_begin(self, txn, sub=False):
... raise CommitFailure('failed')
... def abort(self, txn):
... pass
>>> t = begin()
>>> t.join(FailingDataManager())
>>> t.addBeforeCommitHook(hook, '2')
>>> from transaction.tests.common import DummyFile
>>> from transaction.tests.common import Monkey
>>> from transaction.tests.common import assertRaisesEx
>>> from transaction import _transaction
>>> buffer = DummyFile()
>>> with Monkey(_transaction, _TB_BUFFER=buffer):
... err = assertRaisesEx(CommitFailure, t.commit)
>>> log
["arg '2' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()
Let's register several hooks.
.. doctest::
>>> t = begin()
>>> t.addBeforeCommitHook(hook, '4', dict(kw1='4.1'))
>>> t.addBeforeCommitHook(hook, '5', dict(kw2='5.2'))
They are returned in the same order by getBeforeCommitHooks.
.. doctest::
>>> [(func_name(hook), args, kws) #doctest: +NORMALIZE_WHITESPACE
... for hook, args, kws in t.getBeforeCommitHooks()]
[('hook', ('4',), {'kw1': '4.1'}),
('hook', ('5',), {'kw2': '5.2'})]
And commit also calls them in this order.
.. doctest::
>>> t.commit()
>>> len(log)
2
>>> log #doctest: +NORMALIZE_WHITESPACE
["arg '4' kw1 '4.1' kw2 'no_kw2'",
"arg '5' kw1 'no_kw1' kw2 '5.2'"]
>>> reset_log()
While executing, a hook can itself add more hooks, and they will all
be called before the real commit starts.
.. doctest::
>>> def recurse(txn, arg):
... log.append('rec' + str(arg))
... if arg:
... txn.addBeforeCommitHook(hook, '-')
... txn.addBeforeCommitHook(recurse, (txn, arg-1))
>>> t = begin()
>>> t.addBeforeCommitHook(recurse, (t, 3))
>>> commit()
>>> log #doctest: +NORMALIZE_WHITESPACE
['rec3',
"arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec2',
"arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec1',
"arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec0']
>>> reset_log()
The :meth:`addAfterCommitHook` Method
--------------------------------------
Let's define a hook to call, and a way to see that it was called.
.. doctest::
>>> log = []
>>> def reset_log():
... del log[:]
>>> def hook(status, arg='no_arg', kw1='no_kw1', kw2='no_kw2'):
... log.append("%r arg %r kw1 %r kw2 %r" % (status, arg, kw1, kw2))
Now register the hook with a transaction.
.. doctest::
>>> from transaction import begin
>>> from transaction._compat import func_name
>>> t = begin()
>>> t.addAfterCommitHook(hook, '1')
We can see that the hook is indeed registered.
.. doctest::
>>> [(func_name(hook), args, kws)
... for hook, args, kws in t.getAfterCommitHooks()]
[('hook', ('1',), {})]
When transaction commit is done, the hook is called, with its
arguments.
.. doctest::
>>> log
[]
>>> t.commit()
>>> log
["True arg '1' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()
A hook's registration is consumed whenever the hook is called. Since
the hook above was called, it's no longer registered:
.. doctest::
>>> from transaction import commit
>>> len(list(t.getAfterCommitHooks()))
0
>>> commit()
>>> log
[]
The hook is only called after a full commit, not for a savepoint.
.. doctest::
>>> t = begin()
>>> t.addAfterCommitHook(hook, 'A', dict(kw1='B'))
>>> dummy = t.savepoint()
>>> log
[]
>>> t.commit()
>>> log
["True arg 'A' kw1 'B' kw2 'no_kw2'"]
>>> reset_log()
If a transaction is aborted, no hook is called.
.. doctest::
>>> from transaction import abort
>>> t = begin()
>>> t.addAfterCommitHook(hook, ["OOPS!"])
>>> abort()
>>> log
[]
>>> commit()
>>> log
[]
The hook is called after the commit is done, so even if the
commit fails the hook will have been called. To provoke failures in
commit, we'll add failing resource manager to the transaction.
.. doctest::
>>> class CommitFailure(Exception):
... pass
>>> class FailingDataManager:
... def tpc_begin(self, txn):
... raise CommitFailure('failed')
... def abort(self, txn):
... pass
>>> t = begin()
>>> t.join(FailingDataManager())
>>> t.addAfterCommitHook(hook, '2')
>>> from transaction.tests.common import DummyFile
>>> from transaction.tests.common import Monkey
>>> from transaction.tests.common import assertRaisesEx
>>> from transaction import _transaction
>>> buffer = DummyFile()
>>> with Monkey(_transaction, _TB_BUFFER=buffer):
... err = assertRaisesEx(CommitFailure, t.commit)
>>> log
["False arg '2' kw1 'no_kw1' kw2 'no_kw2'"]
>>> reset_log()
Let's register several hooks.
.. doctest::
>>> t = begin()
>>> t.addAfterCommitHook(hook, '4', dict(kw1='4.1'))
>>> t.addAfterCommitHook(hook, '5', dict(kw2='5.2'))
They are returned in the same order by getAfterCommitHooks.
.. doctest::
>>> [(func_name(hook), args, kws) #doctest: +NORMALIZE_WHITESPACE
... for hook, args, kws in t.getAfterCommitHooks()]
[('hook', ('4',), {'kw1': '4.1'}),
('hook', ('5',), {'kw2': '5.2'})]
And commit also calls them in this order.
.. doctest::
>>> t.commit()
>>> len(log)
2
>>> log #doctest: +NORMALIZE_WHITESPACE
["True arg '4' kw1 '4.1' kw2 'no_kw2'",
"True arg '5' kw1 'no_kw1' kw2 '5.2'"]
>>> reset_log()
While executing, a hook can itself add more hooks, and they will all
be called before the real commit starts.
.. doctest::
>>> def recurse(status, txn, arg):
... log.append('rec' + str(arg))
... if arg:
... txn.addAfterCommitHook(hook, '-')
... txn.addAfterCommitHook(recurse, (txn, arg-1))
>>> t = begin()
>>> t.addAfterCommitHook(recurse, (t, 3))
>>> commit()
>>> log #doctest: +NORMALIZE_WHITESPACE
['rec3',
"True arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec2',
"True arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec1',
"True arg '-' kw1 'no_kw1' kw2 'no_kw2'",
'rec0']
>>> reset_log()
If an after commit hook is raising an exception then it will log a
message at error level so that if other hooks are registered they
can be executed. We don't support execution dependencies at this level.
.. doctest::
>>> from transaction import TransactionManager
>>> from transaction.tests.test__manager import DataObject
>>> mgr = TransactionManager()
>>> do = DataObject(mgr)
>>> def hookRaise(status, arg='no_arg', kw1='no_kw1', kw2='no_kw2'):
... raise TypeError("Fake raise")
>>> t = begin()
>>> t.addAfterCommitHook(hook, ('-', 1))
>>> t.addAfterCommitHook(hookRaise, ('-', 2))
>>> t.addAfterCommitHook(hook, ('-', 3))
>>> commit()
>>> log
["True arg '-' kw1 1 kw2 'no_kw2'", "True arg '-' kw1 3 kw2 'no_kw2'"]
>>> reset_log()
Test that the associated transaction manager has been cleanup when
after commit hooks are registered
.. doctest::
>>> mgr = TransactionManager()
>>> do = DataObject(mgr)
>>> t = begin()
>>> t._manager._txn is not None
True
>>> t.addAfterCommitHook(hook, ('-', 1))
>>> commit()
>>> log
["True arg '-' kw1 1 kw2 'no_kw2'"]
>>> t._manager._txn is not None
False
>>> reset_log()
docs/index.rst
View file @ 3805b636
:mod:`transaction` Documentation
================================
Transaction objects manage resources for an individual activity.
Compatibility issues
--------------------
The implementation of Transaction objects involves two layers of
backwards compatibility, because this version of transaction supports
both ZODB 3 and ZODB 4. Zope is evolving towards the ZODB4
interfaces.
Transaction has two methods for a resource manager to call to
participate in a transaction -- register() and join(). join() takes a
resource manager and adds it to the list of resources. register() is
for backwards compatibility. It takes a persistent object and
registers its _p_jar attribute. TODO: explain adapter
Two-phase commit
----------------
A transaction commit involves an interaction between the transaction
object and one or more resource managers. The transaction manager
calls the following four methods on each resource manager; it calls
tpc_begin() on each resource manager before calling commit() on any of
them.
1. tpc_begin(txn)
2. commit(txn)
3. tpc_vote(txn)
4. tpc_finish(txn)
Before-commit hook
------------------
Sometimes, applications want to execute some code when a transaction is
committed. For example, one might want to delay object indexing until a
transaction commits, rather than indexing every time an object is changed.
Or someone might want to check invariants only after a set of operations. A
pre-commit hook is available for such use cases: use addBeforeCommitHook(),
passing it a callable and arguments. The callable will be called with its
arguments at the start of the commit (but not for substransaction commits).
After-commit hook
------------------
Sometimes, applications want to execute code after a transaction commit
attempt succeeds or aborts. For example, one might want to launch non
transactional code after a successful commit. Or still someone might
want to launch asynchronous code after. A post-commit hook is
available for such use cases: use addAfterCommitHook(), passing it a
callable and arguments. The callable will be called with a Boolean
value representing the status of the commit operation as first
argument (true if successfull or false iff aborted) preceding its
arguments at the start of the commit (but not for substransaction
commits). Commit hooks are not called for transaction.abort().
Error handling
--------------
When errors occur during two-phase commit, the transaction manager
aborts all the resource managers. The specific methods it calls
depend on whether the error occurs before or after the call to
tpc_vote() on that transaction manager.
If the resource manager has not voted, then the resource manager will
have one or more uncommitted objects. There are two cases that lead
to this state; either the transaction manager has not called commit()
for any objects on this resource manager or the call that failed was a
commit() for one of the objects of this resource manager. For each
uncommitted object, including the object that failed in its commit(),
call abort().
Once uncommitted objects are aborted, tpc_abort() or abort_sub() is
called on each resource manager.
Synchronization
---------------
You can register sychronization objects (synchronizers) with the
tranasction manager. The synchronizer must implement
beforeCompletion() and afterCompletion() methods. The transaction
manager calls beforeCompletion() when it starts a top-level two-phase
commit. It calls afterCompletion() when a top-level transaction is
committed or aborted. The methods are passed the current Transaction
as their only argument.
Contents:
.. toctree::
:maxdepth: 2
convenience
doom
savepoint
hooks
datamanager
resourcemanager
api
......
docs/resourcemanager.rst 0 → 100644
View file @ 3805b636
Writing a Resource Manager
==========================
Simple Resource Manager
-----------------------
.. doctest::
>>> from transaction.tests.examples import ResourceManager
This :class:`transaction.tests.examples.ResourceManager`
class provides a trivial resource-manager implementation and doc
strings to illustrate the protocol and to provide a tool for writing
tests.
Our sample resource manager has state that is updated through an inc
method and through transaction operations.
When we create a sample resource manager:
.. doctest::
>>> rm = ResourceManager()
It has two pieces state, state and delta, both initialized to 0:
.. doctest::
>>> rm.state
0
>>> rm.delta
0
state is meant to model committed state, while delta represents
tentative changes within a transaction. We change the state by
calling inc:
.. doctest::
>>> rm.inc()
which updates delta:
.. doctest::
>>> rm.delta
1
but state isn't changed until we commit the transaction:
.. doctest::
>>> rm.state
0
To commit the changes, we use 2-phase commit. We execute the first
stage by calling prepare. We need to pass a transation. Our
sample resource managers don't really use the transactions for much,
so we'll be lazy and use strings for transactions. The sample
resource manager updates the state when we call tpc_vote:
.. doctest::
>>> t1 = '1'
>>> rm.tpc_begin(t1)
>>> rm.state, rm.delta
(0, 1)
>>> rm.tpc_vote(t1)
>>> rm.state, rm.delta
(1, 1)
Now if we call tpc_finish:
>>> rm.tpc_finish(t1)
Our changes are "permanent". The state reflects the changes and the
delta has been reset to 0.
.. doctest::
>>> rm.state, rm.delta
(1, 0)
The :meth:`tpc_begin` Method
-----------------------------
Called by the transaction manager to ask the RM to prepare to commit data.
.. doctest::
>>> rm = ResourceManager()
>>> rm.inc()
>>> t1 = '1'
>>> rm.tpc_begin(t1)
>>> rm.tpc_vote(t1)
>>> rm.tpc_finish(t1)
>>> rm.state
1
>>> rm.inc()
>>> t2 = '2'
>>> rm.tpc_begin(t2)
>>> rm.tpc_vote(t2)
>>> rm.tpc_abort(t2)
>>> rm.state
1
It is an error to call tpc_begin more than once without completing
two-phase commit:
.. doctest::
>>> rm.tpc_begin(t1)
>>> rm.tpc_begin(t1)
Traceback (most recent call last):
...
ValueError: txn in state 'tpc_begin' but expected one of (None,)
>>> rm.tpc_abort(t1)
If there was a preceeding savepoint, the transaction must match:
.. doctest::
>>> rollback = rm.savepoint(t1)
>>> rm.tpc_begin(t2)
Traceback (most recent call last):
,,,
TypeError: ('Transaction missmatch', '2', '1')
>>> rm.tpc_begin(t1)
The :meth:`tpc_vote` Method
---------------------------
Verify that a data manager can commit the transaction.
This is the last chance for a data manager to vote 'no'. A
data manager votes 'no' by raising an exception.
Passed `transaction`, which is the ITransaction instance associated with the
transaction being committed.
The :meth:`tpc_finish` Method
-----------------------------
Complete two-phase commit
.. doctest::
>>> rm = ResourceManager()
>>> rm.state
0
>>> rm.inc()
We start two-phase commit by calling prepare:
>>> t1 = '1'
>>> rm.tpc_begin(t1)
>>> rm.tpc_vote(t1)
We complete it by calling tpc_finish:
>>> rm.tpc_finish(t1)
>>> rm.state
1
It is an error ro call tpc_finish without calling tpc_vote:
.. doctest::
>>> rm.inc()
>>> t2 = '2'
>>> rm.tpc_begin(t2)
>>> rm.tpc_finish(t2)
Traceback (most recent call last):
...
ValueError: txn in state 'tpc_begin' but expected one of ('tpc_vote',)
>>> rm.tpc_abort(t2) # clean slate
>>> rm.tpc_begin(t2)
>>> rm.tpc_vote(t2)
>>> rm.tpc_finish(t2)
Of course, the transactions given to tpc_begin and tpc_finish must
be the same:
.. doctest::
>>> rm.inc()
>>> t3 = '3'
>>> rm.tpc_begin(t3)
>>> rm.tpc_vote(t3)
>>> rm.tpc_finish(t2)
Traceback (most recent call last):
...
TypeError: ('Transaction missmatch', '2', '3')
The :meth:`tpc_abort` Method
-----------------------------
Abort a transaction
The abort method can be called before two-phase commit to
throw away work done in the transaction:
.. doctest::
>>> rm = ResourceManager()
>>> rm.inc()
>>> rm.state, rm.delta
(0, 1)
>>> t1 = '1'
>>> rm.tpc_abort(t1)
>>> rm.state, rm.delta
(0, 0)
The abort method also throws away work done in savepoints:
.. doctest::
>>> rm.inc()
>>> r = rm.savepoint(t1)
>>> rm.inc()
>>> r = rm.savepoint(t1)
>>> rm.state, rm.delta
(0, 2)
>>> rm.tpc_abort(t1)
>>> rm.state, rm.delta
(0, 0)
If savepoints are used, abort must be passed the same
transaction:
.. doctest::
>>> rm.inc()
>>> r = rm.savepoint(t1)
>>> t2 = '2'
>>> rm.tpc_abort(t2)
Traceback (most recent call last):
...
TypeError: ('Transaction missmatch', '2', '1')
>>> rm.tpc_abort(t1)
The abort method is also used to abort a two-phase commit:
.. doctest::
>>> rm.inc()
>>> rm.state, rm.delta
(0, 1)
>>> rm.tpc_begin(t1)
>>> rm.state, rm.delta
(0, 1)
>>> rm.tpc_vote(t1)
>>> rm.state, rm.delta
(1, 1)
>>> rm.tpc_abort(t1)
>>> rm.state, rm.delta
(0, 0)
Of course, the transactions passed to prepare and abort must
match:
.. doctest::
>>> rm.tpc_begin(t1)
>>> rm.tpc_abort(t2)
Traceback (most recent call last):
...
TypeError: ('Transaction missmatch', '2', '1')
>>> rm.tpc_abort(t1)
This should never fail.
The :meth:`savepoint` Method
----------------------------
Provide the ability to rollback transaction state
Savepoints provide a way to:
- Save partial transaction work. For some resource managers, this
could allow resources to be used more efficiently.
- Provide the ability to revert state to a point in a
transaction without aborting the entire transaction. In
other words, savepoints support partial aborts.
Savepoints don't use two-phase commit. If there are errors in
setting or rolling back to savepoints, the application should
abort the containing transaction. This is *not* the
responsibility of the resource manager.
Savepoints are always associated with a transaction. Any work
done in a savepoint's transaction is tentative until the
transaction is committed using two-phase commit.
.. doctest::
>>> rm = ResourceManager()
>>> rm.inc()
>>> t1 = '1'
>>> r = rm.savepoint(t1)
>>> rm.state, rm.delta
(0, 1)
>>> rm.inc()
>>> rm.state, rm.delta
(0, 2)
>>> r.rollback()
>>> rm.state, rm.delta
(0, 1)
>>> rm.tpc_begin(t1)
>>> rm.tpc_vote(t1)
>>> rm.tpc_finish(t1)
>>> rm.state, rm.delta
(1, 0)
Savepoints must have the same transaction:
.. doctest::
>>> r1 = rm.savepoint(t1)
>>> rm.state, rm.delta
(1, 0)
>>> rm.inc()
>>> rm.state, rm.delta
(1, 1)
>>> t2 = '2'
>>> r2 = rm.savepoint(t2)
Traceback (most recent call last):
...
TypeError: ('Transaction missmatch', '2', '1')
>>> r2 = rm.savepoint(t1)
>>> rm.inc()
>>> rm.state, rm.delta
(1, 2)
If we rollback to an earlier savepoint, we discard all work
done later:
.. doctest::
>>> r1.rollback()
>>> rm.state, rm.delta
(1, 0)
and we can no longer rollback to the later savepoint:
.. doctest::
>>> r2.rollback()
Traceback (most recent call last):
...
TypeError: ('Attempt to roll back to invalid save point', 3, 2)
We can roll back to a savepoint as often as we like:
.. doctest::
>>> r1.rollback()
>>> r1.rollback()
>>> r1.rollback()
>>> rm.state, rm.delta
(1, 0)
>>> rm.inc()
>>> rm.inc()
>>> rm.inc()
>>> rm.state, rm.delta
(1, 3)
>>> r1.rollback()
>>> rm.state, rm.delta
(1, 0)
But we can't rollback to a savepoint after it has been
committed:
.. doctest::
>>> rm.tpc_begin(t1)
>>> rm.tpc_vote(t1)
>>> rm.tpc_finish(t1)
>>> r1.rollback()
Traceback (most recent call last):
...
TypeError: Attempt to rollback stale rollback
transaction/tests/savepoint.txt → docs/savepoint.rst
View file @ 3805b636
......@@ -24,6 +24,8 @@ demonstrating the correct operation of savepoint support within the
transaction system. This data manager is very simple. It provides flat
storage of named immutable values, like strings and numbers.
.. doctest::
>>> import transaction
>>> from transaction.tests import savepointsample
>>> dm = savepointsample.SampleSavepointDataManager()
......@@ -31,12 +33,16 @@ storage of named immutable values, like strings and numbers.
As with other data managers, we can commit changes:
.. doctest::
>>> transaction.commit()
>>> dm['name']
'bob'
and abort changes:
.. doctest::
>>> dm['name'] = 'sally'
>>> dm['name']
'sally'
......@@ -52,6 +58,8 @@ account is invalid, we roll back the change for that entry. The success or
failure of an entry is indicated in the output status. First we'll initialize
some accounts:
.. doctest::
>>> dm['bob-balance'] = 0.0
>>> dm['bob-credit'] = 0.0
>>> dm['sally-balance'] = 0.0
......@@ -60,6 +68,8 @@ some accounts:
Now, we'll define a validation function to validate an account:
.. doctest::
>>> def validate_account(name):
... if dm[name+'-balance'] + dm[name+'-credit'] < 0:
... raise ValueError('Overdrawn', name)
......@@ -67,6 +77,8 @@ Now, we'll define a validation function to validate an account:
And a function to apply entries. If the function fails in some unexpected
way, it rolls back all of its changes and prints the error:
.. doctest::
>>> def apply_entries(entries):
... savepoint = transaction.savepoint()
... try:
......@@ -86,6 +98,8 @@ way, it rolls back all of its changes and prints the error:
Now let's try applying some entries:
.. doctest::
>>> apply_entries([
... ('bob', 10.0),
... ('sally', 10.0),
......@@ -109,6 +123,8 @@ Now let's try applying some entries:
If we provide entries that cause an unexpected error:
.. doctest::
>>> apply_entries([
... ('bob', 10.0),
... ('sally', 10.0),
......@@ -123,6 +139,8 @@ Because the apply_entries used a savepoint for the entire function, it was
able to rollback the partial changes without rolling back changes made in the
previous call to ``apply_entries``:
.. doctest::
>>> dm['bob-balance']
30.0
......@@ -132,6 +150,8 @@ previous call to ``apply_entries``:
If we now abort the outer transactions, the earlier changes will go
away:
.. doctest::
>>> transaction.abort()
>>> dm['bob-balance']
......@@ -145,6 +165,8 @@ Savepoint invalidation
A savepoint can be used any number of times:
.. doctest::
>>> dm['bob-balance'] = 100.0
>>> dm['bob-balance']
100.0
......@@ -170,6 +192,8 @@ A savepoint can be used any number of times:
However, using a savepoint invalidates any savepoints that come after it:
.. doctest::
>>> dm['bob-balance'] = 200.0
>>> dm['bob-balance']
200.0
......@@ -203,6 +227,8 @@ Databases without savepoint support
Normally it's an error to use savepoints with databases that don't support
savepoints:
.. doctest::
>>> dm_no_sp = savepointsample.SampleDataManager()
>>> dm_no_sp['name'] = 'bob'
>>> transaction.commit()
......@@ -219,6 +245,8 @@ that databases without savepoint support should be tolerated until a savepoint
is rolled back. This allows transactions to proceed if there are no reasons
to roll back:
.. doctest::
>>> dm_no_sp['name'] = 'sally'
>>> savepoint = transaction.savepoint(1)
>>> dm_no_sp['name'] = 'sue'
......@@ -245,6 +273,8 @@ the transaction is aborted.
In the previous example, we got an error when we tried to rollback the
savepoint. If we try to commit the transaction, the commit will fail:
.. doctest::
>>> transaction.commit() #doctest: +IGNORE_EXCEPTION_DETAIL
Traceback (most recent call last):
...
......@@ -255,11 +285,15 @@ savepoint. If we try to commit the transaction, the commit will fail:
We have to abort it to make any progress:
.. doctest::
>>> transaction.abort()
Similarly, in our earlier example, where we tried to take a savepoint with a
data manager that didn't support savepoints:
.. doctest::
>>> dm_no_sp['name'] = 'sally'
>>> dm['name'] = 'sally'
>>> savepoint = transaction.savepoint() # doctest: +IGNORE_EXCEPTION_DETAIL
......@@ -280,6 +314,8 @@ data manager that didn't support savepoints:
After clearing the transaction with an abort, we can get on with new
transactions:
.. doctest::
>>> dm_no_sp['name'] = 'sally'
>>> dm['name'] = 'sally'
>>> transaction.commit()
......
setup.py
View file @ 3805b636
......@@ -60,7 +60,7 @@ setup(name='transaction',
'zope.interface',
],
extras_require = {
'docs': ['Sphinx'],
'docs': ['Sphinx', 'repoze.sphinx.autointerface'],
'testing': ['nose', 'coverage'],
},
entry_points = """\
......
transaction/compat.py → transaction/_compat.py
View file @ 3805b636
......@@ -18,33 +18,28 @@ else:
binary_type = str
long = long
def text_(s, encoding='latin-1', errors='strict'):
if isinstance(s, binary_type):
return s.decode(encoding, errors)
return s # pragma: no cover
def bytes_(s, encoding='latin-1', errors='strict'):
def bytes_(s, encoding='latin-1', errors='strict'): #pragma NO COVER
if isinstance(s, text_type):
return s.encode(encoding, errors)
return s
if PY3: # pragma: no cover
def native_(s, encoding='latin-1', errors='strict'):
def native_(s, encoding='latin-1', errors='strict'): #pragma NO COVER
if isinstance(s, text_type):
return s
return str(s, encoding, errors)
else:
def native_(s, encoding='latin-1', errors='strict'):
def native_(s, encoding='latin-1', errors='strict'): #pragma NO COVER
if isinstance(s, text_type):
return s.encode(encoding, errors)
return str(s)
if PY3:
if PY3: #pragma NO COVER
from io import StringIO
else:
from io import BytesIO as StringIO
if PY3:
if PY3: #pragma NO COVER
from collections import MutableMapping
else:
from UserDict import UserDict as MutableMapping
......@@ -54,13 +49,13 @@ if PY3: # pragma: no cover
exec_ = getattr(builtins, "exec")
def reraise(tp, value, tb=None):
def reraise(tp, value, tb=None): #pragma NO COVER
if value.__traceback__ is not tb:
raise value.with_traceback(tb)
raise value
else: # pragma: no cover
def exec_(code, globs=None, locs=None):
def exec_(code, globs=None, locs=None): #pragma NO COVER
"""Execute code in a namespace."""
if globs is None:
frame = sys._getframe(1)
......@@ -77,7 +72,7 @@ else: # pragma: no cover
""")
if PY3:
if PY3: #pragma NO COVER
try:
from threading import get_ident as get_thread_ident
except ImportError:
......@@ -87,9 +82,9 @@ else:
if PY3:
def func_name(func):
def func_name(func): #pragma NO COVER
return func.__name__
else:
def func_name(func):
def func_name(func): #pragma NO COVER
return func.func_name
transaction/_manager.py
View file @ 3805b636
......@@ -21,11 +21,11 @@ import threading
from zope.interface import implementer
from transaction.weakset import WeakSet
from transaction._transaction import Transaction
from transaction.interfaces import ITransactionManager
from transaction.interfaces import TransientError
from transaction.compat import reraise
from transaction.weakset import WeakSet
from transaction._compat import reraise
from transaction._transaction import Transaction
# We have to remember sets of synch objects, especially Connections.
......@@ -54,6 +54,7 @@ def _new_transaction(txn, synchs):
# so that Transactions "see" synchronizers that get registered after the
# Transaction object is constructed.
@implementer(ITransactionManager)
class TransactionManager(object):
......@@ -80,7 +81,8 @@ class TransactionManager(object):
return self._txn
def free(self, txn):
assert txn is self._txn
if txn is not self._txn:
raise ValueError("Foreign transaction")
self._txn = None
def registerSynch(self, synch):
......@@ -125,7 +127,8 @@ class TransactionManager(object):
return self.get().savepoint(optimistic)
def attempts(self, number=3):
assert number > 0
if number <= 0:
raise ValueError("number must be positive")
while number:
number -= 1
if number:
......@@ -149,6 +152,7 @@ class ThreadTransactionManager(TransactionManager, threading.local):
Each thread is associated with a unique transaction.
"""
class Attempt(object):
def __init__(self, manager):
......@@ -172,4 +176,3 @@ class Attempt(object):
return self._retry_or_raise(*sys.exc_info())
else:
return self._retry_or_raise(t, v, tb)
transaction/_transaction.py
View file @ 3805b636
......@@ -11,92 +11,6 @@
# FOR A PARTICULAR PURPOSE.
#
############################################################################
"""Transaction objects manage resources for an individual activity.
Compatibility issues
--------------------
The implementation of Transaction objects involves two layers of
backwards compatibility, because this version of transaction supports
both ZODB 3 and ZODB 4. Zope is evolving towards the ZODB4
interfaces.
Transaction has two methods for a resource manager to call to
participate in a transaction -- register() and join(). join() takes a
resource manager and adds it to the list of resources. register() is
for backwards compatibility. It takes a persistent object and
registers its _p_jar attribute. TODO: explain adapter
Two-phase commit
----------------
A transaction commit involves an interaction between the transaction
object and one or more resource managers. The transaction manager
calls the following four methods on each resource manager; it calls
tpc_begin() on each resource manager before calling commit() on any of
them.
1. tpc_begin(txn)
2. commit(txn)
3. tpc_vote(txn)
4. tpc_finish(txn)
Before-commit hook
------------------
Sometimes, applications want to execute some code when a transaction is
committed. For example, one might want to delay object indexing until a
transaction commits, rather than indexing every time an object is changed.
Or someone might want to check invariants only after a set of operations. A
pre-commit hook is available for such use cases: use addBeforeCommitHook(),
passing it a callable and arguments. The callable will be called with its
arguments at the start of the commit (but not for substransaction commits).
After-commit hook
------------------
Sometimes, applications want to execute code after a transaction commit
attempt succeeds or aborts. For example, one might want to launch non
transactional code after a successful commit. Or still someone might
want to launch asynchronous code after. A post-commit hook is
available for such use cases: use addAfterCommitHook(), passing it a
callable and arguments. The callable will be called with a Boolean
value representing the status of the commit operation as first
argument (true if successfull or false iff aborted) preceding its
arguments at the start of the commit (but not for substransaction
commits). Commit hooks are not called for transaction.abort().
Error handling
--------------
When errors occur during two-phase commit, the transaction manager
aborts all the resource managers. The specific methods it calls
depend on whether the error occurs before or after the call to
tpc_vote() on that transaction manager.
If the resource manager has not voted, then the resource manager will
have one or more uncommitted objects. There are two cases that lead
to this state; either the transaction manager has not called commit()
for any objects on this resource manager or the call that failed was a
commit() for one of the objects of this resource manager. For each
uncommitted object, including the object that failed in its commit(),
call abort().
Once uncommitted objects are aborted, tpc_abort() or abort_sub() is
called on each resource manager.
Synchronization
---------------
You can register sychronization objects (synchronizers) with the
tranasction manager. The synchronizer must implement
beforeCompletion() and afterCompletion() methods. The transaction
manager calls beforeCompletion() when it starts a top-level two-phase
commit. It calls afterCompletion() when a top-level transaction is
committed or aborted. The methods are passed the current Transaction
as their only argument.
"""
import binascii
import logging
import sys
......@@ -105,17 +19,30 @@ import traceback
from zope.interface import implementer
from transaction.compat import reraise
from transaction.compat import get_thread_ident
from transaction.compat import native_
from transaction.compat import bytes_
from transaction.compat import StringIO
from transaction.weakset import WeakSet
from transaction.interfaces import TransactionFailedError
from transaction import interfaces
from transaction._compat import reraise
from transaction._compat import get_thread_ident
from transaction._compat import native_
from transaction._compat import bytes_
from transaction._compat import StringIO
_marker = object()
_TB_BUFFER = None #unittests may hook
def _makeTracebackBuffer(): #pragma NO COVER
if _TB_BUFFER is not None:
return _TB_BUFFER
return StringIO()
_LOGGER = None #unittests may hook
def _makeLogger(): #pragma NO COVER
if _LOGGER is not None:
return _LOGGER
return logging.getLogger("txn.%d" % get_thread_ident())
# The point of this is to avoid hiding exceptions (which the builtin
# hasattr() does).
def myhasattr(obj, attr):
......@@ -177,7 +104,7 @@ class Transaction(object):
# directly by storages, leading underscore notwithstanding.
self._extension = {}
self.log = logging.getLogger("txn.%d" % get_thread_ident())
self.log = _makeLogger()
self.log.debug("new transaction")
# If a commit fails, the traceback is saved in _failure_traceback.
......@@ -203,7 +130,7 @@ class Transaction(object):
if self.status is not Status.ACTIVE:
# should not doom transactions in the middle,
# or after, a commit
raise AssertionError()
raise ValueError('non-doomable')
self.status = Status.DOOMED
# Raise TransactionFailedError, due to commit()/join()/register()
......@@ -307,7 +234,6 @@ class Transaction(object):
# be stored when the transaction commits. For other
# objects, the object implements the standard two-phase
# commit protocol.
manager = getattr(obj, "_p_jar", obj)
if manager is None:
raise ValueError("Register with no manager")
......@@ -364,7 +290,7 @@ class Transaction(object):
def _saveAndGetCommitishError(self):
self.status = Status.COMMITFAILED
# Save the traceback for TransactionFailedError.
ft = self._failure_traceback = StringIO()
ft = self._failure_traceback = _makeTracebackBuffer()
t = None
v = None
tb = None
......@@ -380,7 +306,6 @@ class Transaction(object):
finally:
del t, v, tb
def _saveAndRaiseCommitishError(self):
t = None
v = None
......@@ -391,7 +316,6 @@ class Transaction(object):
finally:
del t, v, tb
def getBeforeCommitHooks(self):
""" See ITransaction.
"""
......@@ -566,6 +490,7 @@ class Transaction(object):
# TODO: We need a better name for the adapters.
class MultiObjectResourceAdapter(object):
"""Adapt the old-style register() call to the new-style join().
......@@ -573,7 +498,6 @@ class MultiObjectResourceAdapter(object):
the transaction manager. With register(), an individual object
is passed to register().
"""
def __init__(self, jar):
self.manager = jar
self.objects = []
......@@ -624,6 +548,7 @@ class MultiObjectResourceAdapter(object):
finally:
del t, v, tb
def rm_key(rm):
func = getattr(rm, 'sortKey', None)
if func is not None:
......@@ -634,13 +559,14 @@ def object_hint(o):
This function does not raise an exception.
"""
# We should always be able to get __class__.
klass = o.__class__.__name__
# oid would be great, but may this isn't a persistent object.
# oid would be great, but maybe this isn't a persistent object.
oid = getattr(o, "_p_oid", _marker)
if oid is not _marker:
oid = oid_repr(oid)
else:
oid = 'None'
return "%s oid=%s" % (klass, oid)
def oid_repr(oid):
......@@ -657,6 +583,7 @@ def oid_repr(oid):
else:
return repr(oid)
# TODO: deprecate for 3.6.
class DataManagerAdapter(object):
"""Adapt zodb 4-style data managers to zodb3 style
......@@ -700,6 +627,7 @@ class DataManagerAdapter(object):
def sortKey(self):
return self._datamanager.sortKey()
@implementer(interfaces.ISavepoint)
class Savepoint:
"""Transaction savepoint.
......@@ -742,6 +670,7 @@ class Savepoint:
# Mark the transaction as failed.
transaction._saveAndRaiseCommitishError() # reraises!
class AbortSavepoint:
def __init__(self, datamanager, transaction):
......@@ -752,6 +681,7 @@ class AbortSavepoint:
self.datamanager.abort(self.transaction)
self.transaction._unjoin(self.datamanager)
class NoRollbackSavepoint:
def __init__(self, datamanager):
......
transaction/tests/common.py 0 → 100644
View file @ 3805b636
##############################################################################
#
# Copyright (c) 2012 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE
#
##############################################################################
class DummyFile(object):
def __init__(self):
self._lines = []
def write(self, text):
self._lines.append(text)
def writelines(self, lines):
self._lines.extend(lines)
class DummyLogger(object):
def __init__(self):
self._clear()
def _clear(self):
self._log = []
def log(self, level, msg, *args, **kw):
if args:
self._log.append((level, msg % args))
elif kw:
self._log.append((level, msg % kw))
else:
self._log.append((level, msg))
def debug(self, msg, *args, **kw):
self.log('debug', msg, *args, **kw)
def error(self, msg, *args, **kw):
self.log('error', msg, *args, **kw)
def critical(self, msg, *args, **kw):
self.log('critical', msg, *args, **kw)
class Monkey(object):
# context-manager for replacing module names in the scope of a test.
def __init__(self, module, **kw):
self.module = module
self.to_restore = dict([(key, getattr(module, key)) for key in kw])
for key, value in kw.items():
setattr(module, key, value)
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
for key, value in self.to_restore.items():
setattr(self.module, key, value)
def assertRaisesEx(e_type, checked, *args, **kw):
try:
checked(*args, **kw)
except e_type as e:
return e
raise AssertionError("Didn't raise: %s" % e_type.__name__)
transaction/tests/examples.py 0 → 100644
View file @ 3805b636
##############################################################################
#
# Copyright (c) 2004 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Sample objects for use in tests
"""
class DataManager(object):
"""Sample data manager
Used by the 'datamanager' chapter in the Sphinx docs.
"""
def __init__(self):
self.state = 0
self.sp = 0
self.transaction = None
self.delta = 0
self.prepared = False
def inc(self, n=1):
self.delta += n
def prepare(self, transaction):
if self.prepared:
raise TypeError('Already prepared')
self._checkTransaction(transaction)
self.prepared = True
self.transaction = transaction
self.state += self.delta
def _checkTransaction(self, transaction):
if (transaction is not self.transaction
and self.transaction is not None):
raise TypeError("Transaction missmatch",
transaction, self.transaction)
def abort(self, transaction):
self._checkTransaction(transaction)
if self.transaction is not None:
self.transaction = None
if self.prepared:
self.state -= self.delta
self.prepared = False
self.delta = 0
def commit(self, transaction):
if not self.prepared:
raise TypeError('Not prepared to commit')
self._checkTransaction(transaction)
self.delta = 0
self.transaction = None
self.prepared = False
def savepoint(self, transaction):
if self.prepared:
raise TypeError("Can't get savepoint during two-phase commit")
self._checkTransaction(transaction)
self.transaction = transaction
self.sp += 1
return Rollback(self)
class Rollback(object):
def __init__(self, dm):
self.dm = dm
self.sp = dm.sp
self.delta = dm.delta
self.transaction = dm.transaction
def rollback(self):
if self.transaction is not self.dm.transaction:
raise TypeError("Attempt to rollback stale rollback")
if self.dm.sp < self.sp:
raise TypeError("Attempt to roll back to invalid save point",
self.sp, self.dm.sp)
self.dm.sp = self.sp
self.dm.delta = self.delta
class ResourceManager(object):
""" Sample resource manager.
Used by the 'resourcemanager' chapter in the Sphinx docs.
"""
def __init__(self):
self.state = 0
self.sp = 0
self.transaction = None
self.delta = 0
self.txn_state = None
def _check_state(self, *ok_states):
if self.txn_state not in ok_states:
raise ValueError("txn in state %r but expected one of %r" %
(self.txn_state, ok_states))
def _checkTransaction(self, transaction):
if (transaction is not self.transaction
and self.transaction is not None):
raise TypeError("Transaction missmatch",
transaction, self.transaction)
def inc(self, n=1):
self.delta += n
def tpc_begin(self, transaction):
self._checkTransaction(transaction)
self._check_state(None)
self.transaction = transaction
self.txn_state = 'tpc_begin'
def tpc_vote(self, transaction):
self._checkTransaction(transaction)
self._check_state('tpc_begin')
self.state += self.delta
self.txn_state = 'tpc_vote'
def tpc_finish(self, transaction):
self._checkTransaction(transaction)
self._check_state('tpc_vote')
self.delta = 0
self.transaction = None
self.prepared = False
self.txn_state = None
def tpc_abort(self, transaction):
self._checkTransaction(transaction)
if self.transaction is not None:
self.transaction = None
if self.txn_state == 'tpc_vote':
self.state -= self.delta
self.txn_state = None
self.delta = 0
def savepoint(self, transaction):
if self.txn_state is not None:
raise TypeError("Can't get savepoint during two-phase commit")
self._checkTransaction(transaction)
self.transaction = transaction
self.sp += 1
return SavePoint(self)
def discard(self, transaction):
pass
class SavePoint(object):
def __init__(self, rm):
self.rm = rm
self.sp = rm.sp
self.delta = rm.delta
self.transaction = rm.transaction
def rollback(self):
if self.transaction is not self.rm.transaction:
raise TypeError("Attempt to rollback stale rollback")
if self.rm.sp < self.sp:
raise TypeError("Attempt to roll back to invalid save point",
self.sp, self.rm.sp)
self.rm.sp = self.sp
self.rm.delta = self.delta
def discard(self):
pass
transaction/tests/sampledm.py deleted 100644 → 0
View file @ 7165a6f9
This diff is collapsed.
transaction/tests/savepointsample.py
View file @ 3805b636
......@@ -16,7 +16,7 @@
Sample data manager implementation that illustrates how to implement
savepoints.
See savepoint.txt in the transaction package.
Used by savepoint.rst in the Sphinx docs.
"""
from zope.interface import implementer
......
transaction/tests/test_SampleDataManager.py deleted 100644 → 0
View file @ 7165a6f9
This diff is collapsed.
transaction/tests/test_SampleResourceManager.py deleted 100644 → 0
View file @ 7165a6f9
This diff is collapsed.
transaction/tests/test__manager.py 0 → 100644
View file @ 3805b636
This diff is collapsed.
transaction/tests/test__transaction.py 0 → 100644
View file @ 3805b636
This diff is collapsed.
transaction/tests/test_attempt.py deleted 100644 → 0
View file @ 7165a6f9
import unittest
class TestAttempt(unittest.TestCase):
def _makeOne(self, manager):
from transaction._manager import Attempt
return Attempt(manager)
def test___enter__(self):
manager = DummyManager()
inst = self._makeOne(manager)
inst.__enter__()
self.assertTrue(manager.entered)
def test___exit__no_exc_no_commit_exception(self):
manager = DummyManager()
inst = self._makeOne(manager)
result = inst.__exit__(None, None, None)
self.assertFalse(result)
self.assertTrue(manager.committed)
def test___exit__no_exc_nonretryable_commit_exception(self):
manager = DummyManager(raise_on_commit=ValueError)
inst = self._makeOne(manager)
self.assertRaises(ValueError, inst.__exit__, None, None, None)
self.assertTrue(manager.committed)
self.assertTrue(manager.aborted)
def test___exit__no_exc_abort_exception_after_nonretryable_commit_exc(self):
manager = DummyManager(raise_on_abort=ValueError,
raise_on_commit=KeyError)
inst = self._makeOne(manager)
self.assertRaises(ValueError, inst.__exit__, None, None, None)
self.assertTrue(manager.committed)
self.assertTrue(manager.aborted)
def test___exit__no_exc_retryable_commit_exception(self):
from transaction.interfaces import TransientError
manager = DummyManager(raise_on_commit=TransientError)
inst = self._makeOne(manager)
result = inst.__exit__(None, None, None)
self.assertTrue(result)
self.assertTrue(manager.committed)
self.assertTrue(manager.aborted)
def test___exit__with_exception_value_retryable(self):
from transaction.interfaces import TransientError
manager = DummyManager()
inst = self._makeOne(manager)
result = inst.__exit__(TransientError, TransientError(), None)
self.assertTrue(result)
self.assertFalse(manager.committed)
self.assertTrue(manager.aborted)
def test___exit__with_exception_value_nonretryable(self):
manager = DummyManager()
inst = self._makeOne(manager)
self.assertRaises(KeyError, inst.__exit__, KeyError, KeyError(), None)
self.assertFalse(manager.committed)
self.assertTrue(manager.aborted)
class DummyManager(object):
entered = False
committed = False
aborted = False
def __init__(self, raise_on_commit=None, raise_on_abort=None):
self.raise_on_commit = raise_on_commit
self.raise_on_abort = raise_on_abort
def _retryable(self, t, v):
from transaction._manager import TransientError
return issubclass(t, TransientError)
def __enter__(self):
self.entered = True
def abort(self):
self.aborted = True
if self.raise_on_abort:
raise self.raise_on_abort
def commit(self):
self.committed = True
if self.raise_on_commit:
raise self.raise_on_commit
transaction/tests/test_register_compat.py
View file @ 3805b636
......@@ -23,76 +23,63 @@ API works.
These tests use a TestConnection object that implements the old API.
They check that the right methods are called and in roughly the right
order.
Common cases
------------
First, check that a basic transaction commit works.
>>> cn = TestConnection()
>>> cn.register(Object())
>>> cn.register(Object())
>>> cn.register(Object())
>>> transaction.commit()
>>> len(cn.committed)
3
>>> len(cn.aborted)
0
>>> cn.calls
['begin', 'vote', 'finish']
Second, check that a basic transaction abort works. If the
application calls abort(), then the transaction never gets into the
two-phase commit. It just aborts each object.
>>> cn = TestConnection()
>>> cn.register(Object())
>>> cn.register(Object())
>>> cn.register(Object())
>>> transaction.abort()
>>> len(cn.committed)
0
>>> len(cn.aborted)
3
>>> cn.calls
[]
Error handling
--------------
The tricky part of the implementation is recovering from an error that
occurs during the two-phase commit. We override the commit() and
abort() methods of Object to cause errors during commit.
Note that the implementation uses lists internally, so that objects
are committed in the order they are registered. (In the presence of
multiple resource managers, objects from a single resource manager are
committed in order. I'm not sure if this is an accident of the
implementation or a feature that should be supported by any
implementation.)
The order of resource managers depends on sortKey().
>>> cn = TestConnection()
>>> cn.register(Object())
>>> cn.register(CommitError())
>>> cn.register(Object())
>>> transaction.commit()
Traceback (most recent call last):
...
RuntimeError: commit
>>> len(cn.committed)
1
>>> len(cn.aborted)
3
Clean up:
>>> transaction.abort()
"""
import unittest
class BBBTests(unittest.TestCase):
def setUp(self):
from transaction import abort
abort()
tearDown = setUp
def test_basic_commit(self):
import transaction
cn = TestConnection()
cn.register(Object())
cn.register(Object())
cn.register(Object())
transaction.commit()
self.assertEqual(len(cn.committed), 3)
self.assertEqual(len(cn.aborted), 0)
self.assertEqual(cn.calls, ['begin', 'vote', 'finish'])
def test_basic_abort(self):
# If the application calls abort(), then the transaction never gets
# into the two-phase commit. It just aborts each object.
import transaction
cn = TestConnection()
cn.register(Object())
cn.register(Object())
cn.register(Object())
transaction.abort()
self.assertEqual(len(cn.committed), 0)
self.assertEqual(len(cn.aborted), 3)
self.assertEqual(cn.calls, [])
def test_tpc_error(self):
# The tricky part of the implementation is recovering from an error
# that occurs during the two-phase commit. We override the commit()
# and abort() methods of Object to cause errors during commit.
# Note that the implementation uses lists internally, so that objects
# are committed in the order they are registered. (In the presence
# of multiple resource managers, objects from a single resource
# manager are committed in order. I'm not sure if this is an
# accident of the implementation or a feature that should be
# supported by any implementation.)
# The order of resource managers depends on sortKey().
import transaction
cn = TestConnection()
cn.register(Object())
cn.register(CommitError())
cn.register(Object())
self.assertRaises(RuntimeError, transaction.commit)
self.assertEqual(len(cn.committed), 1)
self.assertEqual(len(cn.aborted), 3)
import doctest
import transaction
class Object(object):
......@@ -102,20 +89,24 @@ class Object(object):
def abort(self):
pass
class CommitError(Object):
def commit(self):
raise RuntimeError("commit")
class AbortError(Object):
def abort(self):
raise RuntimeError("abort")
class BothError(CommitError, AbortError):
pass
class TestConnection:
class TestConnection(object):
def __init__(self):
self.committed = []
......@@ -123,6 +114,7 @@ class TestConnection:
self.calls = []
def register(self, obj):
import transaction
obj._p_jar = self
transaction.get().register(obj)
......@@ -150,7 +142,6 @@ class TestConnection:
self.aborted.append(obj)
def test_suite():
return doctest.DocTestSuite()
# additional_tests is for setuptools "setup.py test" support
additional_tests = test_suite
return unittest.TestSuite((
unittest.makeSuite(BBBTests),
))
transaction/tests/test_savepoint.py
View file @ 3805b636
......@@ -11,82 +11,56 @@
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Tests of savepoint feature
"""
import unittest
import doctest
def testRollbackRollsbackDataManagersThatJoinedLater():
"""
A savepoint needs to not just rollback it's savepoints, but needs to
rollback savepoints for data managers that joined savepoints after the
savepoint:
>>> import transaction
>>> from transaction.tests import savepointsample
>>> dm = savepointsample.SampleSavepointDataManager()
>>> dm['name'] = 'bob'
>>> sp1 = transaction.savepoint()
>>> dm['job'] = 'geek'
>>> sp2 = transaction.savepoint()
>>> dm['salary'] = 'fun'
>>> dm2 = savepointsample.SampleSavepointDataManager()
>>> dm2['name'] = 'sally'
>>> 'name' in dm
True
>>> 'job' in dm
True
>>> 'salary' in dm
True
>>> 'name' in dm2
True
>>> sp1.rollback()
>>> 'name' in dm
True
>>> 'job' in dm
False
>>> 'salary' in dm
False
>>> 'name' in dm2
False
"""
def test_commit_after_rollback_for_dm_that_joins_after_savepoint():
"""
There was a problem handling data managers that joined after a
savepoint. If the savepoint was rolled back and then changes made,
the dm would end up being joined twice, leading to extra tpc calls and pain.
>>> import transaction
>>> sp = transaction.savepoint()
>>> from transaction.tests import savepointsample
>>> dm = savepointsample.SampleSavepointDataManager()
>>> dm['name'] = 'bob'
>>> sp.rollback()
>>> dm['name'] = 'Bob'
>>> transaction.commit()
>>> dm['name']
'Bob'
"""
class SavepointTests(unittest.TestCase):
def testRollbackRollsbackDataManagersThatJoinedLater(self):
# A savepoint needs to not just rollback it's savepoints, but needs
# to # rollback savepoints for data managers that joined savepoints
# after the savepoint:
import transaction
from transaction.tests import savepointsample
dm = savepointsample.SampleSavepointDataManager()
dm['name'] = 'bob'
sp1 = transaction.savepoint()
dm['job'] = 'geek'
sp2 = transaction.savepoint()
dm['salary'] = 'fun'
dm2 = savepointsample.SampleSavepointDataManager()
dm2['name'] = 'sally'
self.assertTrue('name' in dm)
self.assertTrue('job' in dm)
self.assertTrue('salary' in dm)
self.assertTrue('name' in dm2)
sp1.rollback()
self.assertTrue('name' in dm)
self.assertFalse('job' in dm)
self.assertFalse('salary' in dm)
self.assertFalse('name' in dm2)
def test_commit_after_rollback_for_dm_that_joins_after_savepoint(self):
# There was a problem handling data managers that joined after a
# savepoint. If the savepoint was rolled back and then changes
# made, the dm would end up being joined twice, leading to extra
# tpc calls and pain.
import transaction
from transaction.tests import savepointsample
sp = transaction.savepoint()
dm = savepointsample.SampleSavepointDataManager()
dm['name'] = 'bob'
sp.rollback()
dm['name'] = 'Bob'
transaction.commit()
self.assertEqual(dm['name'], 'Bob')
def test_suite():
return unittest.TestSuite((
doctest.DocFileSuite('savepoint.txt'),
doctest.DocTestSuite(),
unittest.makeSuite(SavepointTests),
))
# additional_tests is for setuptools "setup.py test" support
additional_tests = test_suite
if __name__ == '__main__':
unittest.main(defaultTest='test_suite')
transaction/tests/test_transaction.py deleted 100644 → 0
View file @ 7165a6f9
This diff is collapsed.
transaction/tests/test_weakset.py
View file @ 3805b636
......@@ -11,15 +11,12 @@
# FOR A PARTICULAR PURPOSE
#
##############################################################################
import unittest
from transaction.weakset import WeakSet
class Dummy:
pass
class WeakSetTests(unittest.TestCase):
def test_contains(self):
from transaction.weakset import WeakSet
w = WeakSet()
dummy = Dummy()
w.add(dummy)
......@@ -29,6 +26,7 @@ class WeakSetTests(unittest.TestCase):
def test_len(self):
import gc
from transaction.weakset import WeakSet
w = WeakSet()
d1 = Dummy()
d2 = Dummy()
......@@ -40,6 +38,7 @@ class WeakSetTests(unittest.TestCase):
self.assertEqual(len(w), 1)
def test_remove(self):
from transaction.weakset import WeakSet
w = WeakSet()
dummy = Dummy()
w.add(dummy)
......@@ -49,6 +48,7 @@ class WeakSetTests(unittest.TestCase):
def test_as_weakref_list(self):
import gc
from transaction.weakset import WeakSet
w = WeakSet()
dummy = Dummy()
dummy2 = Dummy()
......@@ -64,6 +64,7 @@ class WeakSetTests(unittest.TestCase):
self.assertEqual(set(L), set([dummy, dummy2]))
def test_map(self):
from transaction.weakset import WeakSet
w = WeakSet()
dummy = Dummy()
dummy2 = Dummy()
......@@ -78,9 +79,9 @@ class WeakSetTests(unittest.TestCase):
self.assertEqual(thing.poked, 1)
def test_suite():
return unittest.makeSuite(WeakSetTests)
class Dummy:
pass
if __name__ == '__main__':
unittest.main()
def test_suite():
return unittest.makeSuite(WeakSetTests)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7