Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Z
ZODB
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
nexedi
ZODB
Commits
2e27c167
Commit
2e27c167
authored
Sep 08, 2016
by
Jim Fulton
Committed by
GitHub
Sep 08, 2016
Browse files
Options
Browse Files
Download
Plain Diff
Merge pull request #24 from zopefoundation/setup-and-reference
Setup and reference
parents
73248f68
edbf0628
Changes
11
Hide whitespace changes
Inline
Side-by-side
Showing
11 changed files
with
616 additions
and
3 deletions
+616
-3
buildout.cfg
buildout.cfg
+5
-1
conf.py
conf.py
+3
-1
documentation/guide/index.rst
documentation/guide/index.rst
+4
-0
documentation/guide/install-and-run.rst
documentation/guide/install-and-run.rst
+232
-0
documentation/guide/writing-persistent-objects.rst
documentation/guide/writing-persistent-objects.rst
+3
-1
documentation/reference/index.rst
documentation/reference/index.rst
+11
-0
documentation/reference/storages.rst
documentation/reference/storages.rst
+199
-0
documentation/reference/zodb.rst
documentation/reference/zodb.rst
+149
-0
index.rst
index.rst
+1
-0
requirements.txt
requirements.txt
+5
-0
zodbdocumentationtests/tests.py
zodbdocumentationtests/tests.py
+4
-0
No files found.
buildout.cfg
View file @
2e27c167
[buildout]
develop = .
develop = .
../zodb
parts =
stxpy
test
...
...
@@ -13,6 +13,10 @@ recipe = zc.recipe.egg
eggs =
Sphinx
docutils
ZODB
j1m.sphinxautointerface
j1m.sphinxautozconfig
interpreter = stxpy
scripts =
sphinx-build
...
...
conf.py
View file @
2e27c167
...
...
@@ -24,7 +24,9 @@
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions
=
[]
extensions
=
[
'sphinx.ext.autodoc'
,
'j1m.sphinxautointerface'
,
'j1m.sphinxautozconfig'
]
# Add any paths that contain templates here, relative to this directory.
templates_path
=
[
'.templates'
]
...
...
documentation/guide/index.rst
View file @
2e27c167
...
...
@@ -12,7 +12,11 @@ If you haven't yet, you should read the :ref:`Tutorial <tutorial-label>`.
.. toctree::
:maxdepth: 2
install-and-run
writing-persistent-objects.rst
.. todo:
transaction.rst
storages.rst
configuration.rst
...
...
documentation/guide/install-and-run.rst
0 → 100644
View file @
2e27c167
==========================
Installing
and
running
ZODB
===========================
This
topic
discusses
some
boring
nitty
-
gritty
details
needed
to
actually
run
ZODB
.
Installation
============
Installation
of
ZODB
is
pretty
straightforward
using
Python
's
packaging system. For example, using pip::
pip install ZODB
You may need additional optional packages, such as `ZEO
<https://pypi.python.org/pypi/ZEO>`_ or `RelStorage
<https://pypi.python.org/pypi/RelStorage>`_, depending your deployment
choices.
Configuration
=============
You can set up ZODB in your application using either Python, or
ZODB'
s
configuration
language
.
For
simple
database
setup
,
and
especially
for
exploration
,
the
Python
APIs
are
sufficient
.
For
more
complex
configurations
,
you
'll probably find ZODB'
s
configuration
language
easier
to
use
.
To
understand
database
setup
,
it
's important to understand ZODB'
s
architecture
.
ZODB
separates
database
functionality
from
storage
concerns
.
When
you
create
a
database
object
,
you
specify
a
storage
object
for
it
to
use
,
as
in
::
import
ZODB
,
ZODB
.
FileStorage
storage
=
ZODB
.
FileStorage
.
FileStorage
(
'mydata.fs'
)
db
=
ZODB
.
DB
(
storage
)
So
when
you
define
a
database
,
you
'll also define a storage. In the
example above, we define a :class:`file storage
<ZODB.FileStorage.FileStorage.FileStorage>` and then use it to define
a database.
Sometimes, storages are created through composition. For example, if
we want to save space, we could layer a ``ZlibStorage``
[#zlibstoragefn]_ over the file storage::
import ZODB, ZODB.FileStorage, zc.zlibstorage
storage = ZODB.FileStorage.FileStorage('
mydata
.
fs
')
compressed_storage = zc.zlibstorage.ZlibStorage(storage)
db = ZODB.DB(compressed_storage)
`ZlibStorage <https://pypi.python.org/pypi/zc.zlibstorage>`_
compresses database records [#zlib]_.
Python configuration
--------------------
To set up a database with Python, you'
ll
construct
a
storage
using
the
:
ref
:`
storage
APIs
<
included
-
storages
-
label
>`,
and
then
pass
the
storage
to
the
:
class
:`~
ZODB
.
DB
`
class
to
create
a
database
,
as
shown
in
the
examples
in
the
previous
section
.
The
:
class
:`~
ZODB
.
DB
`
class
also
accepts
a
string
path
name
as
its
storage
argument
to
automatically
create
a
file
storage
.
You
can
also
pass
``
None
``
as
the
storage
to
automatically
use
a
:
class
:`~
ZODB
.
MappingStorage
.
MappingStorage
`,
which
is
convenient
when
exploring
ZODB
::
db
=
ZODB
.
DB
(
None
)
#
Create
an
in
-
memory
database
.
Text
configuration
------------------
ZODB
supports
a
text
-
based
configuration
language
.
It
uses
a
syntax
similar
to
Apache
configuration
files
.
The
syntax
was
chosen
to
be
familiar
to
site
administrators
.
ZODB
's text configuration uses `ZConfig
<https://pypi.python.org/pypi/ZConfig>`_. You can use ZConfig to
create your application'
s
configuration
,
but
it
's more common to
include ZODB configuration strings in their own files or embedded in
simpler configuration files, such as `configarser
<https://docs.python.org/3/library/configparser.html#module-configparser>`_
files.
A database configuration string has a ``zodb`` section wrapping a
storage section, as in::
<zodb>
cache-size-bytes 100MB
<mappingstorage>
</mappingstorage>
</zodb>
.. -> snippet
In the example above, the :ref:`mappingstorage
<mappingstorage-text-configuration>` section defines the storage used
by the database.
To create a database from a string, use
:func:`ZODB.config.databaseFromString`::
>>> import ZODB.config
>>> db = ZODB.config.databaseFromString(snippet)
To load databases from file names or URLs, use
:func:`ZODB.config.databaseFromURL`.
URI-based configuration
-----------------------
Another database configuration option is provided by the `zodburi
<https://pypi.python.org/pypi/zodburi>`_ package. See:
http://docs.pylonsproject.org/projects/zodburi. It'
s
less
powerful
than
the
Python
or
text
configuration
options
,
but
allows
configuration
to
be
reduced
to
a
single
URI
and
handles
most
cases
.
Using
databases
:
connections
============================
Once
you
have
a
database
,
you
need
to
get
a
database
connection
to
to
much
of
anything
.
Connections
take
care
of
loading
and
saving
objects
and
manage
object
caches
.
Each
connection
has
it
's own cache
[#caches-are-expensive]_.
Getting connections
-------------------
Amongst [#amongst]_ the common ways of getting a connection:
db.open()
The database :meth:`~ZODB.DB.open` method opens a
connection, returning a connection object::
>>> conn = db.open()
It'
s
up
to
the
application
to
call
:
meth
:`~
ZODB
.
Connection
.
Connection
.
close
`
when
the
application
is
done
using
the
connection
.
If
changes
are
made
,
the
application
:
ref
:`
commits
transactions
<
commit
-
transactions
>`
to
make
them
permanent
.
db
.
transaction
()
The
database
:
meth
:`~
ZODB
.
DB
.
transaction
`
method
returns
a
context
manager
that
can
be
used
with
the
`
python
with
statement
<
https
://
docs
.
python
.
org
/
3
/
reference
/
compound_stmts
.
html
#
grammar
-
token
-
with_stmt
>`
_
to
execute
a
block
of
code
in
a
transaction
::
with
db
.
transaction
()
as
connection
:
connection
.
root
.
foo
=
1
..
->
src
>>>
exec
(
src
)
>>>
with
db
.
transaction
()
as
connection
:
...
print
connection
.
root
.
foo
1
>>>
_
=
conn
.
transaction_manager
.
begin
()
#
get
updates
on
conn
In
the
example
above
,
we
used
``
as
connection
``
to
get
the
database
connection
used
in
the
variable
``
connection
``.
some_object
.
_p_jar
For
code
that
's already running in the context of an open
connection, you can get the current connection as the ``_p_jar``
attribute of some persistent object that was accessed via the
connection.
Getting objects
---------------
Once you have a connection, you access objects by traversing the
object graph from the root object.
The database root object is a mapping object that holds the top level
objects in the database. There should only be a small number of
top-level objects (often only one). You can get the root object by calling a
connection'
s
``
root
``
attribute
::
>>>
root
=
conn
.
root
()
>>>
root
{
'foo'
:
1
}
>>>
root
[
'foo'
]
1
For
convenience
[#
root
-
convenience
]
_
,
you
can
also
get
top
-
level
objects
by
accessing
attributes
of
the
connection
root
object
:
>>>
conn
.
root
.
foo
1
Once
you
have
a
top
-
level
object
,
you
use
its
methods
,
attributes
,
or
operations
to
access
other
objects
and
so
on
to
get
the
objects
you
need
.
Often
indexing
data
structures
like
BTrees_
are
used
to
make
it
possible
to
search
objects
in
large
collections
.
..
[#
zlibstoragefn
]
`
zc
.
zlibstorage
<
https
://
pypi
.
python
.
org
/
pypi
/
zc
.
zlibstorage
>`
_
is
an
optional
package
that
you
need
to
install
separately
.
..
[#
zlib
]
ZlibStorage
uses
the
:
mod
:`
zlib
`
standard
module
,
which
uses
the
`
zlib
library
<
http
://
www
.
zlib
.
net
/>`
_
.
..
[#
caches
-
are
-
expensive
]
ZODB
can
be
very
efficient
at
caching
data
in
memory
,
especially
if
your
`
working
set
<
https
://
en
.
wikipedia
.
org
/
wiki
/
Working_set
>`
_
is
small
enough
to
fit
in
memory
,
because
the
cache
is
simply
an
object
tree
and
accessing
a
cached
object
typically
requires
no
database
interaction
.
Because
each
connection
has
its
own
cache
,
connections
can
be
expensive
,
depending
on
their
cache
sizes
.
For
this
reason
,
you
'll generally want to limit the number of open
connections you have at any one time. Connections are pooled, so
opening a connection is inexpensive.
.. [#amongst] https://www.youtube.com/watch?v=7WJXHY2OXGE
.. [#root-convenience] The ability to access top-level objects of the
database as root attributes is a recent convenience. Originally,
the ``root()`` method was used to access the root object which was
then accessed as a mapping. It'
s
still
potentially
useful
to
access
top
-
level
objects
using
the
mapping
interface
if
their
names
aren
't valid attribute names.
.. _BTrees: https://pythonhosted.org/BTrees/
documentation/guide/writing-persistent-objects.rst
View file @
2e27c167
...
...
@@ -157,7 +157,7 @@ record from the book record and is managed by ZODB independent of the
management
of
the
book
.
In
addition
to
``
PersistentList
``
and
``
PersistentMapping
``,
general
persistent
data
structures
are
provided
by
the
``
BTrees
``
package
,
persistent
data
structures
are
provided
by
the
BTrees_
package
,
most
notably
``
BTree
``
and
``
TreeSet
``
objects
.
Unlike
``
PersistentList
``
and
``
PersistentMapping
``,
``
BTree
``
and
``
TreeSet
``
objects
are
scalable
and
can
easily
hold
millions
of
...
...
@@ -566,3 +566,5 @@ framework for managing schema-migration scripts.
<https://pypi.python.org/pypi/zope.cachedescriptors>`_ package
provides some descriptors that help implement attributes that cache
data.
.. _BTrees: https://pythonhosted.org/BTrees/
documentation/reference/index.rst
0 → 100644
View file @
2e27c167
=======================
Reference Documentation
=======================
.. toctree::
:maxdepth: 2
zodb.rst
storages.rst
documentation/reference/storages.rst
0 → 100644
View file @
2e27c167
=============
Storage APIs
=============
.. contents::
Storage interfaces
==================
There are various storage implementations that implement standard
storage interfaces. Thet differ primarily in their constructors.
Application code rarely calls storage methods, and those it calls are
generally called indirectly through databases. There are
interface-defined methods that are called internally by ZODB. These
aren't shown below.
IStorage
--------
.. autointerface:: ZODB.interfaces.IStorage
:members: close, getName, getSize, history, isReadOnly, lastTransaction,
__len__, pack, sortKey
IStorageIteration
-----------------
.. autointerface:: ZODB.interfaces.IStorageIteration
IStorageUndoable
----------------
.. autointerface:: ZODB.interfaces.IStorageUndoable
:members: undoLog, undoInfo
IStorageCurrentRecordIteration
------------------------------
.. autointerface:: ZODB.interfaces.IStorageCurrentRecordIteration
IBlobStorage
------------
.. autointerface:: ZODB.interfaces.IBlobStorage
:members: temporaryDirectory
IStorageRecordInformation
-------------------------
.. autointerface:: ZODB.interfaces.IStorageRecordInformation
IStorageTransactionInformation
------------------------------
.. autointerface:: ZODB.interfaces.IStorageTransactionInformation
.. _included-storages-label:
Included storages
=================
FileStorage
-----------
.. autoclass:: ZODB.FileStorage.FileStorage.FileStorage
:members: __init__
.. autointerface:: ZODB.FileStorage.interfaces.IFileStoragePacker
FileStorage text configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
File storages are configured using the ``filestorage`` section::
<filestorage>
path Data.fs
</filestorage>
which accepts the following options:
.. zconfigsectionkeys:: ZODB component.xml filestorage
MappingStorage
--------------
.. autoclass:: ZODB.MappingStorage.MappingStorage
:members: __init__
.. _mappingstorage-text-configuration:
MappingStorage text configuration
---------------------------------
File storages are configured using the ``mappingstorage`` section::
<mappingstorage>
</mappingstorage>
Options:
.. zconfigsectionkeys:: ZODB component.xml mappingstorage
DemoStorage
-----------
.. autoclass:: ZODB.DemoStorage.DemoStorage
:members: __init__, push, pop
DemoStorage text configuration
------------------------------
Demo storages are configured using the ``demostorage`` section::
<demostorage>
<filestorage base>
path base.fs
</filestorage>
<mappingstorage changes>
name Changes
</mappingstorage>
</demostorage>
.. -> src
>>> import ZODB.config
>>> storage = ZODB.config.storageFromString(src)
>>> storage.base.getName()
'base.fs'
>>> storage.changes.getName()
'Changes'
``demostorage`` sections can contain up to 2 storage subsections,
named ``base`` and ``changes``, specifying the demo storage's base and
changes storages. See :meth:`ZODB.DemoStorage.DemoStorage.__init__`
for more on the base anc changes storages.
Options:
.. zconfigsectionkeys:: ZODB component.xml demostorage
Noteworthy non-included storages
================================
A number of important ZODB storages are distriubuted separately, including:
RelStorage
`RelStorage <http://relstorage.readthedocs.io/en/latest/>`_
stores data in relational databases. This is especially
useful when you have requirements or existing infrastructure for
storing data in relational databases. Unlike the included storages,
multiple processes can share the same database.
For more imformation, see http://relstorage.readthedocs.io/en/latest/.
ZEO
`ZEO <https://github.com/zopefoundation/ZEO>`_ is a client-server
database implementation for ZODB. To use ZEO, you run a ZEO server,
and use ZEO clients in your application. Unlike the included
storages, multiple processes can share the same database.
For more imformation, see https://github.com/zopefoundation/ZEO.
ZRS
`ZRS <https://github.com/zc/zrs>`_
provides replication from one database to another. It's most
commonly used with ZEO. With ZRS, you create a ZRS primary database
around a :class:`~ZODB.FileStorage.FileStorage.FileStorage` and in a
separate process, you creatre a ZRS secondary storage around any
:interface:`storage <ZODB.interfaces.IStorage>`. As transactions are
committed on the primary, they're copied asynchronously to
secondaries.
For more imformation, see https://github.com/zc/zrs.
zlibstorage
`zlibstorage <https://pypi.python.org/pypi/zc.zlibstorage>`_
compresses database records using the compression
algorithm used by `gzip <http://www.gzip.org/>`_.
For more imformation, see https://pypi.python.org/pypi/zc.zlibstorage.
beforestorage
`beforestorage <https://pypi.python.org/pypi/zc.beforestorage>`_
provides a point-in-time view of a database that might
be changing. This can be useful to provide a non-changing view of a
production database for use with a :class:`~ZODB.DemoStorage.DemoStorage`.
For more imformation, see https://pypi.python.org/pypi/zc.beforestorage.
cipher.encryptingstorage
`cipher.encryptingstorage
<https://pypi.python.org/pypi/cipher.encryptingstorage/>`_ provided
compression and encryption of database records.
For more informayion see,
https://pypi.python.org/pypi/cipher.encryptingstorage/.
documentation/reference/zodb.rst
0 → 100644
View file @
2e27c167
=========
ZODB APIs
=========
.. contents::
ZODB module functions
=====================
.. method:: DB(storage, *args, **kw)
Create a databse. See :py:class:`ZODB.DB`.
.. autofunction:: ZODB.connection
Databases
=========
.. autoclass:: ZODB.DB
:members: __init__, open, close, pack,
cacheDetail, cacheExtremeDetail, cacheMinimize,
cacheSize, cacheDetailSize, getCacheSize, getCacheSizeBytes,
lastTransaction, getName, getPoolSize, getSize,
getHistoricalCacheSize, getHistoricalCacheSizeBytes,
getHistoricalPoolSize, getHistoricalTimeout,
objectCount, connectionDebugInfo,
setCacheSize, setCacheSizeBytes,
setHistoricalCacheSize, setHistoricalCacheSizeBytes,
setPoolSize, setHistoricalPoolSize, setHistoricalTimeout,
history,
supportsUndo, undoLog, undoInfo, undoMultiple, undo,
transaction, storage
.. _database-text-configuration:
Database text configuration
---------------------------
Databases are configured with ``zodb`` sections::
<zodb>
cache-size-bytes 100MB
<mappingstorage>
</mappingstorage>
</zodb>
A ``zodb`` section must have a storage sub-section specifying a
storage and any of the following options:
.. zconfigsectionkeys:: ZODB component.xml zodb
.. _multidatabase-text-configuration:
For a multi-database configuration, use multiple ``zodb`` sections and
give the sections names::
<zodb first>
cache-size-bytes 100MB
<mappingstorage>
</mappingstorage>
</zodb>
<zodb second>
<mappingstorage>
</mappingstorage>
</zodb>
.. -> src
>>> import ZODB.config
>>> db = ZODB.config.databaseFromString(src)
>>> sorted(db.databases)
['first', 'second']
>>> db._cache_size_bytes
104857600
When the configuration is loaded, a single database will be returned,
but all of the databases will be available through the returned
database's ``databases`` attribute.
Connections
===========
.. autoclass:: ZODB.Connection.Connection
:members: add, cacheGC, cacheMinimize, close, db, get,
getDebugInfo, get_connection, isReadOnly, oldstate,
onCloseCallback, root, setDebugInfo, sync
TimeStamp (transaction ids)
===========================
.. class:: ZODB.TimeStamp.TimeStamp(year, month, day, hour, minute, seconds)
Create a time-stamp object. Time stamps facilitate the computation
of transaction ids, which are based on times. The arguments are
integers, except for seconds, which may be a floating-point
number. Time stamps have microsecond precision. Time stamps are
implicitly UTC based.
Time stamps are orderable and hashable.
.. method:: day()
Return the time stamp's day.
.. method:: hour()
Return the time stamp's hour.
.. method:: laterThan(other)
Return a timestamp instance which is later than 'other'.
If self already qualifies, return self.
Otherwise, return a new instance one moment later than 'other'.
.. method:: minute()
Return the time stamp's minute.
.. method:: month()
Return the time stamp's month.
.. method:: raw()
Get an 8-byte representatin of the time stamp for use in APIs
that require a time stamp.
.. method:: second()
Return the time stamp's second.
.. method:: timeTime()
Return the time stamp as seconds since the epoc, as used by the
``time`` module.
.. method:: year()
Return the time stamp's year.
Loading configuration
=====================
.. automodule:: ZODB.config
:members: databaseFromString, databaseFromFile, databaseFromURL,
storageFromString, storageFromFile, storageFromURL
index.rst
View file @
2e27c167
...
...
@@ -198,6 +198,7 @@ Learning more
documentation/tutorial
documentation/guide/index
documentation/reference/index
documentation/articles/index
* `The ZODB Book (in progress) <http://zodb.readthedocs.org/en/latest/>`_
...
...
requirements.txt
0 → 100644
View file @
2e27c167
Sphinx
docutils
ZODB
j1m.sphinxautointerface
j1m.sphinxautozconfig
zodbdocumentationtests/tests.py
View file @
2e27c167
...
...
@@ -34,11 +34,15 @@ def tearDown(test):
def
test_suite
():
here
=
os
.
path
.
dirname
(
__file__
)
guide
=
join
(
here
,
'..'
,
'documentation'
,
'guide'
)
reference
=
join
(
here
,
'..'
,
'documentation'
,
'reference'
)
return
unittest
.
TestSuite
((
manuel
.
testing
.
TestSuite
(
manuel
.
doctest
.
Manuel
()
+
manuel
.
capture
.
Manuel
(),
join
(
guide
,
'writing-persistent-objects.rst'
),
join
(
guide
,
'install-and-run.rst'
),
join
(
reference
,
'zodb.rst'
),
join
(
reference
,
'storages.rst'
),
setUp
=
setUp
,
tearDown
=
tearDown
,
),
))
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment