Commit 5ff0a4e0 authored by Vincent Pelletier's avatar Vincent Pelletier

erp5: Improve replication setup support.

Document how to setup replication.
Add support for no-zope ERP5 instance, so it is possible to setup a
replicating, full-featured ERP5 Cluster instance tree without providing
service until it needs to take over its upstream, or to become
otherwise independent.
parent a50c91ef
......@@ -4,7 +4,7 @@ Available ``software-type`` values
- ``default``
Recommended for production use.
- ``create-erp5-site``
Automated creation of ERP5Site instance, for easy deployment.
......@@ -22,6 +22,62 @@ Included cloudooo partition is **deprecated**. It is not recommended for
intensive usage. See the ``cloudooo`` Software Release to setup a cloudooo
cluster, more suitable for intensive usage.
Replication
===========
Replication allows setting up an ERP5 instance whose data follows another
instance.
Relations between ERP5 instances in a replication graph depend in what is
supported by individual data managers (ex: a neo cluster can replicate from a
neo cluster which itself replicates from a 3rd).
Replication lag constraints (aka sync/async replication) depends on individual
data managers (ex: neo replication between clusters is always asynchronous).
Ignoring replication lag, replicated data can be strictly identical (ex:
replicating ZODB or SQL database will contain the same data as upstream), or
may imply some remaping (ex: replicating Zope logs from an instance with 2 zope
families with 2 partition of 2 zopes each to an instance with a single zope
total).
Data whose replication is supported
-----------------------------------
- neo database
Data whose replication will eventually be supported
---------------------------------------------------
- mariadb database
- zope ``zope-*-access.log`` and ``zope-*-Z2.log``
- ``mariadb-slow.log``
Data whose replication is not planned
-------------------------------------
- zeo: use neo instead
Setting up replication
----------------------
In addition to your usual parameter set, you needs to provide the following parameters::
{
"zope-partition-dict": {}, So no zope is instanciated
"zodb": [
{
"storage-dict": {
"upstream-masters": ..., As published by to-become upstream ERP5 instance as "neo-masters"
},
"type": "neo", The only ZODB type supporting replication
...
}
...
]
...
}
Port ranges
===========
......
......@@ -331,7 +331,7 @@ rendered = ${monitor-template-dummy:target}
[template-erp5]
<= download-base
filename = instance-erp5.cfg.in
md5sum = 66edf64eeaecded8977459acb26f4424
md5sum = cb0a8dfc15874939ec786a8b7ac5bff1
[template-zeo]
<= download-base
......
......@@ -117,6 +117,10 @@ name = neo-${gen-neo-cluster-base:passwd}
[gen-smtpd-sasl-password]
< = gen-password
{% set zope_partition_dict = slapparameter_dict.get('zope-partition-dict', {'1': {}}) -%}
{% set zope_address_list_id_dict = {} -%}
{% if zope_partition_dict -%}
[request-zope-base]
<= request-common
return =
......@@ -158,7 +162,7 @@ software-type = zope
{% set zope_family_dict = {} -%}
{% set jupyter_zope_family_default = [] -%}
{% for custom_name, zope_parameter_dict in slapparameter_dict.get('zope-partition-dict', {'1': {}}).items() -%}
{% for custom_name, zope_parameter_dict in zope_partition_dict.items() -%}
{% set partition_name = 'zope-' ~ custom_name -%}
{% set section_name = 'request-' ~ partition_name -%}
{% set zope_family = zope_parameter_dict.get('family', 'default') -%}
......@@ -189,7 +193,6 @@ config-webdav = {{ dumps(zope_parameter_dict.get('webdav', False)) }}
{% endif -%}
{# We need to concatenate lists that we cannot read as lists, so this gets hairy. -#}
{% set zope_address_list_id_dict = {} -%}
{% set zope_family_parameter_dict = {} -%}
{% for family_name, zope_section_id_list in zope_family_dict.items() -%}
{% for zope_section_id in zope_section_id_list -%}
......@@ -264,6 +267,8 @@ config-{{ name }} = {{ value }}
return = site_url
{% endif -%}
{% endif -%}{# if zope_partition_dict -#}
[publish]
recipe = slapos.cookbook:publish.serialised
-extends = publish-early
......@@ -271,11 +276,13 @@ recipe = slapos.cookbook:publish.serialised
neo-masters = ${neo-0-final:connection-masters}
neo-admins = ${neo-0-final:connection-admins}
{% endif -%}
{% if zope_address_list_id_dict -%}
{#
Pick any published hosts-dict, they are expected to be identical - and there is
no way to check here.
-#}
hosts-dict = {{ '${' ~ zope_address_list_id_dict.keys()[0] ~ ':connection-hosts-dict}' }}
{% endif -%}
{% for name, value in publish_dict.items() -%}
{{ name }} = {{ value }}
{% endfor -%}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment