This applies only to an empty cluster and makes the cluster transition
from the initial state RECOVERING to BACKINGUP without passing through
state RUNNING:

(empty) -> RECOVERING -> VERIFYING -> STARTING_BACKUP -> BACKINGUP

This is the only way to reach BACKINGUP without having to pass through
RUNNING first and manually transitioning to STARTING_BACKUP. It is the
only way to reach state BACKINGUP fully automatically.

See merge request !25

Even if the user should have left backup mode before truncating
upstream, this change should help fixing his mistake and minimizing
the risk of data corruption.

The previous behaviour was to crash with:

  RuntimeError: upstream DB truncated

This led to 2 issues:

1. As long as upstream last tid remains older, it is impossible to
   start the backup cluster if it's able to connect to upstream:
   then to avoid another upstream downtime, it is required to fake
   a connection failure, e.g. with temporary firewall rules or
   different --upstream-* parameters, which is not practical.
2. Worse, if there's again new commits upstream with last tid newer
   than on backup, the user may miss to also truncate the backup
   cluster (even more if it's setup to restart automatically).

We had a case of a backup cluster that packed whereas --disable-pack
was passed. In fact, for some reasons, we had to hack to pass the
option so things aren't clear.

This is intended as a sanity check, so simple typos in neoctl truncate
command do not easily lead to the entire database being wiped.

See merge request nexedi/neoppod!23

- do not notify about deleted oids, like ZEO
- exchange lists of partitions instead of oids when possible

Packing now requires the DB to be open RW.

The changes consist in:
- indenting many lines without exceeding 80 chars
- setting conn = None ASAP on ConnectionClosed
- adding a try..except clause

Starting from next commit, the exception can be POSException.ReadOnlyError.

Before this change, the only distinction between a timestamp and a TID was
the presence of the decimal separator, ".". As a result, a timestamp
mistakenly provided without a decimal separator would be interpreted as a
TID, which will be somewhere in January 1900 (as TIDs are 64bits with much
finer accuracy than timestamps). When used to truncate a database, and in
the absence of sanity checks, this would simply wipe the database.

So, instead of just relying on a decimal separator, require a longer
string. Make it a prefix for readability. Also, TIDs are more niche than
timestamp, require them to have a mark, and do not require anything from
timestamps.

The spaces are optional so it's cosmetics.

In practice, it may be only required for SQLite (because we don't force index)
on 'obj' (which as several indices).

This will be used by an external GC.

To be pushed upstream.

set_wakeup_fd only works in main thread.

See commit f47dd646.

Since the switch to msgpack, there's no more type checking at protocol level
and for example passing an integer would cause storage nodes to crash.

But as shown here, the type checking of the old protocol was not always
enough, because data structures at client side could anyway get wrong.

Else it leads to DB corruption and a crash of the master.

This fixes commit 0e43dd1f
("Fix signals not always being processed as soon as possible").

See commit b6f821a2.

Julien notes this is very likely unneeded:
nexedi/neoppod!21 (diffs, comment 195929)

We had it like this since 01a01c8c (client: Add support for zodburi),
but I rechecked zodburi codebase now and it does not do any similar
lowering anywhere.

So drop support for case normalization in zurl options.

/cc @levin.zimmermann
/reviewed-by @jm
/reviewed-on nexedi/neoppod!21

Unfortunately after creating SSL context it is not possible, or at least
I could not find how, to retrieve original credentials with which the
context was created. However wendelin.core needs to be able to take a
client storage, reconstruct zurl to refer to that particular storage,
and pass that zurl to wcfs, so that wcfs, in turn, could access the same
ZODB database.

Given a NEO client instance, it is already possible to retrieve
master_nodes, cluster name, and detect whether SSL is being in use.
However without being able to retrieve original SSL credentials,
reconstructed zurl will not be full and wcfs won't be able to use
exactly the same secrets as python part does.

-> Help wendelin.core by remembering which ca/cert/key were used to
build SSL context.

This information is used by zstor_2zurl in wendelin.core here:

https://lab.nexedi.com/nexedi/wendelin.core/blob/885b3556/lib/zodb.py#L390-418

/cc @levin.zimmermann
/reviewed-by @jm
/reviewed-on nexedi/neoppod!21

Similarly to how it is done with e.g. http:// and https:// - if neos://
is given TLS usage is forced and ca/cert/key must be there either in the
URI itself, or in $NEO_CA, $NEO_CERT and $NEO_KEY environment variables
mimicking the way how e.g. for https:// TLS credentials are taken from
host environment, not from the uri.

The latter might be usability convenience, but is also useful for WCFS
which needs to be able to remove secrets from uri on zurl normalization.

Please see discussion at !18 (comment 184439)
for details.

/cc @levin.zimmermann
/reviewed-by @jm
/reviewed-on !21

Because list of masters and cluster name must be already present in
netloc and path. Previously e.g.

	neo://db@α,β,γ?master_nodes=a,b,c"

would mean to use master nodes {a,b,c} not {α,β,γ}. Now it is treated as
invalid URI to remove ambiguity. Same for cluster name.

/cc @levin.zimmermann
/reviewed-by @jm
/reviewed-on !21

Pre-mortem data:
Traceback (most recent call last):
File "neo/master/app.py", line 172, in run
self._run()
File "neo/master/app.py", line 180, in _run
self.listening_conn = ListeningConnection(self, None, self.server)
File "neo/lib/connection.py", line 298, in __init__
connector.makeListeningConnection()
File "neo/lib/connector.py", line 133, in makeListeningConnection
self._error('listen', e)
File "neo/lib/connector.py", line 93, in _error
raise ConnectorException
ConnectorException
Traceback (most recent call last):
  File "neomaster", line 50, in <module>
    sys.exit(neo.scripts.neomaster.main())
  File "neo/scripts/neomaster.py", line 31, in main
    app.run()
  File "neo/master/app.py", line 175, in run
    self.log()
  File "neo/master/app.py", line 167, in log
    if self.pt is not None:
AttributeError: 'Application' object has no attribute 'pt'