Documentation

README

@@ -15,32 +15,28 @@ re6stnet creates a resilient, scalable, ipv6 network on top of an existing ipv4
 network, by creating tunnels on the fly, and then routing targeted traffic
 through these tunnels.
 
-re6stnet can be used to :
+re6stnet can be used to:
 
 - guarantee connectedness between computers connected to the
-  internet, for which there exists a working route ( in case the direct route
-  isn't available ).
-- create very large networks
+  internet, for which there exists a working route (in case the direct route
+  isn't available).
+- create large networks
 - give ipv6 addresses to machines with only ipv4 available
 
-
 HOW IT WORKS
 ============
 
-A re6stnet networks consists of at least one server ( re6st-registry ) and many
-clients or nodes ( re6stnet ). The server delivers certificates for secure
-authentification in establishing tunnels, and - for now - store the complete
-list of nodes running.
-re6stnet detects and take into accounts nodes present on the local network.
+A re6stnet network consists of at least one server (re6st-registry) and many
+nodes (re6stnet). The server is only used to deliver certificates for secure
+authentification in establishing tunnels, and to bootstrap new nodes.
+re6stnet can detect and take into account nodes present on the local network.
 
 Resilience
 ----------
-re6stnet can guarantee that if there exists a route between two machines,
+re6stnet guarantees that if there exists a route between two machines,
 traffic will be correctly routed between these two machines.
-Since the destination of established connection are random, there can be no
-guarantee that such a route exists given which tunnels are established, but
-for big enough networks ( more than a hundred nodes ), the probability that
-the network isn't connected is very low.
+Even if the registry node is down, the probability that the network isn't
+connected is very low for big enough networks (more than a hundred nodes).
 
 Scalability
 -----------
@@ -51,30 +47,23 @@ scalable to tens of thousand of nodes
 Requirements
 ============
 
-- Python 2.7 or later
-
+- Python 2.6 or 2.7
 - OpenSSL binary and development libraries
-
 - OpenVPN
+- Babel_
+- python-miniupnpc for UPnP support (optional)
+- for the demo: miniupnpd_, Graphviz, Screen, Nemu_
 
-- Babel ( http://www.pps.univ-paris-diderot.fr/~jch/software/babel/ )
+See also `setup.py` for Python dependencies.
+
+.. _Babel: http://www.pps.univ-paris-diderot.fr/~jch/software/babel/
+   _Nemu: http://code.google.com/p/nemu/
+   _miniupnpd: http://miniupnp.free.fr/
 
 Installation
 ============
 
-Let's explain how to install the egg
-
-Commercial Support
-==================
-
-Later....
+re6stnet is distributed as an Python egg, and is also packaged for Debian.
 
-.. The organisation of the code
-    re6stnet.py         Client : Init and main loop
-    re6st/plib.py             Launch server/client/routing processes
-    re6st/utils.py            Small functions to do useful jobs
-    re6st/db.py               Manage the peer database
-    re6st/tunnel.py           Manage the tunnels
-    re6st/upnpigd.py          Forward ports
-    re6st-conf.py       Get certificates from server
-    re6st-registry.py   Server : deliver certificates and distribute peers
+See `re6stnet-registry` to set up a re6st network
+and `re6stnet-conf` to join an existing network.

docs/re6st-conf.rst

@@ -12,67 +12,21 @@ Configuration tool for re6stnet
 SYNOPSIS
 ========
 
-``re6st-conf`` ``--server`` `server-url` ``--port`` `server-port` [`command`] 
-[`options`...]
+``re6st-conf`` ``--registry`` `url` [`options`...]
 
 DESCRIPTION
 ===========
 
-re6st-conf is a tool generating the files necessary for re6stnet to run. It
-connects to a distant server to generate the certificates files used by openvpn
-in re6stnet.
+re6st-conf is a tool generating the files necessary to add a node to a re6st
+network. It connects to the re6st registry to get a signed certificate with
+an allocated IPv6 subnet.
 
 USAGE
 =====
 
-re6st-conf needs address of node running re6st-registry.
-
---registry address
-             Public HTTP URL of the registry, which is used for bootstrapping
-             and delivering certificates.
-
-Commands
---------
-
-You may give at most one of the following commands. Giving the two commands
-will not raise any errors, but does not have any meaning.
-
---email email-address
-            The email address to associate with your certificates. Refrain
-            re6st-conf from asking you your email address in a prompt. Useful
-            for automatizing the configuration process.
-            re6st-conf will ask you the token you received by email.
-
---token token
-            Give your token if you have already received one but did not
-            complete the configuration process. re6st-conf will not ask you
-            anything. If you use this option, you do not need to give your
-            email address.
-
-Options
--------
-
--d, ``--dir`` `directory`
-            Path of a directory where will be stored the files generated by the
-            setup. The Setup genereates the following files, in the explicit
-            order :
-
-            - ca.pem : certificate authority file downloaded from the server
-            - cert.key : private key generated by the script
-            - cert.crt : individual certificate file generated by the server
-            - dh2048.pem : dh file for oenvpn server
-
--r, ``--req`` `name` `value`
-            Specify an attribute to add to the certificate request sent to the
-            server. Can be used multiple times.
-            Each use of the --req name value, will add the attribute name with
-            the associated value in the sugbject of the certificate request.
-
---ca-only
-            Stop the script after downloading the certificate authority file
-            from the server
+Use ``re6st-conf --help`` to get the complete list of options.
 
 SEE ALSO
 ========
 
-``re6stnet``\ (1), ``re6st-registry``\ (1)
+``re6stnet``\ (8)

docs/re6st-registry.rst

@@ -12,55 +12,28 @@
 SYNOPSIS
 ========
 
-``re6st-registry`` `port` ``--db`` `db-path` ``--ca`` `ca-path`
-``--key`` `key-path` ``--mailhost`` `mailhost` ``--private`` `private-ip`
-[`options`...]
+``re6st-registry`` ``--db`` `db-path`  ``--ca`` `ca-path` ``--key`` `key-path`
+``--mailhost`` `mailhost` ``--private`` `re6st-ip` [`options`...]
 
 DESCRIPTION
 ===========
 
 re6st-registry is a server for the re6st network. Its role is to deliver
-certificates to new nodes, and to maintain the complete table of peers, so it
-can send part of it to nodes asking for new peers.
-Only one re6st-registry per re6st network should run. The node
-running the re6st-registry must also have a client ( re6stnet ) running.
+certificates to new nodes, and to bootstrap nodes that don't know any address
+of other nodes.
+
+The network can host only one registry, which should run on a re6st node.
 
 USAGE
 =====
 
-The re6st-registry will automatically listen on both ipv4 and ipv6 for incoming
-request.
-
---port port
-            The port on which the server will listen
-            Default: 80
-
---db path
-            Path to the server Database file. A new DB file will be created
-            and correctly initialized if the file doesn't exists.
-            One can give ":memory" as path, the database is then temporary
-
---ca path
-            Path to the certificate authority file. The certificate authority
-            MUST contain the VPN network prefix in its serial number. To
-            generate correct ca and key files for the 2001:db8:42:: prefix,
-            the following command can be used :
-            openssl req -nodes -new -x509 -key ca.key -set_serial \
-                    0x120010db80042 -days 365 -out ca.crt
-
---key path
-            Path to the server key file. To generate a key file, see the --ca
-            option
-
---mailhost mailhost
-            Mailhost to be used to send email containing token for registration
+Use ``re6st-registry --help`` to get the complete list of options.
 
---private ip
-            Ipv6 address of the re6stnet client running on the machine. This
-            address will be advertised only to nodes having a valid
-            certificate.
+The network prefix can be changed by renewing the Certificate authority with
+a different serial number, but keeping the existing key. Then all nodes of the
+network must fetch the new CA with `re6st-conf` and restart `re6stnet`.
 
 SEE ALSO
 ========
 
-``re6stnet``\ (1), ``re6st-conf``\ (1)
+``re6stnet``\ (8)

re6st-conf

 #!/usr/bin/env python
-import argparse, os, subprocess, sqlite3, sys, xmlrpclib
+import argparse, atexit, os, subprocess, sqlite3, sys, xmlrpclib
 from OpenSSL import crypto
 from re6st import utils
 
-def create(path, text, mode=0666):
-    fd = os.open(path, os.O_CREAT | os.O_WRONLY | os.O_EXCL, mode)
+def create(path, text=None, mode=0666):
+    fd = os.open(path, os.O_CREAT | os.O_WRONLY | os.O_TRUNC, mode)
+    if text is None:
+        return fd
     try:
         os.write(fd, text)
     finally:
@@ -12,18 +14,24 @@ def create(path, text, mode=0666):
 
 def main():
     parser = argparse.ArgumentParser(
-            description='Setup script for re6stnet')
+        description="Setup script for re6stnet.",
+        formatter_class=utils.HelpFormatter)
     _ = parser.add_argument
+    _('--registry', required=True, metavar='URL',
+        help="HTTP URL of the server delivering certificates.")
     _('--ca-only', action='store_true',
-            help='To only get CA form server')
-    _('--registry', required=True,
-            help='HTTP URL of the server delivering certificates')
-    _('-d', '--dir', default='/etc/re6stnet',
-            help='Directory where the key and certificate will be stored')
-    _('-r', '--req', nargs=2, action='append',
-            help='Name and value of certificate request additional arguments')
-    _('--email', help='Your email address')
-    _('--token', help='The token you received')
+        help='Only fetch CA from registry and exit.')
+    _('-d', '--dir',
+        help="Directory where the key and certificate will be stored.")
+    _('-r', '--req', nargs=2, action='append', metavar=('KEY', 'VALUE'),
+        help="The registry only sets the Common Name of your certificate,"
+             " which actually encodes your allocated prefix in the network."
+             " You can repeat this option to add any field you want to its"
+             " subject.")
+    _('--email',
+        help="Email address where your token is sent. Use -r option if you"
+             " want to show an email in your certificate.")
+    _('--token', help="The token you received.")
     config = parser.parse_args()
     if config.dir:
         os.chdir(config.dir)
@@ -36,45 +44,60 @@ def main():
     s = xmlrpclib.ServerProxy(config.registry)
 
     # Get CA
-    ca = s.getCa()
-    create(ca_path, ca)
+    create(ca_path, s.getCa())
 
     if config.ca_only:
         sys.exit()
 
-    # Get token
+    # Generating dh file
+    if not os.access(dh_path, os.F_OK):
+        r = subprocess.call(('openssl', 'dhparam', '-out', dh_path, '2048'))
+        if r:
+            sys.exit(r)
+
+    req = crypto.X509Req()
+    subj = req.get_subject()
+    if config.req:
+        for k, v in config.req:
+            if k == 'CN':
+                sys.exit("CN field is reserved.")
+            setattr(subj, k, v)
+
+    cert_fd = token_advice = None
+    try:
         if not config.token:
             if not config.email:
-            config.email = raw_input('Please enter your email address : ')
-        _ = s.requestToken(config.email)
-        config.token = raw_input('Please enter your token : ')
+                config.email = raw_input('Please enter your email address: ')
+            s.requestToken(config.email)
+            token_advice = "Use --token to retry without asking a new token\n"
+            config.token = raw_input('Please enter your token: ')
 
-    # Generate key and cert request
+        # First make sure we can store private key and open certificate file
+        # for writing, to avoid using our token for nothing.
+        print "Generating key and cert requests ..."
+        cert_fd = create(cert_path)
         pkey = crypto.PKey()
         pkey.generate_key(crypto.TYPE_RSA, 2048)
         key = crypto.dump_privatekey(crypto.FILETYPE_PEM, pkey)
 
-    req = crypto.X509Req()
-    subj = req.get_subject()
-    if config.req:
-        for arg in config.req:
-            setattr(subj, arg[0], arg[1])
         req.set_pubkey(pkey)
         req.sign(pkey, 'sha1')
         req = crypto.dump_certificate_request(crypto.FILETYPE_PEM, req)
 
-    # Get certificate
-    cert = s.requestCertificate(config.token, req)
-
-    # Store cert and key
         create(key_path, key, 0600)
-    create(cert_path, cert)
-
-    # Generating dh file
-    if not os.access(dh_path, os.F_OK):
-        r = subprocess.call(('openssl', 'dhparam', '-out', dh_path, '2048'))
-        if r:
-            sys.exit(r)
+        print "Requesting certificate ..."
+        cert = s.requestCertificate(config.token, req)
+        if not cert:
+            token_advice = None
+            sys.exit("Error: invalid or expired token")
+    except:
+        if cert_fd:
+            os.remove(cert_path)
+        if token_advice:
+            atexit.register(sys.stdout.write, token_advice)
+        raise
+    os.write(cert_fd, cert)
+    os.close(cert_fd)
 
     print "Certificate setup complete."
 

re6st-registry

@@ -50,26 +50,35 @@ class main(object):
 
         # Command line parsing
         parser = utils.ArgParser(fromfile_prefix_chars='@',
-                description='Peer discovery http server for re6stnet')
+            description="re6stnet registry used to bootstrap nodes"
+                        " and deliver certificates.")
         _ = parser.add_argument
-        _('--port', type=int, default=80, help='Port of the host server')
-        _('--db', required=True,
-                help='Path to database file')
-        _('--ca', required=True,
-                help='Path to ca.crt file')
+        _('--port', type=int, default=80,
+            help="Port on which the server will listen.")
+        _('-4', dest='bind4', default='0.0.0.0',
+            help="Bind server to this IPv4.")
+        _('-6', dest='bind6', default='::',
+            help="Bind server to this IPv6.")
+        _('--db', default='/var/lib/re6stnet/registry.db',
+            help="Path to SQLite database file. It is automatically initialized"
+                 " if the file does not exist.")
+        _('--ca', required=True, help=parser._ca_help)
         _('--key', required=True,
-                help='Path to certificate key')
+                help="CA private key in .pem format.")
         _('--mailhost', required=True,
-                help='SMTP server mail host; for debugging purpose, it can also'
-                     ' be an absolute or existing path to a mailbox file')
+                help="SMTP host to send confirmation emails. For debugging"
+                     " purpose, it can also be an absolute or existing path to"
+                     " a mailbox file")
         _('--private',
-                help='VPN IP of the node on which runs the registry')
+                help="re6stnet IP of the node on which runs the registry."
+                     " Required for normal operation.")
         _('--prefix-length', default=16,
-                help='Default length of allocated prefixes')
+                help="Default length of allocated prefixes.")
         _('-l', '--logfile', default='/var/log/re6stnet/registry.log',
-                help='Path to logging file')
+                help="Path to logging file.")
         _('-v', '--verbose', default=1, type=int,
-                help='Log level')
+                help="Log level. 0 disables logging."
+                     " Use SIGUSR1 to reopen log.")
         self.config = parser.parse_args()
 
         utils.setupLog(self.config.verbose, self.config.logfile)
@@ -82,6 +91,7 @@ class main(object):
                     'check you configuration')
 
         # Database initializing
+        utils.makedirs(os.path.dirname(self.config.db))
         self.db = sqlite3.connect(self.config.db, isolation_level=None)
         self.db.execute("""CREATE TABLE IF NOT EXISTS token (
                         token text primary key not null,
@@ -111,15 +121,24 @@ class main(object):
         self._email = self.ca.get_subject().emailAddress
 
         # Starting server
-        server4 = SimpleXMLRPCServer4(('0.0.0.0', self.config.port), requestHandler=RequestHandler, allow_none=True)
+        server_list = []
+        if self.config.bind4:
+            server4 = SimpleXMLRPCServer4((self.config.bind4, self.config.port),
+                requestHandler=RequestHandler, allow_none=True)
             server4.register_instance(self)
-        server6 = SimpleXMLRPCServer6(('::', self.config.port), requestHandler=RequestHandler, allow_none=True)
+            server_list.append(server4)
+        if self.config.bind6:
+            server6 = SimpleXMLRPCServer6((self.config.bind6, self.config.port),
+                requestHandler=RequestHandler, allow_none=True)
             server6.register_instance(self)
+            server_list.append(server6)
 
-        # Main loop
+        if len(server_list) == 1:
+            server_list[0].serve_forever()
+        else:
             while True:
                 try:
-                r = select.select([server4, server6], [], [])[0]
+                    r = select.select(server_list[:], [], [])[0]
                 except select.error as e:
                     if e.args[0] != errno.EINTR:
                         raise
@@ -183,8 +202,7 @@ class main(object):
                 try:
                     token, email, prefix_len, _ = self.db.execute("SELECT * FROM token WHERE token = ?", (token,)).next()
                 except StopIteration:
-                    logging.exception('Bad token (%s) in request', token)
-                    raise
+                    return
                 self.db.execute("DELETE FROM token WHERE token = ?", (token,))
 
                 # Get a new prefix

re6st/utils.py

-import argparse, errno, logging, os, signal, struct, socket, time
+import argparse, errno, logging, os, signal, struct, socket, textwrap, time
 from OpenSSL import crypto
 
 logging_levels = logging.WARNING, logging.INFO, logging.DEBUG, 5
@@ -44,8 +44,38 @@ def setupLog(log_level, filename=None, **kw):
     logging.addLevelName(5, 'TRACE')
     logging.trace = lambda *args, **kw: logging.log(5, *args, **kw)
 
+
+class HelpFormatter(argparse.ArgumentDefaultsHelpFormatter):
+
+    def _get_help_string(self, action):
+        return super(HelpFormatter, self)._get_help_string(action) \
+            if action.default else action.help
+
+    def _split_lines(self, text, width):
+        """Preserves new lines in option descriptions"""
+        lines = []
+        for text in text.splitlines():
+            lines += textwrap.wrap(text, width)
+        return lines
+
+    def _fill_text(self, text, width, indent):
+        """Preserves new lines in other descriptions"""
+        kw = dict(width=width, initial_indent=indent, subsequent_indent=indent)
+        return '\n'.join(textwrap.fill(t, **kw) for t in text.splitlines())
+
 class ArgParser(argparse.ArgumentParser):
 
+    class _HelpFormatter(HelpFormatter):
+
+        def _format_actions_usage(self, actions, groups):
+            r = HelpFormatter._format_actions_usage(self, actions, groups)
+            if actions and actions[0].option_strings:
+                r = '[@OPTIONS_FILE] ' + r
+            return r
+
+    _ca_help = "Certificate authority (CA) file in .pem format." \
+               " Serial number defines the prefix of the network."
+
     def convert_arg_line_to_args(self, arg_line):
         arg_line = arg_line.split('#')[0].rstrip()
         if arg_line:
@@ -56,6 +86,13 @@ class ArgParser(argparse.ArgumentParser):
                 if arg.strip():
                     yield arg
 
+    def __init__(self, **kw):
+        super(ArgParser, self).__init__(formatter_class=self._HelpFormatter,
+            epilog="""Options can be read from a file. For example:
+  $ cat OPTIONS_FILE
+  ca /etc/re6stnet/ca.crt""", **kw)
+
+
 def makedirs(path):
     try:
         os.makedirs(path)

re6stnet

@@ -18,56 +18,75 @@ def ovpnArgs(optional_args, ca_path, cert_path, key_path):
 
 def getConfig():
     parser = utils.ArgParser(fromfile_prefix_chars='@',
-            description='Resilient virtual private network application')
+        description="Resilient virtual private network application.")
     _ = parser.add_argument
 
-    # General Configuration options
     _('--ip',
-            help='IP address advertised to other nodes')
-    _('--registry', required=True,
-            help="HTTP URL of the discovery peer server,"
-                 " with public host (default port: 80)")
+        help="IP address advertised to other nodes. Special values:\n"
+             "- upnp: force autoconfiguration via UPnP\n"
+             "- any: ask peers our IP\n"
+             " (default: ask peers if UPnP fails)")
+    _('--registry', required=True, metavar='URL',
+        help="Public HTTP URL of the registry, for bootstrapping.")
     _('-l', '--log', default='/var/log/re6stnet',
-            help='Path to re6stnet logs directory')
+        help="Path to the directory used for log files:\n"
+             "- re6stnet.log: log file of re6stnet itself\n"
+             "- babeld.log: log file of router\n"
+             "- <iface>.log: 1 file per spawned OpenVPN\n")
     _('-s', '--state', default='/var/lib/re6stnet',
-            help='Path to re6stnet state directory')
-    _('-v', '--verbose', default=1, type=int,
-            help='Log level of re6stnet itself')
+        help="Path to re6stnet state directory:\n"
+             "- peers.db: cache of peer addresses\n"
+             "- babeld.state: see option -S of babeld\n")
+    _('-v', '--verbose', default=1, type=int, metavar='LEVEL',
+        help="Log level of re6stnet itself. 0 disables logging."
+             " Use SIGUSR1 to reopen log."
+             " See also --babel-verb and --verb for logs of spawned processes.")
     _('-i', '--interface', action='append', dest='iface_list', default=[],
-            help='Extra interface for LAN discovery')
-
-    # Routing algorithm options
-    _('--babel-pidfile',
-            help='Specify a file to write our process id to')
-    _('--babel-verb', default=0,
-            help='Babel verbosity')
+        help="Extra interface for LAN discovery. Highly recommanded if there"
+             " are other re6st node on the same network segment.")
+
+    _ = parser.add_argument_group('routing').add_argument
+    _('--babel-pidfile', metavar='PID',
+        help="Specify a file to write our process id to"
+             " (option -I of Babel).")
+    _('--babel-verb', default=0, metavar='LEVEL',
+        help="Log level of Babel (option -d of Babel).")
     _('--hello', type=int, default=15,
-            help='Hello interval for babel, in seconds')
+        help="Hello interval in seconds, for both wired and wireless"
+             " connections. OpenVPN ping-exit option is set to 4 times the"
+             " hello interval. It takes between 3 and 4 times the"
+             " hello interval for Babel to re-establish connection with a"
+             " node for which the direct connection has been cut.")
     _('-w', '--wireless', action='store_true',
-            help='''Set all interfaces to be treated as wireless interfaces
-                    for the routing protocol''')
+        help="Assume all interfaces are wireless (option -w of Babel).")
 
-    # Tunnel options
+    _ = parser.add_argument_group('tunnelling').add_argument
     _('--encrypt', action='store_true',
-            help='specify that tunnels should be encrypted')
-    _('--pp', nargs=2, action='append',
-            help='Port and protocol to be used by other peers to connect')
+        help='Specify that tunnels should be encrypted.')
+    _('--pp', nargs=2, action='append', metavar=('PORT', 'PROTO'),
+        help="Port and protocol to be announced to other peers, ordered by"
+             " preference. For each protocol (either udp or tcp), start one"
+             " openvpn server on the first given port."
+             " (default: --pp 1194 udp --pp 1194 tcp)")
     _('--dh', required=True,
-            help='Path to dh file')
-    _('--ca', required=True,
-            help='Path to the certificate authority file')
+        help='File containing Diffie-Hellman parameters in .pem format')
+    _('--ca', required=True, help=parser._ca_help)
     _('--cert', required=True,
-            help='Path to the certificate file')
+        help="Local peer's signed certificate in .pem format."
+             " Common name defines the allocated prefix in the network.")
     _('--key', required=True,
-            help='Path to the private key file')
+        help="Local peer's private key in .pem format.")
     _('--connection-count', default=20, type=int,
-            help='Number of tunnels')
+        help="Maximum number of accepted clients per OpenVPN server."
+             " Also represents the average number of tunnels to peers.")
     _('--tunnel-refresh', default=300, type=int,
-            help='time (seconds) to wait before changing the connections')
+        help="Interval in seconds between two tunnel refresh: the worst"
+             " tunnel is closed if the number of client tunnels has reached"
+             " its maximum number (half of connection-count).")
 
-    # Openvpn options
     _('openvpn_args', nargs=argparse.REMAINDER,
-            help="Common OpenVPN options")
+        help="Use pseudo-argument '--' to forward positional arguments as extra"
+             " arguments to both server and client OpenVPN subprocesses.")
     return parser.parse_args()
 
 

setup.py

@@ -8,6 +8,7 @@ Environment :: Console
 License :: OSI Approved :: GNU General Public License (GPL)
 Natural Language :: English
 Operating System :: POSIX :: Linux
+Programming Language :: Python :: 2.6
 Programming Language :: Python :: 2.7
 Topic :: Internet
 Topic :: System :: Networking