Commit 7409e32c authored by Guillaume Bury's avatar Guillaume Bury

Fixed minor bug in db. Added a HOW TO section in re6stnet man page

parent 5f163f16
......@@ -78,24 +78,3 @@ Later....
re6st/ Forward ports Get certificates from server Server : deliver certificates and distribute peers
.. XXX: write monkey-patch in
Note: On certain version of python (e.g. 2.7.3~rc2-2.1 ) dns lookup is
performed for each request, and cause a delay in response.
To avoid this, one can either upgrade python, fix their resolv.conf or use
the fix at the end of this file.
# Fix for librpcxml to avoid doing reverse dns on each request
# it was causing a 10s delay on each request when no reverse DNS was avalaible
import BaseHTTPServer
def not_insane_address_string(self):
host, port = self.client_address[:2]
return '%s (reverse DNS disabled)' % host # used to call: socket.getfqdn(host)
BaseHTTPServer.BaseHTTPRequestHandler.address_string = not_insane_address_string
To be done :
Male sure the re6stnet ip address is added on only one interface
( tweak the ovpn-server script ), else duplicate addresses appears in routing
Use an algorithm to choose which connections to keep and/or establish
instead of pure randomness
number of routes / tunnel
README : move options doc elsewhere, make it look like a real readme
Use latency in babeld
Warn babeld about the tunnels wich are about to be deleted. Maybe we could just increase the cost.
......@@ -12,8 +12,8 @@ Configuration tool for re6stnet
``re6st-conf`` ``--server`` `server-url` ``--port`` `server-port`
[`command`] [`options`...]
``re6st-conf`` ``--server`` `server-url` ``--port`` `server-port` [`command`]
......@@ -59,13 +59,12 @@ Options
Path of a directory where will be stored the files generated by the
setup. The Setup genereates the following files, in the explicit
order :
- ca.pem : certificate authority file downloaded from the server
- peers.db : peers database initialized for
- cert.key : private key generated by the script
- cert.crt : individual certificate file generated by the server
- dh2048.pem : dh file for oenvpn server
-r, ``--req`` `name` `value`
Specify an attribute to add to the certificate request sent to the
server. Can be used multiple times.
......@@ -13,8 +13,8 @@ SYNOPSIS
``re6st-registry`` `port` ``--db`` `db-path` ``--ca`` `ca-path`
``--key`` `key-path` ``--mailhost`` `mailhost`
``--private`` `private-ip` [`options`...]
``--key`` `key-path` ``--mailhost`` `mailhost` ``--private`` `private-ip`
......@@ -12,9 +12,9 @@ Resilient, Scalable, IPv6 Network application
``re6stnet`` ``--registry`` `registry-url` ``--dh`` `dh-path`
``--ca`` `ca-path` ``--cert`` `cert-path`
``--key`` `key-path` [`options`...] [``--`` [`openvpn-options`...]]
``re6stnet`` ``--registry`` `registry-url` ``--dh`` `dh-path` ``--ca`` `ca-path`
``--cert`` `cert-path` ``--key`` `key-path` [`options`...]
[``--`` [`openvpn-options`...]]
......@@ -164,6 +164,116 @@ Openvpn-options
be passed down to ALL openvpn processes ( including servers )
exactly as they are given.
Here's an example how to deploy your re6st network.
Normal node
In most cases, you only have to start the re6stnet daemon for you to join
the re6st network. Since the number of options to set is currently quite high,
I advise you to utse a configuration file. Here is an example of such a
configuration file::
# Configuration file for re6stnet
# You have to give the complete url of the re6st-registry.
# If you have the ip address and the port of the registry, enter the url as
# following :
# registry http://ipv4:port
# registry http://[ipv6]:port
registry http://localhost:8000
# Here are information about your certificates.
# These options are mandatory.
dh dh2048.pem
ca ca.pem
cert cert.crt
key cert.key
# You can give the external configuration ( ip, port and protocol )
# advertised to other nodes. These information are used by the openvpn
# daemon to connect to your servers. If no --ip otion is given, re6stnet
# will automatically attempt to forward ports vie UPnP.
# You can give as many --ip options you want.
# ip 1194 udp
# ip 1194 tcp-client
# You can specify the directory you want the state files ( peer database,
# babel state file ), to be in. The default is :
# state /var/lib/re6stnet
# Verbose level ( default: 0 )
# 1 is a good verbose level if you want to see what's happening in re6st.
# level 2 and 3 display a whole lot of messages, so it should only be used
# as a debug tool
verbose 1
You can then start re6stnet :
``re6stnet @command_file``
First Node
First, generate the ceritifcates for your network with the following command.
For that, you have to give the address for your network, here we took an
address starting with the ipv6 example prefix `2001:db8::`, and adding a random
number to create a /48 network. Once you have decided on your network ip
address, you have to translate it into hexadecimal, and add a **1** as the
most significant digit. So the network ip address 2001:db8:42::/48 translate
into ``0x120010db80042``. Put that number as the serial umber of your
``openssl req -nodes -new -x509 -key ca.key -set_serial 0x120010db80042
-days 365 -out ca.crt``
With this, you now have a ca.crt and a ca.key file in your current directory.
Then, you have to start a re6st-registry to acquire an ipv6 address for your
first node. In order to do that, you need to run the following command.
You can give any path you like for the --db option, if the file does not
exists, it will be created. The mailhost will be used to send tokens by mail,
so you should make sure it works.
``re6st-registry port_number --db db_path --ca path_to_ca.crt
--key path_to_ca.key --mailhost yourmailhost``
You are now ready to use the re6st configuration tool to generate the
certificates for the first node of your network, i.e. you. This should do the
trick :
``re6st-conf --server localhost --port 8000``
It will generate in your local directory (you can change it with the -d option)
four files (ca.crt, cert.crt, cert.key, dh2048.pem).
Now here's the tricky part. For your network to work, you need to restart the
registry (maybe it will be fixed one day...), this time with more information
than the last time. You need to get your hands on the individual prefix of your
node, and the re6st ipv6 address associated. These should have been printed
at the end of re6st-conf. If you have missed them, for one reason or another,
you can get them in the python interpreter::
>>> from re6st import utils
>>> network = utils.networkFromCa('ca.pem')
>>> re6st_ip, prefix = utils.ipFromCert(network, 'cert.crt')
>>> print re6st_ip
>>> print prefix
Now you can restart your re6st-registry with two more options:
``re6st-registry port_number --db db_path --ca path_to_ca.crt
--key path_to_ca.key --mailhost yourmailhost --private 2001:db8:42:3::1
--bootstrap 0000000000000011``
Finally, you can start your own re6st node following the instrucxtions in the
precedent section.
......@@ -19,13 +19,16 @@ def main():
_('--email', help='Your email address')
_('--token', help='The token you received')
config = parser.parse_args()
ca_path = os.path.join(config.dir, 'ca.pem')
cert_path = os.path.join(config.dir, 'cert.crt')
key_path = os.path.join(config.dir, 'cert.key')
# Establish connection with server
s = xmlrpclib.ServerProxy('http://%s:%u' % (config.server, config.port))
# Get CA
ca = s.getCa()
with open(os.path.join(config.dir, 'ca.pem'), 'w') as f:
with open(ca_path, 'w') as f:
if config.ca_only:
......@@ -56,9 +59,9 @@ def main():
cert = s.requestCertificate(config.token, req)
# Store cert and key
with open(os.path.join(config.dir, 'cert.key'), 'w') as f:
with open(key_path, 'w') as f:
with open(os.path.join(config.dir, 'cert.crt'), 'w') as f:
with open(cert_path, 'w') as f:
# Generating dh file
......@@ -67,5 +70,10 @@ def main():
print "Certificate setup complete."
network = utils.networkFromCa(ca_path)
internal_ip, prefix = utils.ipFromCert(network, cert_path)
print "Your re6st ip : %s" % internal_ip
print "Your prefix : %s" % prefix
if __name__ == "__main__":
......@@ -23,6 +23,10 @@ class SimpleXMLRPCServer4(SimpleXMLRPCServer):
allow_reuse_address = True
def address_string(self):
# Workaround for
return self.client_address[0]
class SimpleXMLRPCServer6(SimpleXMLRPCServer4):
......@@ -59,10 +63,15 @@ class main(object):
_('--bootstrap', action="append",
help='''VPN prefix of the peers to send as bootstrap peer,
instead of random ones''')
_('--private', required=True,
help='VPN IP of the node on which runs the registry')
self.config = parser.parse_args()
if not self.config.private:
logging.warning('You have declared no private address'
', either this is the first start, or you should'
'check you configuration')
# Database initializing
self.db = sqlite3.connect(self.config.db, isolation_level=None)
self.db.execute("""CREATE TABLE IF NOT EXISTS peers (
......@@ -27,17 +27,18 @@ class PeerManager:
date INTEGER DEFAULT (strftime('%s', 'now')))""")
self._db.execute("UPDATE peers SET used = 0")
self._db.execute("CREATE INDEX IF NOT EXISTS _peers_used ON peers(used)")
self._db.execute("""CREATE TABLE IF NOT EXISTS blacklist (
self._db.execute("""CREATE INDEX IF NOT EXISTS
blacklist_flag ON blacklist(flag)""")
self._db.execute("INSERT OR REPLACE INTO blacklist VALUES (?,?)",
(prefix, 1))
self._db.execute("CREATE INDEX IF NOT EXISTS
_peers_used ON peers(used)")
self._db.execute("""CREATE TABLE IF NOT EXISTS config (
name text primary key,
value text)""")
self._db.execute('ATTACH DATABASE ":memory:" AS blacklist')
self._db.execute("""CREATE TABLE blacklist.flag (
self._db.execute("""CREATE INDEX blacklist.blacklist_flag
ON flag(flag)""")
self._db.execute("INSERT INTO blacklist.flag VALUES (?,?)", (prefix, 1))
a, = self._db.execute("SELECT value FROM config WHERE name='registry'").next()
except StopIteration:
......@@ -51,20 +52,20 @@ class PeerManager:
def clear_blacklist(self, flag):'Clearing blacklist from flag %u' % flag)
self._db.execute("DELETE FROM blacklist WHERE flag = ?",
self._db.execute("DELETE FROM blacklist.flag WHERE flag = ?",
(flag,))'Blacklist cleared')
def blacklist(self, prefix, flag):
logging.ninfo('Blacklisting %s' % prefix)
self._db.execute("DELETE FROM peers WHERE prefix = ?", (prefix,))
self._db.execute("INSERT OR REPLACE INTO blacklist VALUES (?,?)",
self._db.execute("INSERT OR REPLACE INTO blacklist.flag VALUES (?,?)",
(prefix, flag))
logging.debug('%s blacklisted' % prefix)
def whitelist(self, prefix):'Unblacklisting %s' % prefix)
self._db.execute("DELETE FROM blacklist WHERE prefix = ?", (prefix,))
self._db.execute("DELETE FROM blacklist.flag WHERE prefix = ?", (prefix,))
logging.debug('%s whitelisted' % prefix)
def refresh(self):
......@@ -102,7 +103,7 @@ class PeerManager:
self._db.executemany("""INSERT OR IGNORE INTO peers (prefix, address)
VALUES (?,?)""", new_peer_list)
self._db.execute("""DELETE FROM peers WHERE prefix IN
(SELECT prefix FROM blacklist)""")
(SELECT prefix FROM blacklist.flag)""")'DB populated')
logging.trace('New peers : %s' % (', '.join(map(str, new_peer_list)),))
......@@ -111,8 +112,11 @@ class PeerManager:
peer_list = self._db.execute("""SELECT prefix, address FROM peers WHERE used
<= 0 ORDER BY used DESC,RANDOM() LIMIT ?""",
if peer_list or populate():
if peer_list:
return peer_list
logging.warning('Cannot find any new peers')
return []
def _bootstrap(self):'Getting Boot peer...')
......@@ -123,9 +127,10 @@ class PeerManager:
p = subprocess.Popen(('openssl', 'rsautl', '-decrypt', '-inkey', self._key_path),
stdin=subprocess.PIPE, stdout=subprocess.PIPE)
bootpeer = p.communicate(bootpeer)[0].split()
self._db.execute("INSERT INTO peers (prefix, address) VALUES (?,?)", bootpeer)
logging.debug('Boot peer added')
return True
if bootpeer[0] != self._prefix:
self._db.execute("INSERT INTO peers (prefix, address) VALUES (?,?)", bootpeer)
logging.debug('Boot peer added')
return True
except socket.error:
except sqlite3.IntegrityError, e:
......@@ -88,7 +88,7 @@ class TunnelManager:
self._client_count = (connection_count + 1) // 2
self._refresh_count = refresh_count
self.free_interface_set = set('client' + str(i)
self.free_interface_set = set('re6stnet' + str(i)
for i in xrange(1, self._client_count + 1))
def refresh(self):
......@@ -3,7 +3,6 @@ from OpenSSL import crypto
logging_levels = logging.WARNING, logging.INFO, logging.DEBUG, 5
def setupLog(log_level):
format='%(asctime)s : %(message)s',
......@@ -11,12 +10,10 @@ def setupLog(log_level):
logging.addLevelName(5, 'TRACE')
logging.trace = lambda *args, **kw: logging.log(5, *args, **kw)
def binFromIp(ip):
ip1, ip2 = struct.unpack('>QQ', socket.inet_pton(socket.AF_INET6, ip))
return bin(ip1)[2:].rjust(64, '0') + bin(ip2)[2:].rjust(64, '0')
def ipFromBin(prefix):
prefix = hex(int(prefix, 2))[2:]
ip = ''
......@@ -24,20 +21,17 @@ def ipFromBin(prefix):
ip += prefix[i:i + 4] + ':'
return ip.rstrip(':')
def ipFromPrefix(re6stnet, prefix, prefix_len):
prefix = bin(int(prefix))[2:].rjust(prefix_len, '0')
ip_t = (re6stnet + prefix).ljust(127, '0').ljust(128, '1')
return ipFromBin(ip_t), prefix
def networkFromCa(ca_path):
# Get network prefix from ca.crt
with open(ca_path, 'r') as f:
ca = crypto.load_certificate(crypto.FILETYPE_PEM,
return bin(ca.get_serial_number())[3:]
def ipFromCert(network, cert_path):
# Get ip from cert.crt
with open(cert_path, 'r') as f:
......@@ -46,10 +40,8 @@ def ipFromCert(network, cert_path):
prefix, prefix_len = subject.CN.split('/')
return ipFromPrefix(network, prefix, int(prefix_len))
def address_str(address_set):
return ';'.join(map(','.join, address_set))
def address_str(address):
return ';'.join(map(','.join, address))
def address_list(address_list):
return list(tuple(address.split(','))
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment