Commit 7409e32c authored by Guillaume Bury's avatar Guillaume Bury

Fixed minor bug in db. Added a HOW TO section in re6stnet man page

parent 5f163f16
...@@ -78,24 +78,3 @@ Later.... ...@@ -78,24 +78,3 @@ Later....
re6st/upnpigd.py Forward ports re6st/upnpigd.py Forward ports
re6st-conf.py Get certificates from server re6st-conf.py Get certificates from server
re6st-registry.py Server : deliver certificates and distribute peers re6st-registry.py Server : deliver certificates and distribute peers
.. XXX: write monkey-patch in __init__.py
Note: On certain version of python (e.g. 2.7.3~rc2-2.1 ) dns lookup is
performed for each request, and cause a delay in response.
To avoid this, one can either upgrade python, fix their resolv.conf or use
the fix at the end of this file.
--------------------------------------------------------------------------------
# Fix for librpcxml to avoid doing reverse dns on each request
# it was causing a 10s delay on each request when no reverse DNS was avalaible
import BaseHTTPServer
def not_insane_address_string(self):
host, port = self.client_address[:2]
return '%s (reverse DNS disabled)' % host # used to call: socket.getfqdn(host)
BaseHTTPServer.BaseHTTPRequestHandler.address_string = not_insane_address_string
--------------------------------------------------------------------------------
To be done : To be done :
Male sure the re6stnet ip address is added on only one interface
( tweak the ovpn-server script ), else duplicate addresses appears in routing
tables.
Use an algorithm to choose which connections to keep and/or establish Use an algorithm to choose which connections to keep and/or establish
instead of pure randomness instead of pure randomness
number of routes / tunnel number of routes / tunnel
README : move options doc elsewhere, make it look like a real readme
Use latency in babeld
Warn babeld about the tunnels wich are about to be deleted. Maybe we could just increase the cost. Warn babeld about the tunnels wich are about to be deleted. Maybe we could just increase the cost.
...@@ -12,8 +12,8 @@ Configuration tool for re6stnet ...@@ -12,8 +12,8 @@ Configuration tool for re6stnet
SYNOPSIS SYNOPSIS
======== ========
``re6st-conf`` ``--server`` `server-url` ``--port`` `server-port` ``re6st-conf`` ``--server`` `server-url` ``--port`` `server-port` [`command`]
[`command`] [`options`...] [`options`...]
DESCRIPTION DESCRIPTION
=========== ===========
...@@ -59,13 +59,12 @@ Options ...@@ -59,13 +59,12 @@ Options
Path of a directory where will be stored the files generated by the Path of a directory where will be stored the files generated by the
setup. The Setup genereates the following files, in the explicit setup. The Setup genereates the following files, in the explicit
order : order :
- ca.pem : certificate authority file downloaded from the server - ca.pem : certificate authority file downloaded from the server
- peers.db : peers database initialized for re6stnet.py
- cert.key : private key generated by the script - cert.key : private key generated by the script
- cert.crt : individual certificate file generated by the server - cert.crt : individual certificate file generated by the server
- dh2048.pem : dh file for oenvpn server - dh2048.pem : dh file for oenvpn server
-r, ``--req`` `name` `value` -r, ``--req`` `name` `value`
Specify an attribute to add to the certificate request sent to the Specify an attribute to add to the certificate request sent to the
server. Can be used multiple times. server. Can be used multiple times.
......
...@@ -13,8 +13,8 @@ SYNOPSIS ...@@ -13,8 +13,8 @@ SYNOPSIS
======== ========
``re6st-registry`` `port` ``--db`` `db-path` ``--ca`` `ca-path` ``re6st-registry`` `port` ``--db`` `db-path` ``--ca`` `ca-path`
``--key`` `key-path` ``--mailhost`` `mailhost` ``--key`` `key-path` ``--mailhost`` `mailhost` ``--private`` `private-ip`
``--private`` `private-ip` [`options`...] [`options`...]
DESCRIPTION DESCRIPTION
=========== ===========
......
...@@ -12,9 +12,9 @@ Resilient, Scalable, IPv6 Network application ...@@ -12,9 +12,9 @@ Resilient, Scalable, IPv6 Network application
SYNOPSIS SYNOPSIS
======== ========
``re6stnet`` ``--registry`` `registry-url` ``--dh`` `dh-path` ``re6stnet`` ``--registry`` `registry-url` ``--dh`` `dh-path` ``--ca`` `ca-path`
``--ca`` `ca-path` ``--cert`` `cert-path` ``--cert`` `cert-path` ``--key`` `key-path` [`options`...]
``--key`` `key-path` [`options`...] [``--`` [`openvpn-options`...]] [``--`` [`openvpn-options`...]]
DESCRIPTION DESCRIPTION
=========== ===========
...@@ -164,6 +164,116 @@ Openvpn-options ...@@ -164,6 +164,116 @@ Openvpn-options
be passed down to ALL openvpn processes ( including servers ) be passed down to ALL openvpn processes ( including servers )
exactly as they are given. exactly as they are given.
HOW TO
======
Here's an example how to deploy your re6st network.
Normal node
-----------
In most cases, you only have to start the re6stnet daemon for you to join
the re6st network. Since the number of options to set is currently quite high,
I advise you to utse a configuration file. Here is an example of such a
configuration file::
# Configuration file for re6stnet
# You have to give the complete url of the re6st-registry.
# If you have the ip address and the port of the registry, enter the url as
# following :
# registry http://ipv4:port
# registry http://[ipv6]:port
registry http://localhost:8000
# Here are information about your certificates.
# These options are mandatory.
dh dh2048.pem
ca ca.pem
cert cert.crt
key cert.key
# You can give the external configuration ( ip, port and protocol )
# advertised to other nodes. These information are used by the openvpn
# daemon to connect to your servers. If no --ip otion is given, re6stnet
# will automatically attempt to forward ports vie UPnP.
# You can give as many --ip options you want.
# ip 192.0.2.130 1194 udp
# ip 192.0.2.130 1194 tcp-client
# You can specify the directory you want the state files ( peer database,
# babel state file ), to be in. The default is :
# state /var/lib/re6stnet
# Verbose level ( default: 0 )
# 1 is a good verbose level if you want to see what's happening in re6st.
# level 2 and 3 display a whole lot of messages, so it should only be used
# as a debug tool
verbose 1
You can then start re6stnet :
``re6stnet @command_file``
First Node
----------
First, generate the ceritifcates for your network with the following command.
For that, you have to give the address for your network, here we took an
address starting with the ipv6 example prefix `2001:db8::`, and adding a random
number to create a /48 network. Once you have decided on your network ip
address, you have to translate it into hexadecimal, and add a **1** as the
most significant digit. So the network ip address 2001:db8:42::/48 translate
into ``0x120010db80042``. Put that number as the serial umber of your
certificate.
``openssl req -nodes -new -x509 -key ca.key -set_serial 0x120010db80042
-days 365 -out ca.crt``
With this, you now have a ca.crt and a ca.key file in your current directory.
Then, you have to start a re6st-registry to acquire an ipv6 address for your
first node. In order to do that, you need to run the following command.
You can give any path you like for the --db option, if the file does not
exists, it will be created. The mailhost will be used to send tokens by mail,
so you should make sure it works.
``re6st-registry port_number --db db_path --ca path_to_ca.crt
--key path_to_ca.key --mailhost yourmailhost``
You are now ready to use the re6st configuration tool to generate the
certificates for the first node of your network, i.e. you. This should do the
trick :
``re6st-conf --server localhost --port 8000``
It will generate in your local directory (you can change it with the -d option)
four files (ca.crt, cert.crt, cert.key, dh2048.pem).
Now here's the tricky part. For your network to work, you need to restart the
registry (maybe it will be fixed one day...), this time with more information
than the last time. You need to get your hands on the individual prefix of your
node, and the re6st ipv6 address associated. These should have been printed
at the end of re6st-conf. If you have missed them, for one reason or another,
you can get them in the python interpreter::
>>> from re6st import utils
>>> network = utils.networkFromCa('ca.pem')
>>> re6st_ip, prefix = utils.ipFromCert(network, 'cert.crt')
>>> print re6st_ip
2001:0db8:0042:0003:0000:0000:0000:0001
>>> print prefix
0000000000000011
Now you can restart your re6st-registry with two more options:
``re6st-registry port_number --db db_path --ca path_to_ca.crt
--key path_to_ca.key --mailhost yourmailhost --private 2001:db8:42:3::1
--bootstrap 0000000000000011``
Finally, you can start your own re6st node following the instrucxtions in the
precedent section.
SEE ALSO SEE ALSO
======== ========
......
...@@ -19,13 +19,16 @@ def main(): ...@@ -19,13 +19,16 @@ def main():
_('--email', help='Your email address') _('--email', help='Your email address')
_('--token', help='The token you received') _('--token', help='The token you received')
config = parser.parse_args() config = parser.parse_args()
ca_path = os.path.join(config.dir, 'ca.pem')
cert_path = os.path.join(config.dir, 'cert.crt')
key_path = os.path.join(config.dir, 'cert.key')
# Establish connection with server # Establish connection with server
s = xmlrpclib.ServerProxy('http://%s:%u' % (config.server, config.port)) s = xmlrpclib.ServerProxy('http://%s:%u' % (config.server, config.port))
# Get CA # Get CA
ca = s.getCa() ca = s.getCa()
with open(os.path.join(config.dir, 'ca.pem'), 'w') as f: with open(ca_path, 'w') as f:
f.write(ca) f.write(ca)
if config.ca_only: if config.ca_only:
...@@ -56,9 +59,9 @@ def main(): ...@@ -56,9 +59,9 @@ def main():
cert = s.requestCertificate(config.token, req) cert = s.requestCertificate(config.token, req)
# Store cert and key # Store cert and key
with open(os.path.join(config.dir, 'cert.key'), 'w') as f: with open(key_path, 'w') as f:
f.write(key) f.write(key)
with open(os.path.join(config.dir, 'cert.crt'), 'w') as f: with open(cert_path, 'w') as f:
f.write(cert) f.write(cert)
# Generating dh file # Generating dh file
...@@ -67,5 +70,10 @@ def main(): ...@@ -67,5 +70,10 @@ def main():
print "Certificate setup complete." print "Certificate setup complete."
network = utils.networkFromCa(ca_path)
internal_ip, prefix = utils.ipFromCert(network, cert_path)
print "Your re6st ip : %s" % internal_ip
print "Your prefix : %s" % prefix
if __name__ == "__main__": if __name__ == "__main__":
main() main()
...@@ -23,6 +23,10 @@ class SimpleXMLRPCServer4(SimpleXMLRPCServer): ...@@ -23,6 +23,10 @@ class SimpleXMLRPCServer4(SimpleXMLRPCServer):
allow_reuse_address = True allow_reuse_address = True
def address_string(self):
# Workaround for http://bugs.python.org/issue6085
return self.client_address[0]
class SimpleXMLRPCServer6(SimpleXMLRPCServer4): class SimpleXMLRPCServer6(SimpleXMLRPCServer4):
...@@ -59,10 +63,15 @@ class main(object): ...@@ -59,10 +63,15 @@ class main(object):
_('--bootstrap', action="append", _('--bootstrap', action="append",
help='''VPN prefix of the peers to send as bootstrap peer, help='''VPN prefix of the peers to send as bootstrap peer,
instead of random ones''') instead of random ones''')
_('--private', required=True, _('--private',
help='VPN IP of the node on which runs the registry') help='VPN IP of the node on which runs the registry')
self.config = parser.parse_args() self.config = parser.parse_args()
if not self.config.private:
logging.warning('You have declared no private address'
', either this is the first start, or you should'
'check you configuration')
# Database initializing # Database initializing
self.db = sqlite3.connect(self.config.db, isolation_level=None) self.db = sqlite3.connect(self.config.db, isolation_level=None)
self.db.execute("""CREATE TABLE IF NOT EXISTS peers ( self.db.execute("""CREATE TABLE IF NOT EXISTS peers (
......
...@@ -27,17 +27,18 @@ class PeerManager: ...@@ -27,17 +27,18 @@ class PeerManager:
used INTEGER NOT NULL DEFAULT 0, used INTEGER NOT NULL DEFAULT 0,
date INTEGER DEFAULT (strftime('%s', 'now')))""") date INTEGER DEFAULT (strftime('%s', 'now')))""")
self._db.execute("UPDATE peers SET used = 0") self._db.execute("UPDATE peers SET used = 0")
self._db.execute("CREATE INDEX IF NOT EXISTS _peers_used ON peers(used)") self._db.execute("CREATE INDEX IF NOT EXISTS
self._db.execute("""CREATE TABLE IF NOT EXISTS blacklist ( _peers_used ON peers(used)")
prefix TEXT PRIMARY KEY,
flag INTEGER NOT NULL)""")
self._db.execute("""CREATE INDEX IF NOT EXISTS
blacklist_flag ON blacklist(flag)""")
self._db.execute("INSERT OR REPLACE INTO blacklist VALUES (?,?)",
(prefix, 1))
self._db.execute("""CREATE TABLE IF NOT EXISTS config ( self._db.execute("""CREATE TABLE IF NOT EXISTS config (
name text primary key, name text primary key,
value text)""") value text)""")
self._db.execute('ATTACH DATABASE ":memory:" AS blacklist')
self._db.execute("""CREATE TABLE blacklist.flag (
prefix TEXT PRIMARY KEY,
flag INTEGER NOT NULL)""")
self._db.execute("""CREATE INDEX blacklist.blacklist_flag
ON flag(flag)""")
self._db.execute("INSERT INTO blacklist.flag VALUES (?,?)", (prefix, 1))
try: try:
a, = self._db.execute("SELECT value FROM config WHERE name='registry'").next() a, = self._db.execute("SELECT value FROM config WHERE name='registry'").next()
except StopIteration: except StopIteration:
...@@ -51,20 +52,20 @@ class PeerManager: ...@@ -51,20 +52,20 @@ class PeerManager:
def clear_blacklist(self, flag): def clear_blacklist(self, flag):
logging.info('Clearing blacklist from flag %u' % flag) logging.info('Clearing blacklist from flag %u' % flag)
self._db.execute("DELETE FROM blacklist WHERE flag = ?", self._db.execute("DELETE FROM blacklist.flag WHERE flag = ?",
(flag,)) (flag,))
logging.info('Blacklist cleared') logging.info('Blacklist cleared')
def blacklist(self, prefix, flag): def blacklist(self, prefix, flag):
logging.ninfo('Blacklisting %s' % prefix) logging.ninfo('Blacklisting %s' % prefix)
self._db.execute("DELETE FROM peers WHERE prefix = ?", (prefix,)) self._db.execute("DELETE FROM peers WHERE prefix = ?", (prefix,))
self._db.execute("INSERT OR REPLACE INTO blacklist VALUES (?,?)", self._db.execute("INSERT OR REPLACE INTO blacklist.flag VALUES (?,?)",
(prefix, flag)) (prefix, flag))
logging.debug('%s blacklisted' % prefix) logging.debug('%s blacklisted' % prefix)
def whitelist(self, prefix): def whitelist(self, prefix):
logging.info('Unblacklisting %s' % prefix) logging.info('Unblacklisting %s' % prefix)
self._db.execute("DELETE FROM blacklist WHERE prefix = ?", (prefix,)) self._db.execute("DELETE FROM blacklist.flag WHERE prefix = ?", (prefix,))
logging.debug('%s whitelisted' % prefix) logging.debug('%s whitelisted' % prefix)
def refresh(self): def refresh(self):
...@@ -102,7 +103,7 @@ class PeerManager: ...@@ -102,7 +103,7 @@ class PeerManager:
self._db.executemany("""INSERT OR IGNORE INTO peers (prefix, address) self._db.executemany("""INSERT OR IGNORE INTO peers (prefix, address)
VALUES (?,?)""", new_peer_list) VALUES (?,?)""", new_peer_list)
self._db.execute("""DELETE FROM peers WHERE prefix IN self._db.execute("""DELETE FROM peers WHERE prefix IN
(SELECT prefix FROM blacklist)""") (SELECT prefix FROM blacklist.flag)""")
logging.info('DB populated') logging.info('DB populated')
logging.trace('New peers : %s' % (', '.join(map(str, new_peer_list)),)) logging.trace('New peers : %s' % (', '.join(map(str, new_peer_list)),))
...@@ -111,8 +112,11 @@ class PeerManager: ...@@ -111,8 +112,11 @@ class PeerManager:
peer_list = self._db.execute("""SELECT prefix, address FROM peers WHERE used peer_list = self._db.execute("""SELECT prefix, address FROM peers WHERE used
<= 0 ORDER BY used DESC,RANDOM() LIMIT ?""", <= 0 ORDER BY used DESC,RANDOM() LIMIT ?""",
(peer_count,)).fetchall() (peer_count,)).fetchall()
if peer_list or populate(): if peer_list:
return peer_list return peer_list
populate()
logging.warning('Cannot find any new peers')
return []
def _bootstrap(self): def _bootstrap(self):
logging.info('Getting Boot peer...') logging.info('Getting Boot peer...')
...@@ -123,9 +127,10 @@ class PeerManager: ...@@ -123,9 +127,10 @@ class PeerManager:
p = subprocess.Popen(('openssl', 'rsautl', '-decrypt', '-inkey', self._key_path), p = subprocess.Popen(('openssl', 'rsautl', '-decrypt', '-inkey', self._key_path),
stdin=subprocess.PIPE, stdout=subprocess.PIPE) stdin=subprocess.PIPE, stdout=subprocess.PIPE)
bootpeer = p.communicate(bootpeer)[0].split() bootpeer = p.communicate(bootpeer)[0].split()
self._db.execute("INSERT INTO peers (prefix, address) VALUES (?,?)", bootpeer) if bootpeer[0] != self._prefix:
logging.debug('Boot peer added') self._db.execute("INSERT INTO peers (prefix, address) VALUES (?,?)", bootpeer)
return True logging.debug('Boot peer added')
return True
except socket.error: except socket.error:
pass pass
except sqlite3.IntegrityError, e: except sqlite3.IntegrityError, e:
......
...@@ -88,7 +88,7 @@ class TunnelManager: ...@@ -88,7 +88,7 @@ class TunnelManager:
self._client_count = (connection_count + 1) // 2 self._client_count = (connection_count + 1) // 2
self._refresh_count = refresh_count self._refresh_count = refresh_count
self.free_interface_set = set('client' + str(i) self.free_interface_set = set('re6stnet' + str(i)
for i in xrange(1, self._client_count + 1)) for i in xrange(1, self._client_count + 1))
def refresh(self): def refresh(self):
......
...@@ -3,7 +3,6 @@ from OpenSSL import crypto ...@@ -3,7 +3,6 @@ from OpenSSL import crypto
logging_levels = logging.WARNING, logging.INFO, logging.DEBUG, 5 logging_levels = logging.WARNING, logging.INFO, logging.DEBUG, 5
def setupLog(log_level): def setupLog(log_level):
logging.basicConfig(level=logging_levels[log_level], logging.basicConfig(level=logging_levels[log_level],
format='%(asctime)s : %(message)s', format='%(asctime)s : %(message)s',
...@@ -11,12 +10,10 @@ def setupLog(log_level): ...@@ -11,12 +10,10 @@ def setupLog(log_level):
logging.addLevelName(5, 'TRACE') logging.addLevelName(5, 'TRACE')
logging.trace = lambda *args, **kw: logging.log(5, *args, **kw) logging.trace = lambda *args, **kw: logging.log(5, *args, **kw)
def binFromIp(ip): def binFromIp(ip):
ip1, ip2 = struct.unpack('>QQ', socket.inet_pton(socket.AF_INET6, ip)) ip1, ip2 = struct.unpack('>QQ', socket.inet_pton(socket.AF_INET6, ip))
return bin(ip1)[2:].rjust(64, '0') + bin(ip2)[2:].rjust(64, '0') return bin(ip1)[2:].rjust(64, '0') + bin(ip2)[2:].rjust(64, '0')
def ipFromBin(prefix): def ipFromBin(prefix):
prefix = hex(int(prefix, 2))[2:] prefix = hex(int(prefix, 2))[2:]
ip = '' ip = ''
...@@ -24,20 +21,17 @@ def ipFromBin(prefix): ...@@ -24,20 +21,17 @@ def ipFromBin(prefix):
ip += prefix[i:i + 4] + ':' ip += prefix[i:i + 4] + ':'
return ip.rstrip(':') return ip.rstrip(':')
def ipFromPrefix(re6stnet, prefix, prefix_len): def ipFromPrefix(re6stnet, prefix, prefix_len):
prefix = bin(int(prefix))[2:].rjust(prefix_len, '0') prefix = bin(int(prefix))[2:].rjust(prefix_len, '0')
ip_t = (re6stnet + prefix).ljust(127, '0').ljust(128, '1') ip_t = (re6stnet + prefix).ljust(127, '0').ljust(128, '1')
return ipFromBin(ip_t), prefix return ipFromBin(ip_t), prefix
def networkFromCa(ca_path): def networkFromCa(ca_path):
# Get network prefix from ca.crt # Get network prefix from ca.crt
with open(ca_path, 'r') as f: with open(ca_path, 'r') as f:
ca = crypto.load_certificate(crypto.FILETYPE_PEM, f.read()) ca = crypto.load_certificate(crypto.FILETYPE_PEM, f.read())
return bin(ca.get_serial_number())[3:] return bin(ca.get_serial_number())[3:]
def ipFromCert(network, cert_path): def ipFromCert(network, cert_path):
# Get ip from cert.crt # Get ip from cert.crt
with open(cert_path, 'r') as f: with open(cert_path, 'r') as f:
...@@ -46,10 +40,8 @@ def ipFromCert(network, cert_path): ...@@ -46,10 +40,8 @@ def ipFromCert(network, cert_path):
prefix, prefix_len = subject.CN.split('/') prefix, prefix_len = subject.CN.split('/')
return ipFromPrefix(network, prefix, int(prefix_len)) return ipFromPrefix(network, prefix, int(prefix_len))
def address_str(address):
def address_str(address_set): return ';'.join(map(','.join, address))
return ';'.join(map(','.join, address_set))
def address_list(address_list): def address_list(address_list):
return list(tuple(address.split(',')) return list(tuple(address.split(','))
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment