README 3.49 KB
Newer Older
Vincent Pelletier's avatar
Vincent Pelletier committed
1 2
NEO is a distributed, redundant and scalable implementation of ZODB API.
NEO stands for Nexedi Enterprise Object.
Yoshinori Okuji's avatar
Yoshinori Okuji committed
3

4 5
Requirements

Vincent Pelletier's avatar
Vincent Pelletier committed
6 7
    - Linux 2.6 or later

8 9
    - Python 2.4 or later

10 11
    - For python 2.4: ctypes http://python.net/crew/theller/ctypes/
      (packaged with later python versions)
12 13 14

    - MySQLdb http://sourceforge.net/projects/mysql-python

15
    - ZODB 3.10 or later (requires python 2.5 or later, only for client nodes)
16

Vincent Pelletier's avatar
Vincent Pelletier committed
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
Overview

    A NEO cluster is composed of the following types of nodes:

    - "master" nodes (mandatory, 1 or more)
       Take care of transactionality. Only one master node is really active
       (the active master node is called "primary master") at any given time,
       extra masters are spares (they are called "secondary masters").

    - "storage" nodes (mandatory, 1 or more)
       Stores data in a MySQL database. All available storage nodes are in use
       simultaneously. This offers redundancy and data distribution.

    - "admin" nodes (mandatory for startup, optional after)
       Accepts commands from neoctl tool and transmits them to the
       primary master, and monitors cluster state.

    - "client" nodes
       Well... Something needing to store/load data in a NEO cluster. 

Disclaimer

    In addition of the disclaimer contained in the licence this code is
    released under, please consider the following.

    NEO does not implement any authentication mechanism between its nodes, and
    does not encrypts data exchanged between nodes either.
    If you want to protect your cluster from malicious nodes, or your data from
    being snooped, please consider encrypted tunelling (such as openvpn).

Aurel's avatar
Aurel committed
47 48
Installation

Vincent Pelletier's avatar
Vincent Pelletier committed
49 50 51
    a. Make neo directory available for python to import (for example, by
       adding its container directory to the PYTHONPATH environment variable).

52
    b. Choose a cluster name and setup a MySQL database
Vincent Pelletier's avatar
Vincent Pelletier committed
53 54 55

    c. Start all required nodes :

56 57 58
       neomaster --cluster=<cluster name>
       neostorage --cluster=<cluster name> --database=user:passwd@host
       neoadmin --cluster=<cluster name>
Vincent Pelletier's avatar
Vincent Pelletier committed
59 60 61

    d. Tell the cluster it can provide service.

62
       neoctl start
Vincent Pelletier's avatar
Vincent Pelletier committed
63 64 65

How to use

Aurel's avatar
Aurel committed
66 67 68 69
1. In zope:

   a. Copy neo directory to /path/to/your/zope/lib/python

Vincent Pelletier's avatar
Vincent Pelletier committed
70 71 72
   b. Edit your zope.conf, add a neo import and edit the `zodb_db` section by
      replacing its filestorage subsection by a NEOStorage one.
      It should look like :
Aurel's avatar
Aurel committed
73 74 75 76 77

      %import neo
      <zodb_db main>
          # Main FileStorage database
          <NEOStorage>
78
                master_nodes 127.0.0.1:10000
79
                name main
Aurel's avatar
Aurel committed
80 81 82 83 84 85 86 87 88 89
          </NEOStorage>
          mount-point /
      </zodb_db>

   c. Start zope

2. In a python script:

   a. Set your PYTHONPATH in order to be able to import Zope lib:

Vincent Pelletier's avatar
Vincent Pelletier committed
90
      $ export PYTHONPATH=/path/to/your/zope/lib/python:$PYTHONPATH
Aurel's avatar
Aurel committed
91 92 93

   b. Just create the storage object and play with it:

Vincent Pelletier's avatar
Vincent Pelletier committed
94 95 96 97 98 99
      from neo.client.Storare import Storage
          s = Storage(master_nodes="127.0.0.1:10010", name="main")
      ...

      "name" and "master_nodes" parameters have the same meaning as in
      configuration file.
Grégory Wisniewski's avatar
Grégory Wisniewski committed
100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123

Deployment:

NEO has no built-in deployment features such as process daemonization. We use
supervisor with configuration like below:

[group:neo]
programs=master_01,storage_01,admin

[program:master_01]
priority=1
command=neomaster -c neo -s master_01 -f /neo/neo.conf
user=neo

[program:storage_01]
priority=2
command=neostorage -c neo -s storage_01 -f /neo/neo.conf
user=neo

[program:admin]
priority=3
command=neoadmin -c neo -s admin -f /neo/neo.conf
user=neo