Commit 185dffa0 authored by Julien Muchembled's avatar Julien Muchembled

Full review of packaging, with multi-eggs setup.py

- multi-eggs setup.py because we really don't want to split the repository
  in several ones
- move 'components.xml' since namespace packages should not contain anything
- workaround python-mock.sf.net because not eggified:
  automatically get a copy of mock.py (license: BSD) when setting up 'neotests'

git-svn-id: https://svn.erp5.org/repos/neo/trunk@2750 71dcc9de-d417-0410-9af5-da40c76e7ee4
parent f78f7339
graft tools
include neoadmin neoctl neomaster neomigrate neostorage neo.conf README TODO
global-exclude *~
......@@ -9,6 +9,8 @@ Requirements
- For python 2.4: ctypes http://python.net/crew/theller/ctypes/
(packaged with later python versions)
Note that setup.py does not define any dependency to 'ctypes' so you will
have to install it explicitely.
- MySQLdb http://sourceforge.net/projects/mysql-python
......@@ -71,7 +73,7 @@ How to use
replacing its filestorage subsection by a NEOStorage one.
It should look like :
%import neo
%import neo.client
<zodb_db main>
# Main FileStorage database
<NEOStorage>
......
##########################################
########### HOW TO INSTALL NEO ###########
##########################################
###########
## Intro ##
###########
Welcome in the neo installation tutorial. This will learn to you how to install
and configure neo on your own computer.
There's currently three types of possible installations
- Quick installation from buildout (1)
- Installation from source (3 then 1)
#############
## Summary ##
#############
0. Requirements
1. Quick installation
2. Running tests
3. Building neo eggs
4. Common errors
######################
### 0.Requirements ###
######################
In order to install neo services on your computer, you'll need the following
requirements :
- Linux 2.6 or higher ( This tutorial has been tested for Mandriva 2010 )
- python2.4 or higher
- ctypes (http://python.net/crew/theller/ctypes/) (If you're using python2.5 or
higher, ctypes is directly included with python)
- mock (http://python-mock.sourceforge.net/) (Only if you plan to make the
neo tests)
############################
### 1.Quick installation ###
############################
You're a neo-newbie ? You don't need neo sources ? You want a full version and
not just one or two modules ? Or, more commonly, you're in a hurry ? So type
the following commands :
$ svn checkout https//svn.erp5.org/repos/neo/trunk/buildout/software-profiles/neo.cfg
When your bootstrap's done, check that you've got bin/buildout script. Then
Edit your buildout.cfg file, and delete all the neo-eggs which corresponds
to the parts you don't need. Then run :
$ cd neo/trunk/eggs
$ ./makeeggs
$ cd ../buildout/software-profiles
$ python -S -c 'import urllib;print urllib.urlopen("http://svn.zope.org/' \
'*checkout*''/zc.buildout/trunk/bootstrap/bootstrap.py").read()' | python -S -
$ bin/buildout
Congratulations, neo's installation is over ! You can now launch the different
services you installed with typing bin/<neo_script>.
#########################
### 2. Running tests ###
#########################
This part is recommended for experienced users, who already installed all other
neo parts, and could manage neo pretty easily.
Export your path to set up your environment:
$ export $PATH=$PATH:<download_neo>/trunk/bin
Add it to your neo installation :
$ bin/buildout -Nvv
Run neo tests :
$ bin/neotestrunner -fzu (functional, unit tests and zodb tests)
If this returns no errors or fails, it means that it's over for you ! Neo's
correctly installed on your machine, congratulations !
#############################
### 3. Building neo eggs ###
#############################
This part is recommended for experienced users, who wants to make a neo
installation from beginning. It explains how to make eggs from a
precise neo part, instead of eggifying the whole project.
Each part of neo (lib, admin, master, storage, neoctl, client) has its own
setup.py file, which is located on <download_neo>/trunk/setup_files, ans which
allows to build the linked egg. See VI. Setup files to check which setup.py
file you need. Then execute the following commands :
$ svn checkout https://svn.erp5.org/repos/neo/trunk
$ cd <download_neo>/trunk/neo
(setup_cmn.py is the main part of neo, needed by all other processes)
$ python setup.*.py bdist_egg
Check in your trunk/bdist folder that you have an egg for each part of neo you
want to install. If it's ok, then go on to part II.
########################
### 4. Common errors ###
########################
Problem : The egg I need is not present in my trunk/bdist folder.
Solution : Select the correct setup_<part>.py in trunk/setup_files
Problem: bin/buildout is not present
Solution: Launch the bootstrap command
Problem : bootstrap command is not working
Solution : Try to use another python version
Tests errors :
'neoadmin' not found
'neomaster' not found
'neostorage' not found
Solution :
Check your path and remember to export <download_neo>/trunk/bin to it
How to eggify Mock projet (Neo dependency)
- Download source http://python-mock.sourceforge.net/ in a mock/ folder
- Write setup.py file
from setuptools import setup, find_packages
setup(
name='mock',
version='1.0',
description='Manipulate mock objects for NEO',
author='Dave Kirby',
author_email='test@admin.com',
url='http://python-mock.sourceforge.net/',
license="BSD",
packages=find_packages(),
package_dir={'mock': 'mock'},
zip_safe=False,
)
- Launch python setup.py bdist_egg
The created egg is in dist/ folder.
#Neo global config file - partitioned
[buildout]
# extends-cache = extends-cache
parts =
neoinstall
find-links =
../../eggs/
[neoinstall]
recipe = zc.recipe.egg
interpreter = python
scripts =
neotestrunner
neomigrate
neomaster
neoadmin
neostorage
neoctl
eggs =
neotests
neoadmin
neoctl
neostorage
neoclient
neomaster
# for unit tests
zope.testing
psutil
#!/bin/sh
PYTHON=python2.6
INSTALL_DIR=eggs/
cd dirname $0/..
case "$1" in
"develop")
MODE=develop
PYTHONPATH=$PYTHONPATH:$INSTALL_DIR;;
*) MODE=bdist_egg ;;
esac
$PYTHON setup_client.py $MODE --dist-dir=$INSTALL_DIR
$PYTHON setup_common.py $MODE --dist-dir=$INSTALL_DIR
$PYTHON setup_admin.py $MODE --dist-dir=$INSTALL_DIR
$PYTHON setup_ctl.py $MODE --dist-dir=$INSTALL_DIR
$PYTHON setup_tests.py $MODE --dist-dir=$INSTALL_DIR
$PYTHON setup_storage.py $MODE --dist-dir=$INSTALL_DIR
$PYTHON setup_master.py $MODE --dist-dir=$INSTALL_DIR
from setuptools import setup, find_packages
setup(name='neo',
version='0.1.0',
description='Distributed, redundant and transactional storage for ZODB',
author='NEOPPOD',
author_email='neo-dev@erp5.org',
url='http://www.neoppod.org/',
license="GPL 2",
packages=find_packages(),
package_dir={'neo': 'neo'},
install_requires=[
'ZODB3',
],
extras_require={
'storage': ['MySQL-python'],
'test': ['MySQL-python', 'mock'],
},
entry_points={
# WKRD: Official Python packaging and buildout are not designed to build
# several binary packages from a single source one. This setup.py
# contains several hacks to work around this horrible limitation, and
# make it possible to develop several eggs in the same tree/repository:
# - in development mode, it behaves somehow like a mono-package
# - otherwise, sdist/bdist commands produces several packages
description = 'Distributed, redundant and transactional storage for ZODB'
setup_common = dict(
version = '1.0a1',
author = 'NEOPPOD',
author_email = 'neo-dev@erp5.org',
url = 'http://www.neoppod.org/',
license = 'GPL 2+',
install_requires = ['neoppod'],
namespace_packages = ['neo', 'neo.scripts'],
zip_safe = True,
)
import os, sys
from distutils.filelist import findall
try:
setup_only, = [l[5:].strip() for l in open('PKG-INFO') if l[:5] == 'Name:']
except IOError:
setup_only = None
def setup(name, packages=(), extra_files=(), _done=[], **kw):
if setup_only and setup_only != name or \
_done and ('--help' in sys.argv or '--help-commands' in sys.argv):
return
from distutils.command.build import build
from setuptools import find_packages, setup
from setuptools.command import egg_info, develop
kw['packages'] = sum((isinstance(p, basestring) and [p] or
list(p[:1]) + [p[0] + '.' + x for x in find_packages(*p[1:])]
for p in packages), [])
# monkey-patch to build package in different folders
build_initialize_options = build.initialize_options
def initialize_options(self):
build_initialize_options(self)
self.build_base = os.path.join(self.build_base, name)
# per-package manifest, instead of walking files under revision control
walk_revctrl = egg_info.walk_revctrl
# create only 1 egg-link for buildout
develop_finalize_options = develop.develop.finalize_options
def finalize_options(self):
develop_finalize_options(self)
self.egg_link = os.devnull
try:
build.initialize_options = initialize_options
egg_info.walk_revctrl = lambda *args, **kw: extra_files
if _done:
develop.develop.finalize_options = finalize_options
setup(name = name, **dict(setup_common, **kw))
finally:
develop.develop.finalize_options = develop_finalize_options
build.initialize_options = build_initialize_options
egg_info.walk_revctrl = walk_revctrl
_done.append(name)
setup(
name = 'neoppod',
description = description + ' - Common part',
packages = ['neo', ('neo.lib', 'neo/lib')],
# Raah!!! I wish I could write something like (for 'neoppod'):
# install_requires = ['python>=2.5|ctypes']
install_requires = (),
namespace_packages = ['neo'],
extra_files = ['TODO'],
)
setup(
name = 'neoadmin',
description = description + ' - Admin part',
packages = ['neo', ('neo.admin', 'neo/admin')],
py_modules = ['neo.scripts.neoadmin'],
entry_points = {
'console_scripts': [
'neoadmin=neo.scripts.neoadmin:main',
'neoctl=neo.scripts.neoctl:main',
'neomaster=neo.scripts.neomaster:main',
'neomigrate=neo.scripts.neomigrate:main',
'neostorage=neo.scripts.neostorage:main',
],
},
extra_files = ['neo.conf'],
)
setup(
name = 'neoclient',
description = description + ' - Client part',
packages = ['neo', ('neo.client', 'neo/client')],
install_requires = ['neoppod', 'ZODB3 >= 3.9'],
py_modules = ['neo.scripts.neomigrate'],
entry_points = {
'console_scripts': [
'neoadmin=neo.scripts.neomigrate:main',
],
},
package_data = {
'neo': [
'neo.client': [
'component.xml',
],
},
zip_safe=False,
)
setup(
name = 'neoctl',
description = description + ' - Controller part',
packages = ['neo', ('neo.neoctl', 'neo/neoctl')],
py_modules = ['neo.scripts.neomigrate'],
entry_points = {
'console_scripts': [
'neoctl=neo.scripts.neomigrate:main',
],
},
)
setup(
name = 'neomaster',
description = description + ' - Master part',
packages = ['neo', ('neo.master', 'neo/master')],
py_modules = ['neo.scripts.neomaster'],
entry_points = {
'console_scripts': [
'neomaster=neo.scripts.neomaster:main',
],
},
extra_files = ['neo.conf'],
)
setup(
name = 'neostorage',
description = description + ' - Storage part',
packages = ['neo', ('neo.storage', 'neo/storage')],
py_modules = ['neo.scripts.neostorage'],
entry_points = {
'console_scripts': [
'neostorage=neo.scripts.neostorage:main',
],
},
extras_require = {
'btree': ['ZODB3'],
'mysqldb': ['MySQL-python'],
},
extra_files = ['neo.conf'],
)
if setup_only in ('neotests', None) and not os.path.exists('mock.py'):
import cStringIO, md5, urllib, zipfile
mock_py = zipfile.ZipFile(cStringIO.StringIO(urllib.urlopen(
'http://downloads.sf.net/sourceforge/python-mock/pythonmock-0.1.0.zip'
).read())).read('mock.py')
if md5.md5(mock_py).hexdigest() != '79f42f390678e5195d9ce4ae43bd18ec':
raise EnvironmentError("MD5 checksum mismatch downloading 'mock.py'")
open('mock.py', 'w').write(mock_py)
setup(
name = 'neotests',
description = description + ' - Testing part',
packages = ['neo', ('neo.tests', 'neo/tests')],
install_requires = [
'neoadmin',
'neoclient',
'neoctl',
'neomaster',
'neostorage[btree, mysqldb]',
'psutil',
'zope.testing',
],
py_modules = ['mock', 'neo.scripts.runner'],
entry_points = {
'console_scripts': [
'neotestrunner=neo.scripts.runner:main',
],
},
extra_files = ['TESTS.txt'] + findall('tools'),
)
from setuptools import setup, find_packages
setup(name='neoadmin',
version=
'0.1.0',
description=
'Distributed, redundant and transactional storage for ZODB-Admin part',
author=
'NEOPPOD',
author_email=
'neo-dev@erp5.org',
url=
'http://www.neoppod.org/',
license=
"GPL 2",
py_modules=[
'neo.scripts.neoadmin'
],
packages=['neo.admin'],
package_dir={
'neo':'neo',
},
namespace_packages=['neo','neo.scripts'],
install_requires=[
'neo',
],
entry_points = {
'console_scripts': [
'neoadmin=neo.scripts.neoadmin:main',
],
},
zip_safe=False,
)
from setuptools import setup, find_packages
setup(name='neoclient',
version=
'0.1.0',
description=
'Distributed, redundant and transactional storage for ZODB-Admin part',
author=
'NEOPPOD',
author_email=
'neo-dev@erp5.org',
url=
'http://www.neoppod.org/',
license=
"GPL 2",
py_modules=[
'neo.scripts.neomigrate',
],
packages=['neo.client','neo.client.handlers'],
package_dir={
'neo':'neo',
},
namespace_packages=['neo','neo.client'],
install_requires=[
'neo',
'ZODB3',
],
zip_safe=False,
)
from setuptools import setup, find_packages
setup(name='neo',
version='0.1.0',
description='Distributed, redundant and transactional storage for ZODB- Common part',
author='NEOPPOD',
author_email='neo-dev@erp5.org',
url='http://www.neoppod.org/',
license="GPL 2",
packages=['neo.lib'],
package_dir={
'neo':'neo',
},
namespace_packages=['neo'],
package_data = {
'neo': [
'component.xml',
],
},
zip_safe=False,
)
from setuptools import setup, find_packages
setup(name='neoctl',
version=
'0.1.0',
description=
'Distributed, redundant and transactional storage for ZODB-Admin part',
author=
'NEOPPOD',
author_email=
'neo-dev@erp5.org',
url=
'http://www.neoppod.org/',
license=
"GPL 2",
py_modules=[
'neo.scripts.neoctl'
],
packages=['neo.neoctl'],
package_dir={
'neo':'neo',
},
namespace_packages=['neo','neo.scripts'],
install_requires=[
'neo',
],
entry_points = {
'console_scripts': [
'neoctl=neo.scripts.neoctl:main',
],
},
zip_safe=False,
)
from setuptools import setup, find_packages
setup(name='neomaster',
version=
'0.1.0',
description=
'Distributed, redundant and transactional storage for ZODB-Admin part',
author=
'NEOPPOD',
author_email=
'neo-dev@erp5.org',
url=
'http://www.neoppod.org/',
license=
"GPL 2",
py_modules=[
'neo.scripts.neomaster',
],
packages=['neo.master','neo.master.handlers'],
package_dir={
'neo':'neo',
},
namespace_packages=['neo','neo.scripts'],
install_requires=[
'neo',
],
entry_points = {
'console_scripts': [
'neomaster=neo.scripts.neomaster:main',
],
},
zip_safe=False,
)
from setuptools import setup, find_packages
setup(name='neostorage',
version=
'0.1.0',
description=
'Distributed, redundant and transactional storage for ZODB-Admin part',
author=
'NEOPPOD',
author_email=
'neo-dev@erp5.org',
url=
'http://www.neoppod.org/',
license=
"GPL 2",
py_modules=[
'neo.protocol',
'neo.scripts.neostorage'
],
packages=['neo.storage','neo.storage.database','neo.storage.handlers'],
package_dir={
'neo':'neo',
},
namespace_packages=['neo','neo.scripts'],
install_requires=[
'neo',
'MySQL-python',
],
entry_points = {
'console_scripts': [
'neostorage=neo.scripts.neostorage:main',
],
},
zip_safe=False,
)
from setuptools import setup, find_packages
setup(name='neotests',
version=
'0.1.0',
description=
'Distributed, redundant and transactional storage for ZODB-Admin part',
author=
'NEOPPOD',
author_email=
'neo-dev@erp5.org',
url=
'http://www.neoppod.org/',
license=
"GPL 2",
py_modules=[
'neo.scripts.runner'
],
packages=['neo.tests','neo.tests.functional','neo.tests.master','neo.tests.storage','neo.tests.zodb','neo.tests.client'],
package_dir={
'neo':'neo',
},
namespace_packages=['neo','neo.scripts'],
install_requires=[
'neo',
'neoadmin',
'neostorage',
'neoclient',
'neomaster',
'neoctl',
'mock'
],
entry_points = {
'console_scripts': [
'neotestrunner=neo.scripts.runner:main',
],
},
zip_safe=False,
)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment