Skip to content

O
olapy
Commit 718df38c authored by mouadh's avatar mouadh
Browse files
Options

update docstring

parent 874fec7f
Hide whitespace changes
Inline Side-by-side
Showing with 243 additions and 117 deletions
docs/conf.py
View file @ 718df38c
......@@ -50,7 +50,7 @@ source_suffix = '.rst'
master_doc = 'index'
# General information about the project.
project = u'olapy'
project = u'OlaPy'
copyright = u'2017, Mouadh Kaabachi'
author = u'Mouadh Kaabachi'
......@@ -59,9 +59,9 @@ author = u'Mouadh Kaabachi'
# built documents.
#
# The short X.Y version.
version = u'0.0.1'
version = u'0.3.1'
# The full version, including alpha/beta/rc tags.
release = u'0.0.1'
release = u'0.3.1'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
......@@ -312,7 +312,7 @@ man_pages = [
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'olapy', u'olapy Documentation',
(master_doc, 'olapy', u'OlaPy Documentation',
author, 'olapy', 'One line description of project.',
'Miscellaneous'),
]
......
docs/cubes.rst
View file @ 718df38c
.. _cubes:
Cubes Creation
Cubes creation
==============
If you want to add new cubes, this is very simple, just paste your csv files under a new folder with (its name will be your new cube name)
in cubes folder, so the path to your cube become, olapy/cubes/YOUR_CUBE/YOU_CSV_FILES
in cubes folder, so the path to your cube become, home-directory/olapy-data/cubes/YOUR_CUBE/{YOU_CSV_FILES}
**IMPORTANT**
Here are the rules to apply in the tables (csv files) so that it works perfectly:
1) make sure that your tables follow the `star schema <http://datawarehouse4u.info/Data-warehouse-schema-architecture-star-schema.html>`_
2) Fact table must have 'Facts' name
2) Fact table should be named 'Facts'
3) the id columns must have _id at the end (product_id, person_id...)
4) the columns name must be in a good order (hierarchy) (example : Continent -> Country -> City...)
5) each column name of each table must be unique from the other tables column names
6) verify that data entered in tables does not contains any white spaces at the end
7) if you want to set up some orders to the tables , put the desired order with tables name , for exmple 1Geography.csv , 2Product.csv
8) time dimension must be named 'Time'
*take a look to the default cubes structure (sales and foodmart)*
**THE SAME THING IF YOU WANT TO WORK WITH POSTGRES TABLES**
-----------------------------------------------------------------------
Here are two examples of dimensions structure:
Examples:
......
docs/customize.rst 0 → 100644
View file @ 718df38c
.. _customize:
Cube customization
==================
If you want to customize a cube, you can do this with a configuration file (home-directory/olapy-data/cubes/cubes-config.xml)
Here is an examples of configuration::
<?xml version="1.0" encoding="UTF-8"?>
<cubes>
<!-- if you want to set an authentication mechanism in excel so to access cube,
user must set a token with login url like 'http://127.0.0.1/admin -->
<!-- default password = admin -->
<xmla_authentication>False</xmla_authentication>
<cube>
<!-- cube name => db name -->
<name>labster</name>
<!-- source : postgres | csv -->
<source>postgres</source>
<!-- star building customized star schema -->
<facts>
<!-- facts table name -->
<table_name>stats_line</table_name>
<keys>
<!-- ref = table_name.column -->
<column_name ref="orgunit.id">departement_id</column_name>
</keys>
<!-- specify measures explicitly -->
<measures>
<!-- by default, all number type columns in facts table, or you can specify them here -->
<name>montant</name>
<name>salaire_brut_mensuel</name>
<name>cout_total_mensuel</name>
</measures>
</facts>
<!-- end building customized star schema -->
<!-- star building customized dimensions display in excel from the star schema -->
<dimensions>
<dimension>
<!-- if you want to keep the same name for excel display, just use the same name in name and displayName -->
<name>stats_line</name>
<displayName>Demande</displayName>
<columns>
<!-- columns order matter -->
<name>type_demande</name>
<name>financeur</name>
<name>wf_state</name>
<name>type_recrutement</name>
</columns>
</dimension>
<dimension>
<!-- if you want to keep the same name for excel display, just use the same name in name and displayName -->
<name>orgunit</name>
<displayName>Organisation</displayName>
<columns>
<!-- columns order matter -->
<name>type</name>
<name>nom</name>
<name>sigle</name>
</columns>
</dimension>
</dimensions>
<!-- end building customized dimensions display in excel from the star schema -->
</cube>
</cubes>
docs/index.rst
View file @ 718df38c
......@@ -3,34 +3,35 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to olapy's documentation!
Welcome to OlaPy's documentation!
=================================
**olapy** is an OLAP engine with MDX support, web interface (slice & dice) and XMLA support for excel client.
**OlaPy** is an OLAP engine with MDX support, web interface (slice & dice) and XMLA support for excel client.
it can be found at
`olapy <https://github.com/abilian/olapy-core>`_.
`OlaPy <https://github.com/abilian/olapy>`_.
Simple use
----------
For the impatient: here's a quick overview of how to use this project. Normally,
all you have to do is install the application first, like this::
For the impatient: here's a quick overview of how to use this project.
All you have to do is install the application first, like this::
python setup.py install
and then, it is already done, from the root directory *(which is olapy-core)* you can
and then, it is already done, you can
**Starting XMLA Server** by using::
**Start XMLA Server** by using::
python olapy ( -c | --console , if you want to print logs only in the console)
and use the url :http://127.0.0.1:8000/xmla in excel the see your :ref:`cubes`.
and use the url :http://127.0.0.1:8000/ in excel the see your :ref:`cubes <cubes>`.
- Be sure to use `Provider=MSOLAP.6 <https://blogs.technet.microsoft.com/excel_services__powerpivot_for_sharepoint_support_blog/2015/10/30/configuring-the-msolap-data-provider-version/>`_ in excel
- If you want to select many measures in excel, select them before the dimensions and then drag and drop 'Values' attribute from COLUMNS to ROWS excel field
you can also :ref:`customize <customize>` your cubes
**Starting Web DEMO** by using::
......@@ -50,11 +51,15 @@ and use the url :http://127.0.0.1:8000/xmla in excel the see your :ref:`cubes`.
python manage.py dropdb
Tests
^^^^^
-----
To run tests, run::
pytest tests
to run tests (make sure that the server is started) ::
or simply (on Unix-like systems)::
python -m pytest tests
make test
For more information about how to use this library, see the :ref:`api`.
......@@ -72,6 +77,7 @@ Contents:
:maxdepth: 2
cubes
customize
api
web
......
olapy/core/mdx/executor/execute.py
View file @ 718df38c
......@@ -22,7 +22,8 @@ class MdxEngine:
"""
The principal class for executing a query
:param cube_name: Cube name , it must be under home_directory/olapy-data/CUBE_FOLDER (example : home_directory/olapy-data/cubes/sales)
:param cube_name: It must be under home_directory/olapy-data/CUBE_FOLDER (example : home_directory/olapy-data/cubes/sales)
"""
CUBE_FOLDER = "cubes"
# (before instantiate MdxEngine I need to access cubes information)
......@@ -198,6 +199,7 @@ class MdxEngine:
def load_tables(self):
"""
load all tables { Table name : DataFrame } of the current cube instance
:return: dict with key as table name and DataFrame as value
"""
......@@ -369,21 +371,27 @@ class MdxEngine:
"""
get all tuples in the mdx query
example:
example::
SELECT {[Geography].[Geography].[All Continent].Members,
[Geography].[Geography].[Continent].[Europe]} ON COLUMNS,
SELECT {
[Geography].[Geography].[All Continent].Members,
[Geography].[Geography].[Continent].[Europe]
} ON COLUMNS,
{[Product].[Product].[Company]} ON ROWS
{
[Product].[Product].[Company]
} ON ROWS
FROM {sales}
it returns :
[['Geography','Geography','Continent'],
['Geography','Geography','Continent','Europe'],
['Product','Product','Company']]
[
['Geography','Geography','Continent'],
['Geography','Geography','Continent','Europe'],
['Product','Product','Company']
]
:param query: mdx query
......@@ -461,13 +469,16 @@ class MdxEngine:
def change_measures(tuples_on_mdx):
"""
set measures to which exists in the query
:param tuples_on_where: list of tuples
example : [ '[Measures].[Amount]' , '[Geography].[Geography].[Continent]' ]
:return: measures columns names
:param tuples_on_mdx: list of tuples:
example : [ '[Measures].[Amount]' , '[Geography].[Geography].[Continent]' ]
:return: measures column's names
"""
return [
tple[-1] for tple in tuples_on_mdx if tple[0].upper() == "MEASURES"
]
......@@ -480,17 +491,17 @@ class MdxEngine:
:param tuple_as_list: list of tuples
example : [ '[Measures].[Amount]' , '[Geography].[Geography].[Continent]' ]
example : [ '[Measures].[Amount]' , '[Geography].[Geography].[Continent]' ]
:return: dimension and columns dict
example :
{
Geography : ['Continent','Country'],
Product : ['Company']
Facts : ['Amount','Count']
}
"""
example :
{
Geography : ['Continent','Country'],
Product : ['Company']
Facts : ['Amount','Count']
}
"""
axes = {}
# TODO optimize
......@@ -665,13 +676,14 @@ class MdxEngine:
columns_to_keep :
Geo -> Continent,Country
Prod -> Company
Time -> Year,Month,Day
we have to use only dimension's columns of current dimension that exist in tuple_as_list a keep other dimensions
columns
we have to use only dimension's columns of current dimension that exist in tuple_as_list and keep other dimensions columns
so if tuple_as_list = ['Geography','Geography','Continent']
columns_to_keep will be:
......@@ -679,30 +691,29 @@ class MdxEngine:
columns_to_keep :
Geo -> Continent
Prod -> Company
Time -> Year,Month,Day
we need columns_to_keep for grouping our columns in the DataFrame
:param tuple_as_list:
example :
['Geography','Geography','Continent']
:param tuple_as_list: example : ['Geography','Geography','Continent']
:param columns_to_keep:
example :
{
'Geography':
['Continent','Country'],
example :
{
'Time':
['Year','Month','Day']
}
'Geography':
['Continent','Country'],
'Time':
['Year','Month','Day']
}
:return: updated columns_to_keep
"""
......@@ -720,7 +731,7 @@ class MdxEngine:
def execute_mdx(self):
"""
execute and MDX Query
execute an MDX Query
usage ::
......@@ -730,10 +741,10 @@ class MdxEngine:
:return: dict with DataFrame execution result and (dimension and columns used as dict)
{
'result' : DataFrame result
'columns_desc' : dict of dimension and columns used
}
{
'result' : DataFrame result
'columns_desc' : dict of dimension and columns used
}
"""
......
olapy/core/mdx/parser/parse.py
View file @ 718df38c
......@@ -32,17 +32,17 @@ class MdxParser:
WHERE [Time].[Calendar].[Year].[2010]
+------------+------------------------------------------------+
| | [Geography].[Geo].[Country].[France] |
| column | |
| | [Geography].[Geo].[Country].[Spain] |
+------------+------------------------------------------------+
| row | [Product].[Prod].[Company].[Crazy Development] |
+------------+------------------------------------------------+
| cube | [Sales] |
+------------+------------------------------------------------+
| condition | [Time].[Calendar].[Year].[2010] |
+------------+------------------------------------------------+
+------------+------------------------------------------------+
| | [Geography].[Geo].[Country].[France] |
| column | |
| | [Geography].[Geo].[Country].[Spain] |
+------------+------------------------------------------------+
| row | [Product].[Prod].[Company].[Crazy Development] |
+------------+------------------------------------------------+
| cube | [Sales] |
+------------+------------------------------------------------+
| condition | [Time].[Calendar].[Year].[2010] |
+------------+------------------------------------------------+
:param query: MDX Query
:param axis: column | row | cube | condition | all
......
olapy/core/mdx/tools/config_file_parser.py
View file @ 718df38c
......@@ -12,7 +12,9 @@ class ConfigParser:
Parse olapy config files
Config file used if you want to show only some measures, dimensions, columns... in excel
Config file should be under 'home-directory/olapy-data/cubes/cubes-config.xml'
Config file Structure::
......@@ -23,6 +25,8 @@ class ConfigParser:
<!-- if you want to set an authentication mechanism in excel so to access cube,
user must set a token with login url like 'http://127.0.0.1/admin -->
<!-- default password = admin -->
<xmla_authentication>False</xmla_authentication>
<cube>
......@@ -123,6 +127,7 @@ class ConfigParser:
def config_file_exist(self):
"""
check whether the config file exists or not
:return: True | False
"""
return os.path.isfile(os.path.join(self.cube_path, self.file_name))
......@@ -130,6 +135,7 @@ class ConfigParser:
def xmla_authentication(self):
"""
check if excel need authentication to access cubes or not. (xmla_authentication tag in the config file)
:return: True | False
"""
......@@ -167,6 +173,7 @@ class ConfigParser:
def construct_cubes(self):
"""
construct cube (with it dimensions) and facts from the config file
:return: list of Cubes instance
"""
......
olapy/core/services/xmla.py
View file @ 718df38c
......@@ -231,9 +231,9 @@ def start_server(write_on_file=False):
"""
start the xmla server
:param write_on_file = :
:param write_on_file:
- False -> server logs will be displayed on console
- True -> server logs will be saved in file
- True -> server logs will be saved in file (~/olapy-data/logs)
:return: server instance
"""
......
olapy/core/services/xmla_discover_tools.py
View file @ 718df38c
......@@ -36,7 +36,7 @@ class XmlaDiscoverTools():
def change_catalogue(self, new_catalogue):
"""
if you change the catalogue(cube) in any request, we have to instantiate the MdxEngine with the new catalogue
if you change the catalogue (cube) in any request, we have to instantiate the MdxEngine with the new catalogue
:param new_catalogue: catalogue name
:return: new instance of MdxEngine with new star_schema_DataFrame and other variables
......
olapy/core/services/xmla_execute_tools.py
View file @ 718df38c
......@@ -17,34 +17,34 @@ class XmlaExecuteTools():
"""
Split DataFrame into multiple ones by dimension
example:
in :
+-------------+----------+----------+---------+---------+
| Continent | Country | Company |Article | Amount |
+=============+==========+==========+=========+=========+
| America | US | MS |Crazy De | 35150 |
+-------------+----------+----------+---------+---------+
out :
'Geography':
+-------------+----------+---------+
| Continent | Country | Amount |
+=============+==========+=========+
| America | US | 35150 |
+-------------+----------+---------+
'Product':
+----------+---------+---------+
| Company |Article | Amount |
+==========+=========+=========+
| MS |Crazy De | 35150 |
+----------+---------+---------+
example::
in :
+-------------+----------+----------+---------+---------+
| Continent | Country | Company |Article | Amount |
+=============+==========+==========+=========+=========+
| America | US | MS |Crazy De | 35150 |
+-------------+----------+----------+---------+---------+
out :
'Geography':
+-------------+----------+---------+
| Continent | Country | Amount |
+=============+==========+=========+
| America | US | 35150 |
+-------------+----------+---------+
'Product':
+----------+---------+---------+
| Company |Article | Amount |
+==========+=========+=========+
| MS |Crazy De | 35150 |
+----------+---------+---------+
:param mdx_execution_result: MdxEngine.execute_mdx() result
......@@ -61,15 +61,12 @@ class XmlaExecuteTools():
def get_tuple_without_nan(tuple):
"""
remove nan from tuple.
example
in :
['Geography','Continent','-1']
out :
example:
['Geography','Continent']
in : ['Geography','Continent','-1']
out : ['Geography','Continent']
:param tuple: tuple as list
:return: tuple as list without -1
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7