IaaS-Advisor is a fully open source comparison service infrastructure as a service aggregating different information such as pricing, availability and performance.

Methodology

IaaS-Advisor is based on several open source projects like:

1 - Technology Components

TestNode:
TestNode is a free software for running tests. It allows you to run different tests in parallel to any code base. TestNode shall check the availability of the code, construction, test management, test execution and reporting of results. TestNode is particularly used by developers in the development phases of software to automate the execution of unit tests and running utilities external test.

SlapOS:
SlapOS is a free software that allows an individual or a company to simply deploy their own cloud infrastructure. SlapOS covers all aspects of the Cloud: IaaS, PaaS, SaaS. Its unified operating infrastructure cloud technology to automate administration, deployment, allocation, configuration, monitoring, backup, recovery, testing, updating, migration and billing enterprise applications.

Cloud-Quote:
Cloud-Quote is the front of IaaS-Advisor offering a great opportunity of filters and sorting. It is based on RenderJS (http://www.renderjs.org/) that relies on JIO (http://www.j-io.org/) for asynchronous document management and storage JSON and jQuery Mobile (http://jquerymobile.com/) for managing display and requirejs (http://requirejs.org/) for cache management and interactions.

ERP5:
ERP5 is an ERP open source integrated accounting, customer relationship management (CRM), purchasing, sales, inventory, product data (PDM), production (SCM) integrated logistics and human resources.

2 - Operation

2.1 - General Principles

TestNode Global Architecture

TestNode operates in standalone mode by monitoring the activity of one or more source code repositories in order to detect any changes (commit). Once a change is made, TestNode checks in a knowledge base if it is to test this revision, the test runs and returns the results.

Some major projects which have a high activity TestNode is designed to not only run tests in parallel, but also to work with other TestNode.

2.2 - Principles of Operation

TestNode Loop Schemas TestNode operates in seven steps:

  1. The TestNode declares itself as available and asks for the test suites to monitored (each test suite defines one or more deposits source to check)
  2. Monitoring the source repository in order to detect any changes (commits).
    If a change is detected by TestNode, it will automatically perform the following steps.
  3. The verification of a request for running tests on these changes.
    If a revision has been detected, TestNode sending its signature to the knowledge base in order to obtain confirmation of this test review.
  4. Automatic reconstruction of the software from source.
    When TestNode must test a particular revision of the project, it will automatically rebuild it.
  5. The request of tests to perform.
    Once the project is rebuilt, which according to the project may take some time, TestNode will ask for the test to perform. Indeed, the architecture is intended to run in parallel several TestNode, so another TestNode could start to work before and after the coming TestNode, so the following test will be awarded..
  6. The test execution. TestNode runs parallel tests as requested.
    These tests can be either unit tests developed in the project, external tools to test or performance measurement project, etc.
  7. Sending results. Upon obtaining the results of a test, TestNode send them to be stored and archived.

2.3 - Deployment with SlapOS

In order to automatically deploy TestNode in virtual machines, they must be declared as a node to a SlapOS Master. This procedure is described at: http://community.slapos.org/wiki/developer-Installing.SlapOS.Package

Various information, such as disk space, the quantity of RAM, processor type and frequency, and the hourly rate charged by the supplier are also indicated in the Master SlapOS during this procedure.

Once node is registered, an install of a TestNode is requested as shown at: http://community.slapos.org/web_page_module/2811/

In the IaaS-Advisor context, the configuration of TestNode installs Python and MariaDB and their testing or benchmarck tools. The configuration also indicates which instance of ERP5 contain the list of tests to be performed and will be responsible of results storage.

2.4 - Tests definition in the IaaS-Advisor context

First test: uptime

The SlapOS Master continuously monitors all SlapOS nodes. This monitoring will detect any unavailability of the virtual machine on the network, that permit us to calculate its real availability. The SlapOS Master also manages the monitoring of started instances.

Virtual machines dedicated to IaaS-Advisor contain only an instance of TestNode, which eliminates any external software pollution that may impact performances measures.

Second test: software building

Before each test to perform, TestNode retrieves the latest version of the sources of software, (re)built it and instantiates it.

This test validates that the building of software is still operational. In IaaS-Advisor context, only Python and MariaDB are -so far- installed then instantiated. Both have a reputation for verbosity to compilation which allows to investigate deeply in logs files in case of issue.

Third test: running Python and MariaDB tests

The tests triggering is usually done when TestNode detects a change in a source repository. In the case of IaaS-Advisor these tests are triggered periodically.

For the moment, IaaS-Advisor tests integrates Python and MariaDB testing tools. These standard tools are commonly used for performance measures. So, the results returned are easily understood through by everyone.

2.5 - Aggregation of results tests in a web service

Cloud-Quote is a new approach to web services. Via renderjs, IaaS-Advisor website does not requires any server-side processing.

All information processed by cloud-Quote is formalized in a json data-file containing all information about virtual machines, availabilities, and test results. This json file is processed in the browser by jIO, which is responsible for extracting the data for display. In order to operate on all screens, IaaS-Advisor web pages are generated with the jQuery Mobile framework. To manage and optimize all javascript components and their dependencies, RequireJS is used.

3 - Conclusion

IaaS-Advisor is only in its first iteration and intended to grow to incorporate more features and become the first fully open source services performance measurement of cloud computing.

Future actions: