============
Rally v0.1.0
============

Information
-----------

+------------------+-----------------------+
| Commits          |        **355**        |
+------------------+-----------------------+
| Bug fixes        |         **90**        |
+------------------+-----------------------+
| Dev cycle        |      **132 days**     |
+------------------+-----------------------+
| Release date     | **25/September/2015** |
+------------------+-----------------------+


Details
-------

This release contains new features, new 42 plugins, 90 bug fixes,
various code and API improvements.


New Features & API changes
~~~~~~~~~~~~~~~~~~~~~~~~~~
* **Improved installation script**

  * Add parameters:

    * ``--develop`` parameter to install rally in editable (develop) mode

    * ``--no-color`` to switch off output colorizing
      useful for automated output parsing and terminals that don't
      support colors.

  * Puts rally.conf under virtualenv etc/rally/ so you can have several
    rally installations in virtualenv

  * Many fixes related to access of different file, like: rally.conf,
    rally db file in case of sqlite

  * Update pip before Rally installation

  * Fix reinstallation


* **Separated Rally plugins & framework**

  Now plugins are here:
  https://github.com/openstack/rally/tree/master/rally/plugins

  Plugins are as well separated common/* for common plugins
  that can be use no matter what is tested and OpenStack related
  plugins


* **New Rally Task framework**

  * All plugins has the same Plugin base:
    rally.common.plugin.pluing.Plugin They have the same mechanisms for:
    discovering, providing information based on docstrings, and in future
    they will use the same deprecation/rename mechanism.

  * Some of files are moved:

    * rally/benchmark ->  rally/task

      *This was done to unify naming of rally task command and
      actually code that implements it.*

    * rally/benchmark/sla/base.py        -> rally/task/sla.py

    * rally/benchmark/context/base.py    -> rally/task/context.py

    * rally/benchmark/scenarios/base.py  -> rally/task/scenario.py

    * rally/benchmark/runners/base.py    -> rally/task/runner.py

    * rally/benchmark/scenarios/utils.py -> rally/task/utils.py

    This was done to:

    * avoid doing rally.benchmark.scenarios import base as scenario_base

    * remove one level of nesting

    * simplify framework structure


  * Some of classes and methods were renamed

    * Plugin configuration:

      * context.context() -> context.configure()

      * scenario.scenario() -> scenario.configure()

      * Introduced runner.configure()

      * Introduced sla.configure()

      This resolves 3 problems:

      * Unifies configuration of different types of plugins

      * Simplifies plugin interface

      * Looks nice with new modules path:
          >>> from rally.task import scenario
          >>> @scenario.configure()


  * Atomic Actions were changed:

    * New rally.task.atomic module

      This  allow us in future to reuse atomic actions in Context plugins

    * Renames:

       rally.benchmark.scenarios.base.AtomicAction
       -> rally.task.atomic.ActionTimer

       rally.benchmark.scenarios.base.atomic_action()
       -> rally.task.atomic.action_timer()

  * **Context plugins decide how to map their data for scenario**

    Now Context.map_for_scenario method can be override to decide
    how to pass context object to each iteration of scenario.

  * Samples of NEW vs OLD context, sla, scenario and runner plugins:

    * Context

    .. code-block:: python

        # Old
        from rally.benchmark.context import base

        @base.context(name="users", order=100)
        class YourContext(base.Context):

            def setup(self):
                # ...

            def cleanup(self):
                # ...

        # New
        from rally.task import context

        @context.configure(name="users", order=100)
        class YourContext(context.Context):

            def setup(self):
                # ...

            def cleanup(self):
                # ...

            def map_for_scenario(self):
                # Maps context object to the scenario context object
                # like context["users"] -> context["user"] and so on.
    ..


    * Scenario

    .. code-block:: python

        # Old Scenario

        from rally.benchmark.scenarios import base
        from rally.benchmark import validation

        class ScenarioPlugin(base.Scenario):

            @base.scenario()
            def some(self):
                self._do_some_action()


            @base.atomic_action_timer("some_timer")
            def _do_some_action(self):
                # ...

        # New Scenario

        from rally.task import atomic
        from rally.task import scenario
        from rally.task import validation

        # OpenStack scenario has different base now:
        # rally.plugins.openstack.scenario.OpenStackScenario
        class ScenarioPlugin(scenario.Scenario):

            @scenario.configure()
            def some(self):
                self._do_some_action()

            @atomic.action_timer("some_action")
            def _do_some_action(self):
                # ...
    ..

    * Runner

    .. code-block:: python

        ## Old

        from rally.benchmark.runners import base

        class SomeRunner(base.ScenarioRunner):

             __execution_type__ = "some_runner"

            def _run_scenario(self, cls, method_name, context, args)
              # Load generation

            def abort(self):
              # Method that aborts load generation

        ## New

        from rally.task import runner

        @runner.configure(name="some_runner")
        class SomeRunner(runner.ScenarioRunner):

            def _run_scenario(self, cls, method_name, context, args)
                # Load generation

            def abort(self):
                # Method that aborts load generation

    ..

    * SLA

    .. code-block:: python

        # Old

        from rally.benchmark import sla

        class FailureRate(sla.SLA):
            # ...

        # New

        from rally.task import sla

        @sla.configure(name="failure_rate")
        class FailureRate(sla.SLA):
            # ...
    ..


* **Rally Task aborted command**

  Finally you can gracefully shutdown running task by calling:

  .. code:: bash

    rally task abort <task_uuid>
  ..

* **Rally CLI changes**

  * [add] ``rally --plugin-paths`` specify the list of directories with plugins

  * [add] ``rally task report --junit`` - generate a JUnit report
    This allows users to feed reports to tools such as Jenkins.

  * [add] ``rally task abort`` - aborts running Rally task
    when run with the ``--soft`` key, the ``rally task abort`` command is
    waiting until the currently running subtask is finished, otherwise the
    command interrupts subtask immediately after current scenario iterations
    are finished.

  * [add] ``rally plugin show`` prints detailed information about plugin

  * [add] ``rally plugin list`` prints table with rally plugin names and titles

  * [add] ``rally verify genconfig`` generates tempest.conf without running it.

  * [add] ``rally verify install`` install tempest for specified deployment

  * [add] ``rally verify reinstall`` removes tempest for specified deployment

  * [add] ``rally verify uninstall`` uninstall tempest of specified deployment

  * [fix] ``rally verify start --no-use`` --no-use was always turned on

  * [remove] ``rally use`` now each command has subcommand ``use``

  * [remove] ``rally info``

  * [remove] ``rally-manage tempest`` now it is covered by ``rally verify``


* **New Rally task reports**

  * New code is based on OOP style which is base step to make pluggable Reports

  * Reports are now generated for only one iteration over the resulting data
    which resolves scalability issues when we are working with large
    amount of iterations.

  * New Load profiler plot that shows amount of iterations that are working
    in parallel

  * Failed iterations are shown as a red areas on stacked are graphic.

Non backward compatible changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* [remove] ``rally use`` cli command

* [remove] ``rally info`` cli command

* [remove] ``--uuid`` parameter from  ``rally deployment <any>``

* [remove ``--deploy-id`` parameter from:
  ``rally task <any>``, ``rally verify <any>``, ``rally show <any>``

Specs & Feature requests
~~~~~~~~~~~~~~~~~~~~~~~~

  [feature request] Explicitly specify existing users for scenarios

  [feature request] Improve install script and add --uninstall and --version

  [feature request] Allows specific repos & packages in install-rally.sh

  [feature request] Add ability to capture logs from tested services

  [feature request] Check RPC queue perfdata

  [spec] Refactoring Rally cleanup

  [spec] Consistent resource names

Plugins
~~~~~~~

* **Scenarios**:

  [new] CinderVolumes.create_volume_backup

  [new] CinderVolumes.create_and_restore_volume_backup

  [new] KeystoneBasic.add_and_remove_user_role

  [new] KeystoneBasic.create_and_delete_role

  [new] KeystoneBasic.create_add_and_list_user_roles

  [new] FuelEnvironments.list_environments

  [new] CinderVolumes.modify_volume_metadata

  [new] NovaServers.boot_and_delete_multiple_servers

  [new] NeutronLoadbalancerV1.create_and_list_pool

  [new] ManilaShares.list_shares

  [new] CeilometerEvents.create_user_and_get_event

  [new] CeilometerEvents.create_user_and_list_event_types

  [new] CeilometerEvents.create_user_and_list_events

  [new] CeilometerTraits.create_user_and_list_trait_descriptions

  [new] CeilometerTraits.create_user_and_list_traits

  [new] NeutronLoadbalancerV1.create_and_delete_pools

  [new] NeutronLoadbalancerV1.create_and_update_pools

  [new] ManilaShares.create_and_delete_share

  [new] ManilaShares.create_share_network_and_delete

  [new] ManilaShares.create_share_network_and_list

  [new] HeatStacks.create_and_delete_stack

  [new] ManilaShares.list_share_servers

  [new] HeatStacks.create_snapshot_restore_delete_stack

  [new] KeystoneBasic.create_and_delete_ec2credential

  [new] KeystoneBasic.create_and_list_ec2credentials

  [new] HeatStacks.create_stack_and_scale

  [new] ManilaShares.create_security_service_and_delete

  [new] KeystoneBasic.create_user_set_enabled_and_delete

  [new] ManilaShares.attach_security_service_to_share_network

  [new] IronicNodes.create_and_delete_node

  [new] IronicNodes.create_and_list_node

  [new] CinderVolumes.create_and_list_volume_backups

  [new] NovaNetworks.create_and_list_networks

  [new] NovaNetworks.create_and_delete_network

  [new] EC2Servers.list_servers

  [new] VMTasks.boot_runcommand_delete_custom_imagea

  [new] CinderVolumes.create_and_update_volume


* **Contexts**:

  [new] ManilaQuotas

        Add context for setting up Manila quotas:
        shares, gigabytes, snapshots, snapshot_gigabytes, share_networks

  [new] ManilaShareNetworks

        Context for share networks that will be used in case of usage
        deployment with existing users. Provided share networks via context
        option "share_networks" will be balanced between all share creations
        of scenarios.

  [new] Lbaas

        Context to create LBaaS-v1 resources

  [new] ImageCommandCustomizerContext

        Allows image customization using side effects of a command execution.
        E.g. one can install an application to the image and use these image
        for 'boot_runcommand_delete' scenario afterwards.

  [new] EC2ServerGenerator

        Context that creates servers using EC2 api

  [new] ExistingNetwork

        This context lets you use existing networks that have already been
        created instead of creating new networks with Rally. This is useful
        when, for instance, you are using Neutron with a dumb router that is
        not capable of creating new networks on the fly.


* **SLA**:

  [remove] max_failure_rate - use failure_rate instead


Bug fixes
~~~~~~~~~

**90 bugs were fixed, the most critical are**:

* Many fixes related that fixes access of rally.conf and DB files

* Incorrect apt-get "-yes" parameter in install_rally.sh script

* Rally bash completion doesn't exist in a virtualenv

* Rally show networks CLI command worked only with nova networks

* RPS runner was not properly generating load

* Check is dhcp_agent_scheduler support or not in network cleanup

* NetworkContext doesn't work with Nova V2.1

* Rally task input file was not able to use jinja2 include directive

* Rally in docker image was not able to

* Rally docker image didn't contain samples

* Do not update the average duration when iteration failed


Documentation
~~~~~~~~~~~~~

* **Add plugin reference page**

  :ref:`Rally Plugins Reference page <plugin-reference>` page contains a
  full list with

* **Add maintainers section on project info page**

  :ref:`Rally Maintainers section <project_info>` contains information
  about core contributors of OpenStack Rally their responsibilities and
  contacts. This will help us to make our community more transparent and open
  for newbies.

* **Added who is using section in docs**

* **Many small fixes**
