Cross-service testing – The Tempest plugin

This post is the second part of a series about writing a Tempest plugin for cross-service integration testing in OpenStack.
If you missed the first post you might want to go back and read the introduction at least.

Integration tests for Nova and Neutron are included in Tempest, but those for Designate and Heat are not, so, after preparing the test environment, the second thing I needed was Tempest plugins for both projects. The Tempest plugin interface allows plugins to expose new service clients so that other plugins can easily configure and use them to write integration tests. Unfortunately, that interface was implemented by neither Designate nor Heat.

The Designate team maintains a Tempest plugin. I forked the plugin to add the service client plugin interface. The changes have since been merged back into the official plugin. The designate plugin included, at the time of writing, three different service clients, so I implemented the plugin interface to expose them to other plugins as follows:

For this to work, all the clients must be available in the same module. However as a common practice both Tempest and plugins separate the service client for each API in a dedicated module. This problem can be solved by overriding __all__ in the service client __init__.py module:

The Heat team maintains a Tempest plugin as well, but they don’t maintain a service client in Tempest format at all. I forked the existing code, added a service client and put the code on GitHub.

Preparing the new Tempest plugin

With all preconditions in place, I was ready to start working on the cross-service Tempest plugin.
The first step is to create the plugin skeleton using cookiecutter:

I set project and repo_name to cross_service_tempest_plugin and the test class name to CrossServiceTempestPlugin.
The plugin must be an installable python package. Similar to what most OpenStack projects do, I used pbr to simplify the setup.py.
I added three files for this, which are available on the workshop GitHub repo:

  • setup.cfg
  • setup.py
  • requirements.txt

Files and entry points in setup.cfg must match the project and test class values passed to cookiecutter:

The requirements file includes only Tempest and the two Tempest plugins. Since the plugins are not on PyPi, I specified them using their full git url:

Implement plugin.py

The next step was to implement in the plugin.py module all the methods exposed by Tempest in its Plugin interface plugins.TempestPlugin.

The first step was to create a class that inherits from Tempest’s one and implement the load_tests method, to make tests form the plugin discoverable by Tempest.

Then I wanted to make the plugin configurable. Since I wanted to be able to customise the DNS domain name, I needed a way to tell the test what domain name to use and expect. Tempest plugins allow extending the standard Tempest configuration file with plugin custom configuration groups and values.

The new configuration option was defined in a new module cross_service_tempest_plugin/config.py

I extended the CrossServiceTempestPlugin class two includes the implementation of two more methods from the Tempest plugin interface:

  • register_opts is used by Tempest to register the extra options
  • get_opt_lists is used for config option discovery, used for instance to generate a sample config file

The implementation is straight-forward:

The plugin interface allows plugins to extend existing configuration groups with new configuration items. Plugins associated with a specific service usually extend tempest ServiceAvailableGroup with their own service. This is how it’s done in the plugin for designate: 

It’s important to note that configuration groups extended this way should not be registered in register_opts nor returned by register_opts since they are already known to Tempest.

Further examples are available in the documentation.

The fourth and last method of the plugin interface is used to expose service clients. I already had all the required service clients from either Tempest or the existing plugins, so I didn’t need to define any new one.
I extended the CrossServiceTempestPlugin class with the following implementation of the service clients interface:

At this point, I was ready to add a test class an start writing the test code. I created a new module test_cross_service.py under cross_service_tempest_plugin/tests/scenario. The basic class definition is:

With this code added to a git repo, I had an installable Tempest plugin, with configuration options and one discoverable test.
Since Tempest is installed in a python virtual environment by devstack, it was possible to test everything done to this point:

In the next blog post, I will explain how to write the scenario test in Tempest.

Leave a Reply