This post is the second part of a series about writing a Tempest plugin for cross-service integration testing in OpenStack.
If you missed the first post you might want to go back and read the introduction at least.
Integration tests for Nova and Neutron are included in Tempest, but those for Designate and Heat are not, so, after preparing the test environment, the second thing I needed was Tempest plugins for both projects. The Tempest plugin interface allows plugins to expose new service clients so that other plugins can easily configure and use them to write integration tests. Unfortunately, that interface was implemented by neither Designate nor Heat.
The Designate team maintains a Tempest plugin. I forked the plugin to add the service client plugin interface. The changes have since been merged back into the official plugin. The designate plugin included, at the time of writing, three different service clients, so I implemented the plugin interface to expose them to other plugins as follows:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
def get_service_clients(self): dns_config = config.service_client_config('dns') admin_params = { 'name': 'dns_admin', 'service_version': 'dns.admin', 'module_path': 'designate_tempest_plugin.services.dns.admin', 'client_names': ['QuotasClient'] } v1_params = { 'name': 'dns_v1', 'service_version': 'dns.v1', 'module_path': 'designate_tempest_plugin.services.dns.v1', 'client_names': ['DomainsClient', 'RecordsClient', 'ServersClient'] } v2_params = { 'name': 'dns_v2', 'service_version': 'dns.v2', 'module_path': 'designate_tempest_plugin.services.dns.v2', 'client_names': ['BlacklistsClient', 'PoolClient', 'QuotasClient', 'RecordsetClient', 'TldClient', 'TransferRequestClient', 'TsigkeyClient', 'ZoneExportsClient', 'ZoneImportsClient', 'ZonesClient'] } admin_params.update(dns_config) v1_params.update(dns_config) v2_params.update(dns_config) return [admin_params, v1_params, v2_params] |
For this to work, all the clients must be available in the same module. However as a common practice both Tempest and plugins separate the service client for each API in a dedicated module. This problem can be solved by overriding __all__ in the service client __init__.py module:
|
1 2 3 4 5 |
from designate_tempest_plugin.services.dns.v1.json.domains_client import DomainsClient from designate_tempest_plugin.services.dns.v1.json.records_client import RecordsClient from designate_tempest_plugin.services.dns.v1.json.servers_client import ServersClient __all__ = ['DomainsClient', 'RecordsClient', 'ServersClient'] |
The Heat team maintains a Tempest plugin as well, but they don’t maintain a service client in Tempest format at all. I forked the existing code, added a service client and put the code on GitHub.
Preparing the new Tempest plugin
With all preconditions in place, I was ready to start working on the cross-service Tempest plugin.
The first step is to create the plugin skeleton using cookiecutter:
|
1 2 |
pip install cookiecutter cookiecutter https://git.openstack.org/openstack/tempest-plugin-cookiecutter.git |
I set project and repo_name to cross_service_tempest_plugin and the test class name to CrossServiceTempestPlugin.
The plugin must be an installable python package. Similar to what most OpenStack projects do, I used pbr to simplify the setup.py.
I added three files for this, which are available on the workshop GitHub repo:
- setup.cfg
- setup.py
- requirements.txt
Files and entry points in setup.cfg must match the project and test class values passed to cookiecutter:
|
1 2 3 4 5 6 7 |
[files] packages = cross_service_tempest_plugins [entry_points] tempest.test_plugins = cross_service = cross_service_tempest_plugin.plugin:CrossServiceTempestPlugin |
The requirements file includes only Tempest and the two Tempest plugins. Since the plugins are not on PyPi, I specified them using their full git url:
|
1 2 3 |
tempest>=17.1.0 # Apache-2.0 -e git+https://github.com/afrittoli/designate-tempest-plugin#egg=designate_tempest_plugin -e git+https://github.com/afrittoli/heat-tempest-plugin#egg=heat_tempest_plugin |
Implement plugin.py
The next step was to implement in the plugin.py module all the methods exposed by Tempest in its Plugin interface plugins.TempestPlugin.
The first step was to create a class that inherits from Tempest’s one and implement the load_tests method, to make tests form the plugin discoverable by Tempest.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
import os from tempest.test_discover import plugins class CrossServiceTempestPlugin(plugins.TempestPlugin): def load_tests(self): base_path = os.path.split(os.path.dirname( os.path.abspath(__file__)))[0] test_dir = "cross_service_tempest_plugin/tests" full_test_dir = os.path.join(base_path, test_dir) return full_test_dir, base_path |
Then I wanted to make the plugin configurable. Since I wanted to be able to customise the DNS domain name, I needed a way to tell the test what domain name to use and expect. Tempest plugins allow extending the standard Tempest configuration file with plugin custom configuration groups and values.
The new configuration option was defined in a new module cross_service_tempest_plugin/config.py
|
1 2 3 |
cfg.StrOpt('dns_domain', default='my-workshop-domain.org.', help="The DNS domain used for testing.") |
I extended the CrossServiceTempestPlugin class two includes the implementation of two more methods from the Tempest plugin interface:
- register_opts is used by Tempest to register the extra options
- get_opt_lists is used for config option discovery, used for instance to generate a sample config file
The implementation is straight-forward:
|
1 2 3 4 5 6 7 8 9 10 |
def register_opts(self, conf): conf.register_group(cs_config.cross_service_group) conf.register_opts(cs_config.CrossServiceGroup, cs_config.cross_service_group) def register_opts(self): return [ (cs_config.cross_service_group.name, cs_config.CrossServiceGroup), ] |
The plugin interface allows plugins to extend existing configuration groups with new configuration items. Plugins associated with a specific service usually extend tempest ServiceAvailableGroup with their own service. This is how it’s done in the plugin for designate:
|
1 2 3 4 5 6 7 8 9 10 |
from oslo_config import cfg service_available_group = cfg.OptGroup(name="service_available", title="Available OpenStack Services") ServiceAvailableGroup = [ cfg.BoolOpt("designate", default=True, help="Whether or not designate is expected to be available."), ] |
It’s important to note that configuration groups extended this way should not be registered in register_opts nor returned by register_opts since they are already known to Tempest.
Further examples are available in the documentation.
The fourth and last method of the plugin interface is used to expose service clients. I already had all the required service clients from either Tempest or the existing plugins, so I didn’t need to define any new one.
I extended the CrossServiceTempestPlugin class with the following implementation of the service clients interface:
|
1 2 3 |
def get_service_clients(self): # No extra service client defined by this plugin return [] |
At this point, I was ready to add a test class an start writing the test code. I created a new module test_cross_service.py under cross_service_tempest_plugin/tests/scenario. The basic class definition is:
|
1 2 3 4 5 6 |
from tempest import test class HeatDriverNeutronDNSIntegration(test.BaseTestCase): def test_floating_ip_with_name_from_port_to_dns(self): pass |
With this code added to a git repo, I had an installable Tempest plugin, with configuration options and one discoverable test.
Since Tempest is installed in a python virtual environment by devstack, it was possible to test everything done to this point:
|
1 2 3 4 5 |
source ~/tempest/.tox/tempest/bin/activate pip install cross_service_tempest_plugin cd ~/tempest tempest run --regex test_cross_service --list tempest run --regex test_cross_service |
In the next blog post, I will explain how to write the scenario test in Tempest.