Releasing working software artifacts is always a challenge. Releasing a huge set of projects like OpenStack increases the difficulty. Luckily, the quality assurance (QA) project has been one of the pillars of OpenStack since its inception. Still, quality is really a shared responsibility in the OpenStack community, and each project should work with the QA team to actively follow, use, and locally adapt the testing tools and guidelines.
In this talk we’ll show how testing is organized in the Sahara world and performed in Sahara’s continuous improvement (CI) system. We’ll discuss the reasons for the creation of the separate Sahara tests repository and the new capabilities it allows. We’ll also share the design of the scenario tests framework itself (end-to-end), and we’ll talk about the consolidation of the Tempest-based and other high-level testing artifacts for related projects.