Methodology

 

All technology developed will be tested in practice in a large number of industrial pilots that the academic partners perform together with our industrial partners and their customers. This evaluation will feedback into the project itself, and will be used to further improve and extend the technology developed.

The industrial partners are actively involved in the development of the technologies and tools, and the services developed will be made available as open source products or commercialized. A large number of technologies will enhance existing offerings of our industrial partners and in that way already have a channel out into industry. Having typical role models of our target group as industrial partners in the PROWESS project, increases the adoption by other similar companies. Moreover we have set up an advisory board, composed of a number of additional companies, which have expressed an interest in participating in PROWESS activities. These advisory board members will provide further exploitation channels for our results, and will help evaluate project deliverables in industrially grounded pilot studies.

Exploitation of the results will be achieved via existing commercial channels, as well as via fairs and exhibitions, via research publications and conferences and via teaching of both academic courses and industrially-oriented tutorials.

The rationale for PROWESS

The top-level goal of PROWESS is to extend property-based testing for web services and internet applications in order to achieve a real improvement of testing effectiveness, so that more faults are found in an earlier stage, and efficiency, with less overall testing activity needed overall. PBT is delivered by an ‘ecosystem’ of tools, libraries, techniques and methodologies, with the QuickCheck tool from Quviq at its heart.

PROWESS and property-based testing, with their positions indicating how they extend and enhance the overall PBT workflow. The essential point of this is to highlight that all the different aspects of PROWESS build to the same goal, namely that of enhancing property-based testing.

To deliver these enhancements it is not simply a matter of enhancing the QuickCheck tool: as well as doing this, we will develop and extend other tools, techniques and methodologies. QuickCheck here is shorthand for the whole property-based testing ecosystem, which is illustrated here:

The PROWESS project will integrate these new methodologies, libraries and tools into a common framework for property-based development for internet services: the PROWESS property-based testing framework. The project will provide regular releases of the framework, conduct tutorial dissemination activities, and perform thorough industrially-driven evaluations of each release.

The concept of such a framework clearly makes sense: property-based testing requires good tools (QuickCheck and others developed in PROWESS), requires methodologies that provide effective advice on how to construct test models and effectively utilise the PROWESS tools to test systems against models, and last but certainly not least requires tutorials in the application of methodologies and tools. As methodologies will depend crucially on the availability of certain tool functionalities and libraries, releasing them together as part of a common framework makes precise the connection.

Framework releases

The combination of the QuickCheck integrated tool platform (including e.g. QuickCheck itself, libraries, Wrangler, and other open source tools) and the supporting methodology documentation will be packaged and released under appropriate licences two times during the course of the project.

Tutorials and dissemination

Initial tutorials on applying the property-based testing framework will be offered during the Erlang Factory events organized by the project partner Erlang Solutions Ltd. These events, which take place regularly every year in different European and world-wide locations, have proven to be excellent forums for dissemination through tutorials on property-based development. Although the name of these events indicates that the Erlang community is targeted, in reality the Erlang Factories attract a much wider audience of managers, technology evangelists, investors, and developers, with an interest in improving practises for developing complex systems in general, and almost all intrigued by the state-of-the-art testing technology offered by property-based testing.

Evaluation

Each platform release will be evaluated, as a whole, by industrial partners in industrial pilots.