About PROWESS project

This project will develop advanced software engineering approaches to improve quality assurance: the challenge is to reduce time spent on testing, whilst increasing software quality, in order to quickly launch new, or enhancements of existing, web services and internet applications. We will develop this property-based testing for web services and internet applications in order to achieve a real improvement of testing efficiency.

Property-based testing

Property-based testing (PBT) will deliver more effective tests, more efficiently, and thus deliver economic benefits to the European software industry. Testing with properties as objects improves the competitiveness of software developers, as they can deliver higher quality software for a lower price. It allows collaborating companies to improve the definition of their software interfaces and so improve the compatibility between their services.

Project goals

The project will advance property-based testing into the domain of web services and other open, evolving systems.

Scaling the PBT approach

Work on PBT has concentrated on testing certain components and a subset of the functionality of these systems. We will develop methods to compose models into a single system model, develop techniques that use the same property models to mock components of the system.

PBT for web services

Creating properties and writing models is more abstract than writing traditional test cases; we will develop general techniques and methods that can be applied by a wide range of developers in web services and user-interface layer. In addition, we will develop techniques to extract properties from unit tests for web services. Reading properties can be a challenge for newcomers: so we will develop graphical and natural language renderings of properties to allow users to understand them more easily.

Dealing with multiplicity and evolution

In the open environment of the world wide web, it is natural that there will be multiple providers of services, and that these service provisions – both specifications and implementations – will evolve. This multiplicity gives the integrators of these services a set of challenges about how to choose between different providers, and how these choices work in an evolving environment. We will develop mechanisms and tools to support effective decision making in situations where there are competing implementations of requirements. We will provide users with models that describe the difference between implementations, or between an implementation and the requirements upon it, based on our models and toolsets.

PBT of non-functional requirements

Internet services, like any other software component, need to fulfil a number of non-functional requirements on top of the natural functional requirements. In particular for evolving, scalable systems, it is important that performance requirements can be met when popularity of the service dramatically increases. We will use property-based testing tools and techniques to test non-functional aspects of systems such as performance and dependability, including fault tolerance, safety and availability. These measurements will not only be absolute, but also relative to competing implementations. We will also develop a cloud testing framework to enable realistic testing of non-functional requirements, including stress testing and testing scalability for deployments of web services.

QA for property-based testing

We determine mechanisms to estimate the quality of PBT. First, with properties that span a large number of requirements and tests randomly generated from these properties, requirement traceability’s needs to be addressed in a novel way: our results will answer the question of whether a certain requirement is tested with the generated test cases. Secondly, new coverage metrics will record the different states visited in a model during testing, thus providing a measure on the total coverage of the model state. Thirdly, we will assess the quality of models and properties using the analogue of mutation testing.

AT A GLANCE

Project title:

Property-based testing of web services

Area:

Cloud Computing, Internet of Services and Advanced Software Engineering
(ICT-2011.1.2)

Project coordinator:

John Derrick, University of Sheffield (UK)

Partners:

University of Kent (UK)
CHALMERS (SE)
Universidad Politecnica de Madrid (ES)
Universidade da Coruña (ES)
QUVIQ (SE)
Erlang Solutions Limited (UK)
Interoud Innovation (ES)
SP Sveriges Tekniska Forskningsinstitut AB (SE)

Start date:

October 2012

Duration:

3 years

Total cost:

€ 4.42 M

Factsheet:

PROWESS_factsheet