An approach for full scale off-line testing to evaluate the iWIDGET system performance.
Paula Vieira, Jose Barateiro, Dalia Loureiro, Joao Coelho, Aisha Mamade
Thursday 2 july 2015
14:35 - 14:50h
Themes: (T) Special session, (ST) FP7 ICT and water
Parallel session: 12I. Special session: FP7 ICT and water
iWIDGET is an ongoing European Commission FP7 project aiming at improved water efficiencies through the use of novel ICT technologies for integrated supply-demand side management, focusing on an integrated approach to water resources management. The project contributes to advance knowledge about smart metering in order to develop novel, robust and cost-effective ICT tools both for water utilities and consumers. Within the project, a set of relevant applications derived from smart metering real-time data was identified, characterized and implemented in a software prototype. This prototype is a critical asset that must be evaluated to ensure its compliance with the requirements specified during system analysis and design and to verify its acceptance according to end-users’ needs. The prototype evaluation followed the software engineering best practices, being done through a set of software tests with close collaboration between consumers, utility stakeholders and software developers. Once individual components (data management and analytical) have passed unit testing, integrated tests took place. The first phase of integrated tests used off-line historical smart metering data as opposed to the second phase (on-line testing) that uses near real-time data. This paper presents the standardized method designed in the project to carry out the off-line testing. The off-line testing method is based on a test scenario/test case approach and includes functional (i.e., tests to verify if functional requirements are met) and non-functional (i.e., test to verify the quality of the software in terms of usability, security or compatibility, for example) testing. For each application to be evaluated, test scenarios were designed as well as the corresponding test cases (in total, more than 50 test scenarios and 90 test cases). Success criteria for determining whether an observed behaviour of the system is correct and key performance indicators (KPI) to assess the achievement of success criteria were also defined for each test case step. This paper also presents results of the method´s application, which included collecting historical data from a full scale case study, feeding this data to the prototype, analysing the results and evaluating KPI in order to identify corrections and improvements.
