Citadel - Deliverable D6.1 Strategic Evaluation Methodology

Research output: Book/ReportCommissioned report


This document is designed to set out Citadel... on the Move's evaluation strategy. As described in Work package (WP) 6, it aims to produce systematic insight into the factors that underlie the usability experiences and technology acceptance of the Citadel... on the Move solution in particular, as well as the acceptance and value of the project in general. To this end, the unique characteristics of this project are combined with clear-cut evaluation methodologies guided by the Living Lab approach that has been customised to mirror the specific needs of Citadel... on the Move.
The aim of the Citadel... on the Move solution is to enable citizen developers to develop mobile applications using open access data provided by the local government administrations, and to demonstrate the interoperability of these applications with different, localized data sources. For this purpose, five mobile application templates were developed (WP4) and open data management was set up through the Open Data Commons (WP3), facilitating the implementation of services and mobile applications in the four pilot cities of Athens, Ghent, Issy-les-Moulineaux and Manchester. Prior to engaging in piloting activities, end-to-end testing will be conducted so as to ensure that all the components are working properly (e.g. interoperable) and can be used to develop and test the different smart services. Note that this will be particularly undertaken in WP 4 Innovation and Pilot Preparation, particularly in T4.6 Testing and Review.
In the context of the pilots, the templates for generating mobile applications will be evaluated first by an expert group of selected application (citizen) developers in order to achieve high quality standards. And once approved, the (modified or developed) mobile applications will be tested by users such as citizens and SMEs. A benchmarking exercise will be employed to evaluate the proof of concept (or, Shared Services Validation) in which the (extended) service catalogue of mobile applications will be shared between pilot cities and their end-users, and which underpins the evaluation of the overall project. The evaluation of the pilots is aligned with the operationalization of each of the pilots (as described in D5.1 Pilot Operations Plan), and takes into account the three distinct phases, respectively, the closed phase, open phase, and the shared services validation phase. The closed and open phase consist of two (iterative) cycles and the last phase of only one.
For the evaluation, we use both quantitative and qualitative methods, in order to provide an overview of the users' stances towards and usage of the solution and to capture suggestions for its further development and applicability. Participants will evaluate, among others, the easiness of use, usefulness, accessibility, quality of information, and attitude towards the solution. The results of the evaluation processes will be included in the upcoming WP6 deliverables, liaising with other work packages to collect and analyse relevant outputs such as in WP5 and WP2, and will be exploited both internally and externally towards the scientific and business communities so as to validate the Citadel... on the Move solution.
Original languageEnglish
Publication statusPublished - 2012


  • mobile applications
  • open data
  • governments


Dive into the research topics of 'Citadel - Deliverable D6.1 Strategic Evaluation Methodology'. Together they form a unique fingerprint.

Cite this