All Smart City projects need tomonitor and evaluate their activities. But what do they actually mean with this? With this article, we will explain what monitoring and evaluation means and why those are crucial in a project, as well as illustrate some related challenges.
ATELIER is piloting a range of innovative solutions to create Positive Energy Districts (PEDs): technological, economic and social ones The PED pilots and our other project activities would be useless unless we monitor and evaluate their. Did the solutions work? If not, why not? And what lessons can we draw from them?
Evaluation areas in ATELIER
Key Performance Indicators (KPIs) capture the main impact categories so that they can be monitored correctly and especially so that targets can be set. However, the use of KPIs has its limitations: for instance, impact in some areas requires a different evaluation approach, and sometimes, it is difficult to express the kind of impact with only a few indicators. For this purpose, we will complement the performance based evaluation (using KPIs) with reflective evaluation approaches.
Reflective evaluation approach to monitor the PED Innovation Ateliers
We will illustrate the difference in KPIs by looking at two areas: energy performance and social impact & citizen engagement.
Energy KPIs
Energy flows in the PED will be monitored by metering at all the right places (measuring production and demand).. The performance of the pilots regarding these indicators is crucial, as an important goal of a PED is to be energy positive.
KPIs for social impact and citizen engagement
KPIs in the social domain have quite a different character. Social impacts and the results of citizen engagement cannot be fully captured by numbers only. For instance, the participation of stakeholders in a meeting does not say much about the actual impact of the meeting. Therefore, the social indicators need to be high level and more aggregated. They capture the PED’s impact on quality of life, energy citizenship, changes in behaviour, and the level of satisfaction. Social sciences provide us with the methods and tools that we can use to monitor progress, such as surveys and interviews. It is important to ensure that the people you ask are representative for the whole target group.
In ATELIER, we have used the experience of previous projects as a basis for our own monitoring and evaluation framework, as well as for our choice of KPIs. In the coming period, we will discuss these KPIs with all ATELIER partners and with our beneficiaries, including citizens and the cities. We need to make sure that these KPIs are feasible, transparent and that they comply with the way cities and other stakeholders view urban transitions.
Our approach to monitoring and evaluation will be published soon on https://smartcity-atelier.eu/
Mark van Wees, Amsterdam University of Applied Sciences