Module 5 - Impact
Assessing the progress and impact of intercultural integration policies and programmes is an issue at the core of the Intercultural Cities programme, and a thinking in constant evolution. Monitoring and evaluation of both the tools and the local intercultural strategies are critical to achieve the intercultural vision, develop strategies and initiatives that are evidence-based, and maintain trust between residents and authorities.
Furthermore interculturalism is based on the idea that, while it is important to close the gap between newcomers and older residents in terms of level of education, employment, health, housing etc; symbolic, emotional, psychological aspects associated with living in society such as sense of belonging, of being valued across cultural difference, community cohesion, attitudes to diversity, levels of perceived discrimination, are also extremely important dimensions of integration. Therefore, attention should also be devoted to monitoring and evaluating attitudes and perceptions of diversity and wellbeing.
In this module on monitoring implementation and measuring progress we will look at the following topics:
- Collect data
- Identify indicators
- Monitor progress
Monitoring implementation and measuring progress
Given the changing nature of diversity and the local context, monitoring and evaluation of intercultural practices is a complex task for cities and the Intercultural Cities community. Cities have to overcome several challenges such as the lack of available data (including disaggregated data, quantitative and qualitative), lack of or shrinking budgets, and some methodological issues (such as identifying relevant indicators or establishing causality).
Next, please read pages 54-55 of the guide The intercultural city step by step. You can also save the document if you wish to return to it later.
There are three main activities you need to undertake when assessing your progress:
5.1 Collect data
An ideal intercultural city has systems and mechanisms in place to collect quantitative data, qualitative data and evidence in all the areas presented here and beyond, according to the local context. This way, the city develops a better understanding of existing gaps and opportunities. While there is no need to replace existing monitoring and evaluation systems, an intercultural city will have to explore how they can adjust to apply an intercultural lens.
This will affect the way, and the type of data, which should be collected in order to assess the situation, identify the issues, gaps and drivers of change. This will also impact the way and the types of indicators identified.
To make sure evidence collected reflect the reality, particularly of disadvantaged groups and of those whose conditions often remain hidden from the authorities and services (such as people without documentation), data collection in an intercultural context requires innovative approaches and involvement of various actors reflecting the diversity of the city.
5.2 Identify indicators
The use of core indicators is important to allow local policymakers to set and monitor targets for a better implementation of their local intercultural policies. Cities have a number of indicators at their disposal while developing their intercultural monitoring and evaluation systems. Cities can either use or combine indicators developed by other cities or organisations, or co-design their own indicators with local communities.
The Intercultural Cities Index is as complementary tool, capable of visually illustrating the level of achievement of each city, progress over time, and enabling comparison with other cities. Although the Intercultural Cities Index is not intended to be a scientific tool, nor a ranking tool, it contains a number of strong indicators which make it easier to identify and communicate where a city stands in relation to intercultural integration, where effort should be focused in the future and which other cities could be a source of good practice in these particular areas.
Cities are encouraged to measure the outcomes and impact of their overall intercultural policies. This should be done by collecting and monitoring demographic data regarding access to services and the closing gap between newcomers and older residents in terms of education levels, employment, etc. Perceptions and attitudes of the resident population are also important and relevant indicators to this end.
5.3 Monitor progress
The Community-Based Results Accountability (CBRA) approach developed by the Washington Centre for the Study of Social Policy is a useful tool for structuring the development of the intercultural strategy and monitoring results.
The principle of results based accountability means that public agencies measure their performance not on the basis of the efforts they make to address a social issue, such as good community relations, but on the basis of results on the ground, which are agreed with the participation of all community stakeholders. Targets are set and indicators for success are identified through a community consultation process and results are constantly monitored. Solutions to issues are imagined collectively and involve the participation of agencies but also families and the broader community. CBRA is not a management tool, but a tool to mobilise people and public agencies towards a common goal.
Types of indicators
- Turning the curve: show progress towards the goals over a certain period of time, from the baseline.
- Performance indicators: how well you are performing on the tasks will help you to turn the curve.
- Accomplishments: positive activities not included above – for instance, we opened a community centre.
- Anecdotes: even if you are not able to achieve a critical mass, at least tell the stories behind the statistics that show how individuals are better off.
The CBRA process involves the following steps:
An Intercultural Task Force, in co-operation with the Forum of Intercultural Champions, the body steering the development of the intercultural strategy, also carries out the CBRA procedures. The first step would be to provide them with information/training on the CBRA approach and convince them of its usefulness.
Selection of results: on the basis of the results of the intercultural mapping, and in broad consultation with organisations and citizens, identify diversity management goals (or re-state the goals defined by the city council if this has been done through an inclusive participatory process), define priorities and obstacles to reaching the goals. Goals identified by the process might not, in the first instance, be related to diversity, but when discussing the issue people might realise that cultural isolation or prejudice are obstacles to reaching these goals and decide to address these problems.
Establish indicators to measure the achievement of goals: indicators need to be measurable, but the measurements could be very loose, for instance how security staff in some public spaces feel that conflicts between people from different cultural backgrounds have diminished. There should not be too many goals and indicators. Strong, welcoming and diverse cities are the general goals identified. In many cases, the success of the programme will actually be its steady progress over 2-3 years.
Tips for your intercultural strategy
- Establish data sources and data collection processes based on the indicators to support the monitoring process. Ensure that the data are both quantitative and qualitative.
- Develop relevant indicators. An indicator is a measure that helps quantify the achievement of a result. A good indicator should be based on common sense as well as compelling (communication power), say something important about the result (i.e. not be marginal), and have data power (consistent data to measure the result). It is very important to discuss the extent to which perceptions are relevant indicators. For instance, “what does it mean for you to be a diversity-friendly community”?
- Involve diverse actors to identify relevant indicators and review.
- Establish the baseline data that you will be using as a starting point to measure your progress later. This data should demonstrate the efforts of people such as youth and social workers who try to make communities more cohesive and often do not see their efforts reflected in city-wide indicators.
- Select strategies: define the actions through which results can be achieved.
- Design financing strategies, partnerships with various institutions and organisations.
- Develop an accountability system (design the procedure for communicating with the stakeholders, reporting, making changes to the strategy and the system etc.). Both success and lack of it should be reported, for instance at community summits, through newsletters etc.
Next, please read pages 54-57 of the guide The intercultural city step by step. You can also save the document if you wish to return to it later.
- Guide: Building Intercultural Cities with citizens the Community Based Results Accountability Approach
- Barometer: Barometer on perceptions and attitudes towards the population of foreign origins (Basque Observatory on Immigration)
- Study: How the intercultural integration approach leads to a better quality of life in diverse cities (Migration Policy Group)
- Toolkit: Intercultural Cities Index
- Video: How to fill in the index?
