Description of the Clean City Index

← Back to Clean City Lab

There are many reasons why it is important for a city to measure its urban cleanliness. To track the progress and effectiveness of cleanliness policies and services; to have a cleaner, safer, more beautiful and vibrant city; to have a stable residential community with a well-improved infrastructure; and to be attractive as a tourist and cultural entertainment destination or an international business and innovation center. Whatever the objective, cities need metrics to measure their cleanliness against their goals.

Contrary to other domains such as road quality or winter road management, there are no standards for urban cleanliness. Historically, the first cleanliness evaluation methods were developed by urban cleaning services to manage their operations. Cleaning departments need to be able to measure their performance in order to evaluate, compare or improve their services. However, it is not the only reason. Later appeared other methodologies proposed by organizations independent from the cleaning departments to investigate the attractiveness of a city for economical or touristic reasons or for evaluating the quality of life — since city cleanliness and waste management as a pillar of urban sanitation, directly affect the quality of life. For instance, Slagin Parakatil, Principal at Mercer — an American asset management firm — says: "we continue to recognize emerging cities that are increasingly becoming competitors to traditional business and finance centers. These so called 'second-tier emerging cities' are investing, particularly in infrastructure to improve their quality-of-living standards and ultimately attract more foreign companies." Finally, some evaluation methods have been developed and used to monitor the progress and effectiveness of national or international (e.g. European) cleanliness policies, or to measure citizen satisfaction. For example, the European Commission regularly conducts a series of surveys as part of a study called "Quality of Life in European Cities" (2).

Different methodologies have been developed around the world to satisfy various needs and goals. Objective and subjective approaches made it possible to measure urban cleanliness. Since littering measurement is a necessary step to improve the cleanliness process in cities, Cortexia aimed to review the methodologies that have been used worldwide and test them in four cities of Western Switzerland. The results of this review helped to understand the aim, state-of-the-art, and comparability of existing methods. It was necessary to identify the underlying criteria and find common ground between the different methodologies. In this Clean City Lab article, we will provide a brief history on cleanliness measurement, compare some of the objective methods, and share the results of our practical experiment in western Switzerland.

History of attempts to measure cleanliness

Different methodologies have been developed worldwide and several approaches made it possible to objectively and subjectively assess urban cleanliness.

"Surveys", which have been developed to assess citizens' perception and impression of cleanliness, made it possible to assess the city cleaning performance subjectively. Although this method does not provide quantitative information on littering, it has been used as an evaluation method.

More objective methodologies such as city cleanliness "grading" or litter "counting" are the other methods that have been used to assess the cleanliness of urban areas. The New York City Street Cleanliness Scorecard Project, initiated in 1973, was the very first objective method that adopted a street cleanliness grading system. After that, in 2000, Waste Management and Recycling Zurich (ERZ) introduced the first cleanliness index based on litter counting. These and many other methodologies have provided municipalities, politicians and responsible organizations with a tool for managing and monitoring litter in urban areas. Among some objective methods we can mention the AVPU (Association des villes pour la propreté urbaine) method in France, the CROW method in the Netherlands, the INFA GmbH (Institut für Abfall-, Abwasser und Infrastruktur Management) method in Germany, the Local Environmental Audit and Management System (LEAMS) method adapted and used in Scotland and Wales, as well as the method used by the City of Miami Beach.

Interestingly, some studies show a representative correlation between subjective (surveys) and objective (ranking and counting) methods, demonstrating their reliability. For example, Ryzin et al compare the results of the well-established and reliable scorecard project with citizens' ratings of the cleanliness of their neighborhood streets. This comparison shows a clear and consistent correlation. The results of this study demonstrate that citizen judgments of government performance can closely match more objectively measured outcomes — and that citizen surveys can provide valid and useful performance measures, at least for some local government services. On the other hand, in a study conducted by the Department of Psychology at Humboldt University in Berlin in partnership with the VKU (Verband kommunaler Unternehmen), measures and criteria considered important by city cleaning departments are tested against citizen perceptions. This empirical study, which consists of surveys and eye movement tests, lends credibility to objective methods or helps to improve them.

Comparison of some existing clean city indices

For a better understanding, we have separated the methods into three categories: surveys, counting systems and grading systems. We have tried to construct a rationale for comparing these methodologies. Here we decided to compare methodologies that are based on counting or rating.

In the grading system, cleanliness is normally rated globally and the littering types are not recorded. A group of trained panel members visit the pre-defined zones of the city periodically and assign a numerical rating for the actual street condition at each visit, according to a set of photographic standards and descriptions. Depending on the methodology, they may consider a weighted index to account for the importance of the waste types. For instance, in the case of the New York scorecard, there are seven levels of street cleanliness, defined as follows:

In the counting system, detailed information on how to count for each type of litter and how to choose the audit points are described to teams of trained observers. For example, AVPU (Association des villes pour la propreté urbaine) in France uses a counting methodology where the trained auditing panel records and registers the information on the site, number and type of litter, and the surface under survey manually in an evaluation table.

In the paper presented, Cortexia compares the objective methodologies of: AVPU (Association des villes pour la propreté urbaine); Scorecard project of New York City; INFA (Institut für Abfall-, Abwasser und Infrastruktur Management); Swiss methodology developed by Zurich; Local Environmental Audit and Management System (LEAMS); Pilot SQI index in Wales; and CROW methodology used in the Netherlands. The approaches each method uses change according to the objective and the local build-up of the methodology. Our evaluation compares:

• The location where each methodology performs the measurement; the target areas: district, city or country; the local perimeter: street, sidewalk, parks, etc. and their exact definition; as well as the selection of the locations to be audited.
• Time of the day, week, or month at which the audits are performed; the frequency, and the way that the audits are planned.
• The goal of the methodology (informative and/or improvement tool); the type of the methodology (grading/counting); littering types that have been considered; and additional recordings and goals.
• How the index is calculated; what the index means, how it is scaled and weighted in relation to litter nuisance, citizen perception, etc.

Comparative table of methodologies

*Some methodologies assign the measured area to zones. These zones can be regions (e.g. town centers, high- or low-density residential areas) or local types of surroundings (e.g. road sections, squares, public transportation stations, pedestrian subways, waste collection points or parks).

In the different methodologies, depending on the goal of the measurement, the frequency of the measurements and the locations chosen for the audits differ. In most cases, the frequency is once or several times a month, except for national studies (e.g. LEAMS) where audits are carried out once to three times a year. Measurements are more frequent in methodologies where the goal is to improve the cleanliness of the city and the measurement data are used directly for operations. All methodologies are based on a discrete number of locations to be audited and there is little information available on how these audited areas are defined. The boundaries of the areas to be audited also differ from study to study. For example, AVPU and LEAMS consider one side of the street, including the sidewalk and some gutters, but excluding the parking lot and road. Some, such as New York or the Wales pilot project, assess the entire street, and in the Swiss methodology, the majority of assessments are made outside of streets and squares, in places such as crosswalks, underpasses, waste collection points, public transport stations and promenades.

Some types of litter, such as dog feces and glass shards, are considered more offensive and can pose safety risks. Some methodologies take this into account and include a measure of weighting by waste type based on perception or safety impact studies (AVPU, INFA, Swiss, SQI).

Test of some methodologies in western cities of Switzerland

In order to compare the different indices, Cortexia decided to test some of these methodologies in 4 different cities in western Switzerland: (1) City A — a touristic town; (2) City B — an industrial municipality; (3) City C — a cantonal capital; (4) City D — a small countryside town.

In each city, we selected similar zones: a street between the railway station and a neighboring school, a pedestrian zone, a swimming pool and a shopping area. The audits were conducted under the same weather conditions between July 1st and 15th, 2016. The different methodologies were used on the same day one after the other in a given city.

Indices achieved by application of five different methodologies in four cities in western Switzerland

Rankings obtained by the different methodologies

The evaluation of the results shows that different methodologies led to different ranking orders among the studied cities. Although most of the methodologies agree on City A's ranking, this is not the case for the other cities.

A simple, easy to measure, and cost-effective Clean City Index

Reviewing the existing methodologies, it is evident that there are fundamental differences between the approaches (considered litters, location and time of audits, index calculation, etc.). This makes the comparison of indices not straightforward. This has been shown practically in our study on some western Swiss cities, where the rankings achieved by the different methodologies were not in agreement with each other.

On the other hand, not all methodologies share the same goal. For example, surveys are commonly used to assess quality of life. Other, more objective methods are used as a tool by sanitation departments to develop policies, plan and evaluate citywide operations; by municipalities to track and monitor the work of cleaning departments over time; or by community councils and other public interest groups to learn about cleanliness conditions in local neighborhoods. The common goal of most cities and organizations is to assess the level of cleanliness of a city in order to improve it. It therefore makes sense to have a common standard methodology. In line with the Clean Europe Network, we hope that the development of a common strategy for measuring the cleanliness of cities will allow cities and countries to compare and share best practices and define environmental policies in a more comprehensive and comparable way. However, it is very difficult to get all countries to change and agree on a common standard.

In this regard, by reviewing existing methodologies, we have attempted to develop a simple, practical, and cost-effective method for measuring and monitoring litter. The method we have developed is mainly inspired by the Zürich methodology, as we have found this method to be very effective. The unique feature of our solution is that it is the first solution in the world to manage city cleanups based on objective, real-time litter data. Mobile cameras combined with AI algorithms provide information on the type, location and quantity of litter. In addition, a weighted index ranging from zero to five and a user-friendly platform make data interpretation and presentation very easy for users. More detailed information on our clean city index can be found in our previous Clean City Lab article: Description of the Clean City Index.