Blog 1: AI governance principles and rights in the cases of Barcelona and Montréal

06 September 2022

Over the past two decades, an increasing number of municipal administrations all over the world have embarked on city planning initiatives that involve the digitalization of urban space. Indeed, the use of information and communication technology (ICTs) to collect and process information about urban activities promises more efficient, sustainable, and safe cities. However, more recently, a number of cities have developed smart urbanism visions that distance themselves from a techno-centric approach and, instead, embrace a more human-centric one.

Initially, urban developers and city officials seemed to be particularly enthusiastic about the employment of technology, emphasizing the functional and economic benefits this would bring about in terms of urban management and economic boost. Among these, Barcelona and Montréal serve as prime examples of cities that have developed their own set of AI principles with the aim to guide the responsible development and usage of AI and data by their local administrations. 

Montreal Declaration of Responsible AI

Screen Shot 2022 09 09 At 2.37.33 Pm

An early example of a municipality developing a set of guiding principles, albeit not binding, for the development and deployment of AI for public purposes is the one of Montréal. In November 2016, the University of  Montréal launched a co-construction process with citizens to design the Montréal Declaration for a Responsible Development of Artificial Intelligence. This involved a series of public consultations and citizen assemblies with 500 citizens, experts, and stakeholders aiming to increase participation, transparency, and legitimacy of future AI-related policies. The Montréal Declaration particularly promotes the following principles:

  1. Well-being: The development and use of Artificial Intelligence Systems (AIS) must permit the growth of the well-being of all sentient beings.
  2. Privacy and Intimacy: Privacy and intimacy must be protected from AIS intrusion and data acquisition and archiving systems (DAAS).
  3. Respect for Autonomy: AIS must be developed and used while respecting people’s autonomy and with the goal of increasing people’s control over their lives and their surroundings.
  4. Responsibility: The development and use of AIS must not contribute to lessening the responsibility of human beings when decisions must be made.
  5. Democratic Participation: AIS must meet intelligibility, justifiability, and accessibility criteria, and must be subjected to democratic scrutiny, debate, and control.
  6. Equity: The development and use of AIS must contribute to the creation of a just and equitable society.
  7. Solidarity: The development of AIS must be compatible with maintaining the bonds of solidarity among people and generations.
  8. Diversity and Inclusion: The development and use of AIS must be compatible with maintaining social and cultural diversity and must not restrict the scope of lifestyle choices or personal experiences.
  9. Prudence: Every person involved in AIS development must exercise caution by anticipating, as far as possible, the adverse consequences of AIS use and by taking the appropriate measures to avoid them.
  10. Sustainable Development: The development and use of AIS must be carried out so as to ensure the strong environmental sustainability of the planet.

Barcelona's AI Strategy

Screen Shot 2022 09 09 At 2.37.19 Pm

A more recent example, then, is the one of Barcelona and its AI Strategy. In April 2021, Barcelona revealed its strategy for the ethical use of algorithms and data. Building off of previous smart city principles, such as the “Declaration for the proper development and usage of artificial intelligence” in 2017 and the  Digital City initiative pioneered in October 2016, the Barcelona AI Strategy is oriented towards improving public knowledge about the use of algorithms by public services and redistributing among citizens the economic and social value produced by their data. The AI Strategy outlines the following seven governing principles required for local AI projects to uphold: 

  • Human action and supervision: Any AI initiative that impacts residents must be overseen by humans to ensure that autonomous decision-making does not harm any one person. The level of human oversight is to be proportional to the risk that the technology used has to people.
  • Technical robustness and security: The City Council must be proactive in protecting and securing its technology from cyberattacks to prevent data leaks and data manipulation. Routine audits on the security of public AI are to be conducted.
  • Data privacy and governance: Across the entire data lifecycle, protecting and maintaining the privacy of residents must be upheld by collecting only the minimum amount of data and anonymizing data. Public data should be of high quality and published for public use, and, where possible, be accompanied by information about the data’s provenance and bias mitigation strategies. Moreover, data used by public bodies has to be bias-corrected before use and be employed for “legitimate and proportional” as well as GDPR-complaint reasons.
  • Transparency and information: The entire process undertaken by an AI system—from how data is labeled for an algorithm to the way in which an algorithm makes its decision—must be explained and made available to the public to trace the whole decision-making process. The use and limitations of AI need to be clearly communicated and the technology and its creators must be easily identifiable to the lay public.
  • Diversity, inclusion, and fairness: To prevent discrimination from publicly used AI, the City Council must hold citizen consultations to understand its impact on people and ensure digital accessibility via a “universal-design approach” that allows people with disabilities to wholly interact with the technology.
  • Social and environmental commitment: The ways in which AI is used should contribute to attaining the 17 Sustainable Development Goals outlined by the UN. For Barcelona, this means using smart systems to uphold and bolster democratic processes, elections, and civic engagement. 
  • Responsibility, accountability, and democratic control: When decisions are taken through AI, humans affected by these decisions should be able to get an explanation of why the decision is made in a language they can understand, and they should be able to challenge the decision with reasoned arguments. AI applications thus need to be intelligible and audited to ensure that their design and use do not harm residents.

Further Examples

The emergence of these guidelines is an encouraging sign that some cities are paying attention to how AI is being used and developed by local administrations in order to become laboratories of innovation in AI governance. Additional examples include data use charters to oversee the development and evolution of AI and algorithm within the public sector:

  • In 2021, the City of London unveiled its fourth iteration of its Emerging Technology Charter, a voluntary charter that outlines guidelines for testing and deploying “data-enabled smart city technology.” The Charter has updated provisions for using AI and data in light of COVID-19 recovery plans in a way that increases the transparency and explainability of data tools.
  • In 2019, the French city of Nantes unveiled a metropolitan data charter that outlined four commitments to data sovereignty of the community, data protection, data transparency, and data innovation. This charter was developed to increase citizen engagement and participation with the evolving use of data in public services.

Conclusion

The above illustrates the potential of principles to govern AI at the local level. Importantly, both Barcelona and Montréal highlight the presence and need for active collaboration and dialogue with non-governmental and grassroots organizations when designing community principles. Further, while these principles appear robust on paper, it is crucial to note the need for constant and critical reevaluation through a participatory approach to monitor and evaluate the success of these guidelines. To this end, we posit that re-evaluation and monitoring also be added to existing principles to ensure that other principles are current.

Moving forward, the AI Localism program will continue to analyze the growing use of AI within cities, looking at its developments and evolutions to ultimately deepen and leverage the AI Localism Canvas to map improving city technology and local governance of AI. 

***

We are deeply grateful to Beatriz Botero, Professor of Law at Sciences Po, and Mona Sloane, Sociologist at New York University and University of Tübingen AI Center, for reviewing this blog.

In our next blog post, we will explore how AI is procured by local public agencies and legislation around the use of AI for employment screening.