Design Principles for Responsible Use of AI to Enhance Customer Experience Using Public Procurement
Submitted by MKeegan on Tue, 01/16/2024 - 20:22
Tuesday, January 16, 2024
Co-Authors: Kayla Schwoerer, Andrea Patrucco, and Ilia Murtazashvili
This is an excerpt of Chapter 8 from our new book Transforming the Business of Government: Insights on Resiliency, Innovation, and Performance.
Government agencies are increasingly turning to artificial intelligence (AI) tools such as machine learning, chatbots, and generative AI to enhance customer experience (CX) and meet federal mandates for service improvement. While these tools offer significant opportunities to enhance public services and customer interactions, they also pose challenges,1 including bias, discrimination, and unauthorized data access. Addressing these challenges requires strategies that extend beyond conventional management practices and necessitates a comprehensive policy framework.
This chapter explores the service delivery process, including the systems, people, and processes that indirectly influence CX. It highlights the pivotal role the procurement process in acquiring and implementing ethical and responsible AI tools that enhance CX. It proposes two ways that public procurement processes can improve CX through AI and presents seven design principles to guide the procurement of AI tools in the public sector. Overall, addressing these policy issues and incorporating the proposed design principles into a comprehensive AI policy framework can unlock the potential of AI in public procurement, thereby enhancing CX, building public trust, and advancing agency missions.
The Complexity of Customer Experience
CX is sometimes viewed as a mere public encounter between a government representative and citizen at a specific point in time. Such a view masks the complexity of CX, missing insight into how agencies can contribute to a truly exceptional experience. Therefore, two core ideas guide our consideration of CX.
First, CX includes the entire service delivery process, encompassing not only the initial experience but also the culmination of systems, people, and processes that work together in the provision of public services. Second, the notion of the “customer” cannot be limited to individual citizens, residents, taxpayers, or beneficiaries of government programs and services, as there is a diverse range of “customers” involved in public service provision. Most significantly, CX depends on organizations that benefit from public programs and services, which means that vendors and suppliers can also be considered customers.
Embracing the complexity of CX means acknowledging that CX involves meeting individual and organizational needs to advance organizational missions, improve service, and build trust. Therefore, customer-centric approaches to public service delivery must consider a more holistic perspective that acknowledges each stage of the process and its influence on the customer’s perception, satisfaction, and experience.
From this holistic perspective, AI can play a key role in delivering a great CX. AI tools can improve service delivery, decision making, and policy in several ways. For example, AI’s ability to collect and analyze large amounts of data can help agencies tailor and personalize public services and communications. AI-powered chatbots and virtual assistants can provide quick and accurate responses to customer inquiries, reducing wait times, and improving CX. AI can also contribute to good governance by providing customers with greater transparency and insights into government decision-making processes, thereby promoting public trust.
Public procurement holds a pivotal role in ensuring the availability of essential goods and services for public service delivery, including AI technologies. This role is twofold, with procurement processes both contributing to the deployment of AI and benefiting from the use of AI.
First, procurement processes serve as the primary avenue for acquiring AI technologies.2 As such, procurement offices act as “gatekeepers” for the procurement, influencing the implementation and management of AI systems. They assess and select credible vendors that supply responsible and ethically designed AI systems. This gatekeeping role ensures that the pro- cured AI technologies are designed and implemented in a way that respects ethical guidelines, promotes transparency, and is user-friendly.3 for instance, procurement agencies can incorporate performance metrics related to user experience and customer satisfaction into contracts and RFPs. Also, this encourages suppliers to develop innovative solutions that meet agency specifications and enhance customer experience. This contributes to a positive CX by building trust in AI systems, making their decision-making processes understandable, and ensuring that the systems are easy to use and helpful for internal users and external stakeholders (such as citizens).
Second, AI tools can directly support the procurement process in the public sector.4 This support manifests in two distinct ways: operational efficiency and strategic decision making.
On the operational front, AI can facilitate efficient and effective public procurement processes, thereby improving agency operations and overall CX. For example, AI can automate and streamline procurement operations, reducing paperwork, improving response times, and leading to faster procurement cycles. AI can also assist in supplier selection and product evaluation based on selective criteria such as quality, innovation, and sustainability. By leveraging AI algorithms to analyze supplier capabilities, track records, and product features, agencies can make informed decisions that align with customer expectations. This results in the procurement and delivery of high-quality products and services, enhancing overall CX for both internal users and suppliers.
On the strategic front, public procurement plays a crucial role in promoting responsible practices that contribute to CX and public value more broadly. AI can automate the evaluation process, enabling agencies to efficiently evaluate supplier sustainability performance, conduct environmental impact assessments, and assess the social and ethical aspects of suppliers’ operations.5 AI can also quickly analyze supplier databases and certifications to aid in identifying and evaluating a diverse range of suppliers, including local businesses, minority-owned enterprises, and social enterprises. This not only promotes inclusion through public procurement but also fosters economic development and social impact. Moreover, AI enhances transparency in the procurement process by monitoring and analyzing data across the supply chain and tracking suppliers’ adherence to labor standards, environmental regulations, and ethical practices. Improved transparency helps identify potential risks and ensures that public procurement decisions align with broader public value.
This strategic use of AI in procurement can shape the overall CX by promoting a procurement process that is transparent, ethical, and integral, thereby increasing satisfaction among internal users, suppliers, citizens, and government officials.
Policy Obstacles: design Principles for the Procurement of Ethical, responsible, and Effective AI
Despite the clear benefits of AI in public procurement, extensive deployment in the public sector remains limited. This is largely due to policy issues that amplify the inherent challenges associated with AI, such as data security concerns, the complexity of decision-making processes, and the potential for bias and discrimination.6 The absence of comprehensive policies guiding the design, procurement, and use of AI exacerbates these challenges, creating a barrier to the effective deployment of AI in public procurement.
In the face of these challenges, a policy framework can guide the procurement of AI technologies in a manner that fosters a positive CX, promotes ethical and responsible use of AI, and advances agency missions. To this end, seven design principles should be incorporated into such a policy framework. These principles aim to address the key challenges associated with AI and provide a roadmap for the responsible procurement and use of AI in the public sector. Table 1 at the end of this chapter summarizes their characteristics.
Principle 1: Adhere to Ethical Guidelines and Principles
AI technologies procured by public organizations should adhere to ethical guidelines and principles, which can promote responsible use, transparency, and accountability. Such AI tools, once deployed, can better contribute to a positive CX by ensuring that AI systems operate in a trustworthy and morally acceptable manner. For example, when procuring an AI-powered facial recognition system that may help to improve CX by streamlining identification verification processes, procurement agencies can ensure that systems adhere to ethical guidelines by incorporating fairness measures to mitigate biases and facilitating transparency through explainable algorithms.
Principle 2: Prioritize Privacy Protection
AI technologies procured by public organizations should prioritize privacy protection since doing so can help ensure that individuals’ personal data are handled responsibly, protecting their personal data and privacy rights. This contributes to a positive CX by building trust and confidence in the organization’s commitment to safeguarding personal information. For example, when procuring AI-driven data analytics platforms, procurement agencies can ensure tools have strong data anonymization and encryption protocols to protect individuals’ personal information.
Principle 3. Address and Minimize Bias
AI technologies procured by public organizations should have protocols in place for addressing and minimizing the impacts of bias. Additionally, AI tools should have reliable methods for detecting potential biases and mechanisms in place to mitigate the harmful impacts of bias. By ensuring that AI systems do not discriminate or perpetuate systemic biases, organizations can promote fairness and equality in public service delivery.
For example, when procuring an AI-enabled recruitment system, procurement agencies can ensure that systems have undergone rigorous bias testing. Tools should also include features to identify and mitigate potential biases in the training data and algorithms to help avoid discriminatory outcomes and promote fairness in public sector hiring practices, enhancing the experience for job applicants.
Principle 4: Promote Interpretability and Explainability
AI technologies procured by public organizations should go beyond transparency to offer interpretability and explainability of AI-driven decisions. Interpretable and explainable AI can help empower customers by providing greater clarity about AI-driven decisions, increasing individuals’ understanding of and trust in the decision.
For instance, when purchasing an AI-based system for processing applications for social assistance, procurement agencies can ensure that systems offer transparent explanations for the decisions they influence. This level of transparency can help individuals understand the factors that influenced whether their application was approved or denied, foster trust, and empower them to contest decisions, as is their right when they feel decisions may be biased or inaccurate.
Principle 5: Maximize Sustainability and Minimize Environmental Impact
AI technologies procured by public organizations should consider the sustainability and environmental impact of the AI technologies procured. Such actions can contribute uniquely to CX since sustainable AI systems can help to reduce waste and resource consumption, thereby promoting more efficient and economical organizations and more innovative and responsible means of delivering public services.
For example, when procuring an AI-driven waste management optimization system, procurement agencies can ensure that the system is designed to minimize fuel consumption and carbon emissions by optimizing routes and schedules. This can lead to more efficient service delivery with reduced environmental impact, often at a lower cost to taxpayers.
Principle 6: Promote Social Impact and Inclusion
AI technologies procured by public organizations should promote social impact and inclusion since government serves a diversity of needs, abilities, interests, and perspectives. Improving CX, especially for those who have been historically underserved, requires a level of awareness and sensitivity to the needs and experiences of those whom public organizations are meant to serve. AI tools, and especially the data that powers them, should best reflect the features, and needs of the public in order to enhance CX, as such tools can provide more accurate insights for designing services that are tailored, accessible, and inclusive of customers’ needs.
For instance, when purchasing an AI-based language translation system for public schools, procurement agencies can ensure that it supports multiple languages and dialects, including those spoken by marginalized communities, to foster inclusivity and facilitate equal access to services.
Principle 7: Consider responsible vendor Governance
AI technologies procured by public organizations should be from vendors with responsible governance practices, as vendors with responsible governance practices can better ensure that AI solutions are designed with ethics, transparency, and accountability in mind. This contributes to CX by fostering trust in agencies and the services they deliver vis à vis the vendor’s commitment to responsible AI use and delivery of high-quality products and services. For example, procurement agencies should take responsible governance practices such as adherence to data privacy and security protocols into account when evaluating potential suppliers of AI technologies. Partnering with trust- worthy vendors can, in turn, enhance customer trust and satisfaction and ensure the responsible development and use of AI.
Looking Forward
The responsible use of AI in the public sector is not only crucial for enhancing customer experience but also presents a unique opportunity for positive transformation of agency performance. However, realizing this potential requires addressing the challenges that AI poses, which demand strategies that go beyond currently established practices and perspectives.
The first step to take in addressing this challenge is acknowledging the complexity of the full-service delivery experience, including the systems, people, and processes that contribute to a great CX. This includes recognizing the critical and unique role that public procurement plays in the deployment and use of AI.
Public procurement is instrumental in two key ways. First, procurement processes are responsible for the acquisition of ethical, responsible, and trustworthy AI technologies that can enhance CX by improving customer satisfaction and public trust. Second, the use of AI to streamline and improve procurement processes can have downstream effects that contribute to the overall CX. However, the potential of AI in public procurement remains largely untapped due to policy issues that amplify the inherent challenges associated with AI.
To address these issues, the seven design principles described in this chapter can guide the evaluation and procurement of AI technologies. These principles aim to foster a positive CX while promoting the ethical and responsible use of AI in the public sector. These principles are not meant to be exhaustive nor definitive but rather to serve as a starting point for the
development of comprehensive AI policies in the public sector. They provide a framework for assessing, evaluating, and procuring AI technologies, with a focus on promoting ethical, responsible, and customer-centric AI tools and their deployment.
The principles also prompt further empirical exploration to understand their strengths, weaknesses, and impacts on AI deployments in the public sector for CX. While there are numerous examples of successful AI integration in public organizations, there have also been numerous failed attempts. These efforts provide fertile ground for the development of case studies to better understand why the implementation of new technology policy tools is some- times unsuccessful. as such, they should be the focus of researchers and practitioners alike.
In conclusion, the ideas put forth in this chapter, particularly the seven design principles serve as a compass for agencies to navigate the complexities of AI adoption. They provide a roadmap for maximizing the benefits of AI, paving the way toward a future where responsible AI tools can help agencies deliver a great experience for customers, thereby building public trust and enhancing the relationship between government and the people it serves.
Endnotes
1 Hickok, Merve. “Public procurement of artificial intelligence systems: new risks and future proofing.” AI & society (2022): 1-15. https://doi.org/10.1007/s00146-022-01572-2
2 World Economic Forum. “Unlocking Public Sector Artificial Intelligence.” https://www.weforum.org/projects/unlocking-public-sector-artificial-intelligence/
3 Dor, Lavi M. Ben, and Cary Coglianese. “Procurement as AI governance.” IEEE Transactions on Technology and Society 2, no. 4 (2021): 192-199.
4 Chenok, Dan. “How Can Governments Use AI to Improve Procurement?” The Regulatory Review, June 30, 2022, https://www.theregreview.org/2022/06/30/chenok-how-can-governments-use-ai-to-improve-procurement/.
5 Ahmadi, Mohammad and Dr. Justin B. Bullock, Government Procurement and Acquisition: Opportunities and Challenges Presented by Artificial Intelligence and Machine Learning, IBM Center for The Business of Government, Washington, D.C., 2023.
6 Autio, Chloe, Kate Cummings, Brinson S. Elliot, and Beth Simone Noveck. “A Snapshot of Artificial Intelligence Procurement Challenges: Diagnosing perceived and actual risks impeding responsible AI acquisitions in government.” June 2023. The GovLab. https://files.thegovlab.org/a-snapshot-of-ai-procurement-challenges-june2023.pdf.
Previous Blog Posts on Transforming the Business of Government: Insights on Resiliency, Innovation, and Performance
Chapter One - Emergency Preparedness and Response
Chapter Five - Workforce
Chapter Six - Eight Areas for Government Action - Insights on Resiliency
Chapter Seven - AI Literacy: A Prerequisite for the Future of AI and Automation in Government