Artificial Intelligence (AI) research has seen exponential growth in the past few years. With a global growth rate of over 180% in the number of papers published on AI in the ten years between 2013 and 2023, it is safe to say AI research has boomed in popularity.
However, the rise in interest in AI has also been coupled with public unease towards the technology. The field of responsible AI (abbreviated as RAI) broadly addresses the lack of trust in AI technologies, by understanding the risks AI poses and strategies to mitigate them.
This article addresses some of the key topics addressed in research on responsible AI and highlights the key takeaways from the research.
Ethics
Ethics is a term that is frequently associated with responsible AI. Broadly speaking, AI Ethics covers what human values AI models should follow and the values its practitioners should follow in its development and implementation.
AI Ethics is an increasingly active research field that has grown in response to rising concerns about the impact of AI. Due to its complexity and its interdisciplinary nature, we selected to highlight “A high-level overview of AI ethics”, by Kazim and Koshiyama.
The work focuses on providing a comprehensive overview of the current state of ethics research in AI. It provides good summaries of the intersection of ethics and other topics in responsible AI, such as privacy.
Among the various approaches to AI ethics is the “ethical-by-design” approach, which represents a commitment to building AI systems ethically, with the goal of preventing harm. One way that this can take shape is through interdisciplinarity in the design processes of AI systems, for example by bringing together experts from fields like philosophy, psychology, and law, in addition to the AI engineers, in the development stage to understand the ethical concerns and potential impact of the AI system.
This paper is easy to digest for non-experts and lays the foundation for contextualizing ethics in other areas of responsible AI. It provides easy-to-understand definitions and offers a great starting point for readers interested in learning more about the basics of AI ethics.
Explainability
Most modern AI models, notably deep neural network models, are often referred to as “black boxes” in the literature due to the non-transparent nature of how these models arrive at a result.
This lack of explainability is problematic, especially in high-risk settings, as we don’t know if a model utilizes biased reasoning to arrive at a conclusion (such as using factors like race or gender to determine if someone is eligible for a bank loan).
In this connection, tools for diagnosing models are becoming increasingly relevant. For this article, we selected the work of T. Räuker et al., “Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks”, as it offers a nice overview of the state of explainability research.
The paper details the main approaches used in explainability research, while also not being afraid of highlighting some of the failures of current research practices, such as cherry-picking results that align with the preconceived hypothesis of the paper.
The authors note that much of the research on deep neural networks tends to present successful applications of their methods, however, some methods may perform well only on a small number of “cherry-picked” examples. Focusing solely on best-case performance can mislead evaluations.
The authors provide several recommendations and directions for future research, namely, the need for rigorous benchmarks to assess interpretability methods, which should ideally measure how useful methods are for providing valuable insights. In addition, they argue that interpretability should be considered essential for AI systems deployed in critical settings. By better understanding how models make decisions and their potential shortcomings, many safety or ethical concerns regarding AI systems can be mitigated.
By addressing these topics, future work can help make interpretability tools more useful in real-world applications. Although this paper is quite technical, it is suitable for non-experts and gives a great introduction to the topic.
Alignment
Due to the “black box” nature of most modern AI models highlighted earlier, it is difficult to understand how models arrive at their conclusions. However, another danger to consider is whether these models actually learn to perform the task we want them to perform, instead of a different task that happens to align with our measured success metrics.
For a more human analogy, think of students who cheat on their homework. These students might still receive a high score on their assignment (the measured metric), but they are misaligned with the intended goal set by the class they are taking (learning the subject). This is, roughly speaking, the AI alignment problem.
As AI alignment is a broad research subject, we decided to highlight the paper of Hendrycks et al., “Unsolved Problems in ML Safety”. This paper gives a great overview of Machine Learning (ML) safety, with a very readable and beginner-friendly section on alignment in AI.
The authors propose a roadmap for ML safety, detailing the technical problems that must be addressed by researchers. The paper details four key problems in need of research:
- Robustness: withstanding hazards like adversarial attacks.
- Monitoring: including identifying hazards and model inspection.
- Alignment: ensuring that powerful ML systems have aligned goals.
- Systematic Safety: mitigating hazards arising from deployment
The paper also highlights several promising directions for future research. As the authors note, the machine learning community should shift its focus beyond accuracy and scalability to safety as a top research priority. In conjunction with the numerous citations to other quality papers, makes this a great starting point for newcomers to the field of AI alignment and AI safety.
Governance
AI governance is a keyword in the literature that is widely used, yet rarely defined. An important step to any successful research project is deciding what is being studied and outlining the context of the project within the existing literature. To establish context, researchers have to be able to agree on the meaning of terminology and methodologies, such that there can be consensus.
For this article, we decided to highlight the work of Corrêa et al., “Worldwide AI ethics: A review of 200 guidelines and recommendations for AI governance”, which undertakes exactly the task of defining AI governance. Due to the nature of the work, the paper explores a broad range of the AI governance research landscape to find salient themes between them.
A few takeaways:
- As a result of their meta-analysis of 200 governance policies and ethical guidelines for AI use from across the globe, the authors identified 17 recurring principles that emerged. These include terms like accountability, human rights, alignment, fairness, privacy, trustworthiness, transparency, and privacy.
- Among the policies and guidelines studied, there was less attention given to AI’s long-term safety impact. The authors found that terms like “safety” and “alignment” were often dismissed by the authors of the assessed policies and guidelines. This is part of a larger trend of AI safety not being a primary research focus. Only about 2% of papers submitted to NeurIPS 2021 were related to AI safety, according to the authors.
This paper gives a good overview of AI governance research and is a good resource for those looking to dive deeper into the literature.
Sustainability
Sustainability with regard to AI’s environmental footprint on our planet is becoming an ever greater concern. As a first step to addressing the problem, we first need to understand its nature and scope.
For this, we selected a report from the United Nations Environment Program (UNEP), on the environmental impact of the AI life cycle (“Artificial Intelligence (AI) end-to-end: The Environmental Impact of the Full AI Lifecycle Needs to be Comprehensively Assessed”). The report provides an executive summary of the current stage of research on AI’s impact on the environment, with links to other articles for further reading.
One interesting observation in the report is that “an AI query utilizing a Large Language Model (LLM) roughly uses 9 times more power than a conventional query using a search engine”.
This and other environmental concerns about the use of AI models present an opportunity for research teams and companies to develop more efficient AI models and tools.
Additional takeaways:
- When considering the environmental impact of AI, both the software and hardware lifecycles of AI must be taken into consideration. The software lifecycle covers the lifetime of AI models from data collection to development, training, deployment, and maintenance (all the way to retirement), whereas the hardware lifecycle includes the manufacturing of computer chips (including GPUs) which are crucial to the training and deployment of AI systems.
- Estimates on the water consumption of AI suggest that the global demand for water stemming from AI could reach 4.2 – 6.6 billion cubic meters in 2027. For context, this amount represents more than half of the annual water use in the UK in 2023.
Conclusion
In this article, we went over some of the key research topics in responsible AI, including some top papers and reports in the field. This list is not exhaustive, as there are more keywords and papers that are relevant in their respective fields. We focused primarily on texts that are relatively easy for non-experts to understand, and that let readers get familiar with the literature.
While responsible AI may seem like a relatively niche subtopic of AI, this research area will become increasingly important to ensure that the technology is deployed in a trustworthy manner.
Although the general public might not be aware of the research and discussion around responsible AI and AI ethics, the field of responsible AI is an active area of research that is expanding rapidly. With a broad area of applications, it will be exciting to see how responsible AI will continue to develop.
