Applying Ethical AI Frameworks in practice: Evaluating conversational AI chatbot solutions
| Suzanne Atkins |
Artificial Intelligence
https://doi.org/10.54822/QXOM4114
This paper has been published in the “Computer and Society Research Journal“, a free to publish, open access journal for socially relevant computer science research.
PDF Download
Abstract
Ethical AI frameworks are designed to encourage the accountability, responsibility and transparency of AI applications. They provide principles for ethical design. To be truly transparent, it should be clear to the user of the AI application that the designers followed responsible AI principles. In order to test how easy it is for a user to assess the responsibility of an AI system and to understand the differences between ethical AI frameworks, we evaluated four commercial chatbots against four responsible AI frameworks. We found that the ethical frameworks produced quite different assessment scores. Many ethical AI frameworks contain requirements/principles that are difficult to evaluate for anyone except the chatbot developer. Our results also show that domain-specific ethical AI guidelines are easier to use and yield more practical insights than domain-independent frameworks. We conclude that ethical AI researchers should focus on studying specific domains and not AI as a whole, and that ethical AI guidelines should focus more on creating measurable standards and less on stating high level principles.
Accompanying data: Framework_chatbot_scores
How to cite this paper
We recommend you to use the following code to cite the paper:
@article{Atkins2021chatbots,
title={Applying Ethical {A}{I} Frameworks in practice: {E}valuating conversational {A}{I} chatbot solutions},
author={Atkins, S. and Badrie, I. and van Otterloo, S.},
journal={Computers and Society Research Journal},
number={1},
year={2021},
doi={\url{https://doi.org/10.54822/QXOM4114}}
}
It should look similar to this, depending on your template:
Atkins, Badrie and van Otterloo, 2021. Applying Ethical AI Frameworks in practice: Evaluating conversational AI chatbot solutions, Computers and Society Research Journal, 1, https://doi.org/10.54822/QXOM4114
Dr. Suzanne Atkins is a geophysics researcher and was consultant at ICT Institute from 2021-2022. She is now working at TNO as machine learning expert in Geophysics.