Be sure to catch the talk from Senior Researcher, OpenSearch Project - Aparna Sundar, PhD
“User Perception as it Informs AI Perception of Performance”
Testing is an important aspect of improving AI models. An untapped and less understood area of product development is to test results in a way that provides greater opportunities to frame AI responses. In this talk, I offer an example of user testing, take aways and how we improved the user experience of an AI Assistant. I cover discoveries at the early stages of development, the way in which users use design heuristics in evaluating AI. Users make judgments on information presented by AI in a way that can overwhelm, undermine or boost trust or confidence in the AI model. I conclude by providing design decisions that can help the user disambiguate the information offered by the AI model.