- Large language models can seem to reason or possess human-like intelligence.
- Humans who interact with AI may still fundamentally have difficulties in trusting AI systems.
- Eric Jing Du is collaborating with a UF psychologist to gain deeper insights into the mistrust in human-AI relationships
In a project that’s part engineering and part psychology, Eric Jing Du, Ph.D., is examining the trust gap between humans and the artificial intelligence systems on which they depend.
Rapid advances in AI systems, including large language models, or LLMs, have given rise to tools that can seem to reason or possess human-like intelligence. But here’s the problem Du is exploring: Humans who depend on LLM outputs — think warfighters — may fundamentally not trust the system.
Du, a professor in the University of Florida’s Department of Civil & Coastal Engineering, and his collaborator, Brian Odegaard, Ph.D., assistant professor in UF’s Department of Psychology, plan to gain deeper insights into the underlying causes of mistrust in human-AI relationships.
“As AI systems become increasingly capable of making complex decisions, the challenge is no longer simply whether AI can reach the right answer, but whether humans can understand and trust how that answer was reached,” Du said. “By studying the similarities and differences between human and AI cognition, we hope to design AI systems that are more transparent, predictable and aligned with human reasoning, ultimately making them safer and more effective partners in high-stakes environments.”
Part of the trust gap, according to Du, is that AI systems are largely perceived to be black boxes. Humans, even those who are working closely with automated intelligent systems, simply don’t know what’s happening inside the proverbial black box that is making decisions for and with them.
To explore the trust gap, and to clarify what’s happening inside the black box, Du and Odegaard are focusing on drone-assisted intelligence, surveillance and reconnaissance, or ISR, operations, a common scenario involving human-AI collaboration.
The research team hypothesizes that complex AI systems actually mirror human decision-making processes. Like a human reconnoitering unknown territory, an AI system controlling a drone in the same situation faces many of the same challenges. Both contend with conflicting objectives and unpredictable hazards, as well as possibly updating their beliefs based on new data or policies from leadership.
In the project, funded by the Air Force Office of Scientific Research, or AFOSR, Du and Odegaard are designing multi-expert AI systems to quantify AI’s decision-making stages, essentially modeling how models arrive at their decisions.
They then conduct human-subject experiments comparing human and AI decision making, measuring the differences between the two.
By running tightly controlled parallel human-subject experiments where participants perform the same simulated drone ISR tasks, the researchers can directly benchmark and quantify misalignments between humans and AIs at each stage of the process.
Based on their observations, they hope to improve future AI systems and align them more with human cognition styles.
For Du, the work is fundamentally an interdisciplinary effort. It treats advanced AI decision-making as a cognitive process rather than a purely algorithmic one.
“We approached the collaboration by importing and adapting core models from cognitive psychology and neuroscience,” Du said. “We aim to deconstruct exactly how the AI perceives its environment, updates beliefs, reconciles conflicting objectives and self-assesses its own decisions.”
For Du and Odegaard, the goal is to make AI “thinking” legible in human cognitive terms, which they believe is crucial for enhancing human trust in AI systems and ensuring their effective and ethical integration into society.