Generative AI (GenAI) chatbots are becoming increasingly integrated into virtual assistant technologies, yet their success hinges on the ability to gather meaningful user feedback to improve interaction quality, system outcomes, and overall user acceptance. Successful chatbot interactions can enable organizations to build long-term relationships with their customers and users, supporting customer loyalty and furthering the organization’s goals. This study explores the impact of two distinct narratives and feedback collection mechanisms on user engagement and feedback behavior: a standard AI-focused interaction versus a hybrid intelligence (HI)-framed interaction. Initial findings indicate that while survey measures showed no significant differences in user willingness to leave feedback, use the system, or trust the system, participants exposed to the HI narrative provided more detailed feedback. These findings offer insights into designing effective feedback systems for GenAI virtual assistants, balancing user effort with system improvement potential.
Teaser
Can reframing chatbot feedback as a collaboration rather than a task make users more engaged? This small-scale study explores how Hybrid Intelligence narratives can turn feedback from routine interaction into a co-creative exchange — shifting focus from technology performance to human partnership.
Short Intro
Generative AI (GenAI) chatbots are becoming common in digital services, yet collecting meaningful user feedback remains difficult. This exploratory study, conducted by researchers at the Center for Hybrid Intelligence and Adobe Inc., investigates how a Hybrid Intelligence (HI) narrative—which frames feedback as a collaborative effort—affects user engagement. By comparing AI and HI framings of feedback, the study examines how shifting the story we tell users can make feedback more reflective, relational, and motivating.
The Research Challenge
Traditional AI systems define success through efficiency and technical accuracy, while overlooking how users experience collaboration with AI. The researchers proposed a shift from technology-oriented success metrics toward partnership-oriented measures, evaluating how “engaged,” “trusted,” and “co-creative” the human–AI relationship feels. This perspective acts as a prelude to the 4P Hybrid Intelligence framework, foregrounding the People and Partnership dimensions that drive meaningful AI adoption.
The HI Approach
To explore this idea, two feedback designs were tested in a prototype marketing assistant:
- A standard AI condition with simple thumbs-up/down feedback.
- A Hybrid Intelligence condition framed as collaborative improvement, explaining how user input helps the system learn.
A total of 19 marketing professionals (8 in the AI condition, 11 in the HI condition) participated. Their responses were analyzed both quantitatively and qualitatively. While no significant differences appeared in conventional survey metrics (trust, satisfaction, willingness to use), participants in the HI condition wrote longer, more thoughtful, and improvement-oriented feedback — indicating a higher willingness to contribute when framed as collaborators.
Findings
- The HI narrative encouraged a shift in user behavior, with participants showing increased willingness to contribute and elaborate on their input compared to the standard AI framing.
- Qualitative coding revealed more specific, constructive, and partnership-driven feedback (39% vs. 30%).
- These results demonstrate that framing feedback as co-creation can subtly reshape engagement, even without measurable changes in trust or satisfaction.
- The study thus positions Hybrid Intelligence not as a performance tool, but as a relational paradigm for human–AI collaboration.
Impact
This exploratory work marks an early step toward redefining AI success metrics around partnership and engagement rather than technical output.
By showing how a simple narrative change can increase users’ willingness to participate in improving AI systems, the study provides a conceptual foundation for more reflective, people-centered feedback design — and contributes directly to the broader vision of Hybrid Intelligence frameworks like the 4P Model.
Keywords: Hybrid Intelligence, chatbots, feedback, user engagement, partnership, co-creation, GenAI, reflection
Suggested citation: Rafner, J., Guloy, R. Q., Wen, E. W., Chiodo, C. M., & Sherson, J. (2025). From interaction to collaboration: How hybrid intelligence enhances chatbot feedback. arXiv:2504.13848. https://doi.org/10.48550/arXiv.2504.13848