Evaluating Non-Expert Stakeholder Interaction with Artificial Intelligence on Energy Urban Domain Using VIRTSI: The Case of ChatGPT

Abstract

This paper investigates the interaction of non-expert stakeholders with Artificial Intelligence (AI) in the energy urban domain, using the VIRTSI model and focusing on the capabilities of ChatGPT. VIRTSI (Variability and Impact of Reciprocal Trust States towards Intelligent systems), is a rigorous computational model for human-AI Interaction that simulates human trust states, spanning from overtrust to distrust, through user modelling and quantifies the efficiency of the interaction in VIRTSI-adapted confusion matrices. The research employed an 16-question survey, evaluating the accuracy and usefulness of ChatGPT's responses regarding energy consumption, cost-effective solutions, and renewable energy production for residential buildings in Greece. Each answer was assessed by human stakeholders who were non-expert in the energy Urban Domain (e.g house owners, building managers, etc.), who either accepted or rejected the responses based on validation processes. The analysis highlighted key aspects such as repetition, specification, and objections in the interaction with ChatGPT, offering insights into the effectiveness of AI in supporting energy-related decisions. The findings reveal that while AI can provide valuable information, user validation and expert consultation are critical for practical implementation. This highlights both the potential and limitations of integrating AI tools like ChatGPT in enhancing non-expert stakeholder engagement and decision-making in urban energy management, emphasizing the ongoing need for human expert involvement.