Institutions | About Us | Help | Gaeilge
rian logo


Mark
Go Back
A Proposal for the Evaluation of Adaptive Information Retrieval Systems using Simulated Interaction
MULWA, CATHERINE
The Centre for Next Generation Localisation (CNGL) is involved in building interactive adaptive systems which combine Information Retrieval (IR), Adaptive Hypermedia (AH) and adaptive web techniques and technologies. The complex functionality of these systems coupled with the variety of potential users means that the experiments necessary to evaluate such systems are difficult to plan, implement and execute. This evaluation requires both component-level scientific evaluation and user- based evaluation. Automated replication of experiments and simulation of user interaction would be hugely beneficial in the evaluation of adaptive information retrieval systems (AIRS). This paper proposes a methodology for the evaluation of AIRS which leverages simulated interaction. The hybrid approach detailed combines: (i) user-centred methods for simulating interaction and personalisation; (ii) evaluation metrics that combine Human Computer Interaction (HCI), AH and IR techniques; and (iii) the use of qualitative and quantitative evaluations. The benefits and limitations of evaluations based on user simulations are also discussed.
Keyword(s): Information Storage and Retrieval; Relevance Feedback; Intelligent Content & Communications; Smart & Sustainable Planet
Publication Date:
2010
Type: Conference item
Peer-Reviewed: Yes
Language(s): English
Contributor(s): LAWLESS, SEAMUS; WADE, VINCENT; SHARP, MARY
Institution: Trinity College Dublin
Citation(s): Catherine Mulwa, Wei Li, Séamus Lawless, and Gareth Jones, A Proposal for the Evaluation of Adaptive Information Retrieval Systems using Simulated Interaction, Workshop on the Simulation of Interaction in Automated Evaluation of Interactive Information Retrieval, SimInt 2010, at the 33rd Annual ACM SIGIR Conference, SIGIR, Geneva, Switzerland., July 19th-23rd, ACM, 2010, 5 - 6
Publisher(s): ACM
Alternative Title(s): Workshop on the Simulation of Interaction in Automated Evaluation of Interactive Information Retrieval, SimInt 2010, at the 33rd Annual ACM SIGIR Conference, SIGIR
First Indexed: 2014-05-13 05:16:10 Last Updated: 2015-07-15 05:45:00