Institutions | About Us | Help | Gaeilge
rian logo


Mark
Go Back
A new metric for patent retrieval evaluation
Magdy, Walid ; Jones, Gareth J.F.
Patent retrieval is generally considered to be a recall-oriented information retrieval task that is growing in importance. Despite this fact, precision based scores such as mean average precision (MAP) remain the primary evaluation measures for patent retrieval. Our study examines different evaluation measures for the recall-oriented patent retrieval task and shows the limitations of the current scores in comparing different IR systems for this task. We introduce PRES, a novel evaluation metric for this type of application taking account of recall and user search effort. The behaviour of PRES is demonstrated on 48 runs from the CLEF-IP 2009 patent retrieval track. A full analysis of the performance of PRES shows its suitability for measuring the retrieval effectiveness of systems from a recall focused perspective taking into account the expected search effort of patent searchers.
Keyword(s): Information retrieval; PRES; Patent Retrieval; Evaluation Metric
Publication Date:
2010
Type: Conference item
Peer-Reviewed: Yes
Language(s): English
Institution: Dublin City University
Citation(s): Magdy, Walid and Jones, Gareth J.F. (2010) A new metric for patent retrieval evaluation. In: the 1st International Workshop on Advances in Patent Information Retrieval (AsPIRe'10) at 32nd European Conference on Information Retrieval (ECIR 2010), 28 March 2010, Milton Keynes, U.K..
File Format(s): application/pdf
Related Link(s): http://doras.dcu.ie/16181/1/A_New_Metric_for_Patent_Retrieval_Evaluation.pdf
First Indexed: 2011-06-10 05:12:30 Last Updated: 2011-06-10 05:12:30