Institutions | About Us | Help | Gaeilge
rian logo


Mark
Go Back
Statistically-constrained shallow text marking: techniques, evaluation paradigm and results
VOGEL, CARL
We present three natural language marking strategies based on fast and reliable shallow parsing techniques, and on widely available lexical resources: lexical substitution, adjective conjunction swaps, and relativiser switching. We test these techniques on a random sample of the British National Corpus. Individual candidate marks are checked for goodness of structural and semantic fit, using both lexical resources, and the web as a corpus. A representative sample of marks is given to 25 human judges to evaluate for acceptability and preservation of meaning. This establishes a correlation between corpus based felicity measures and perceived quality, and makes qualified predictions. Grammatical acceptability correlates with our automatic measure strongly (Pearson?s r = 0.795, p = 0.001), allowing us to account for about two thirds of variability in human judgements. A moderate but statistically insignificant (Pearson?s r = 0.422, p = 0.356) correlation is found with judgements of meaning preservation, indicating that the contextual window of five content words used for our automatic measure may need to be extended.
Keyword(s): information hiding, shallow parsing, web corpus, human judgement, correlation; Ageing; Cancer; Creative Arts Practice; Creative Technologies; Digital Humanities; Inclusive Society; Intelligent Content & Communications; International Integration; Smart & Sustainable Planet; Telecommunications; Computational linguistics
Publication Date:
2007
Type: Conference item
Peer-Reviewed: Yes
Language(s): English
Institution: Trinity College Dublin
Citation(s): Brian Murphy and Carl Vogel, Statistically-constrained shallow text marking: techniques, evaluation paradigm and results, Proceedings of SPIE - The International Society for Optical Engineering, Security, Steganography, and Watermarking of Multimedia Contents IX;, San Jose, California, February 2007, Edward J. Delp III, Ping Wah Wong, 6505, International Society for Optical Engineering, 2007, 65050Z
Publisher(s): International Society for Optical Engineering
Alternative Title(s): Proceedings of SPIE - The International Society for Optical Engineering -Security, Steganography, and Watermarking of Multimedia Contents IX;
File Format(s): application/pdf
First Indexed: 2014-05-13 05:24:33 Last Updated: 2017-04-26 07:51:55