Statistical learning methods are commonly applied in content-based video and image retrieval. Such methods require a large number of examples which are usually obtained through a manual annotation process, that is human raters review images and assign semantic concept labels. The human judgement, however, cannot be regarded as the ultimate truth because of its subjectiveness and the likelihood of human error. We can address these issues by using multiple judgements per example, but evaluating and resolving disagreement between raters is problematic. Moreover, the nature of rater disagreement and how to minimise it are not yet well explored. In this paper we present results of a user study that was specifically designed to investigate human judgement of digital imagery. We discuss the influence of factors such as size and type of semantic vocabulary on inter-rater agreement. We demonstrate the application of latent class analysis for combining multiple judgements. Known from applications in themedical and social sciences, this statistic allows robust, quantitative evaluation of multiple judgements per subject. We believe it is well suited for application during the evaluation and modelling phase in semantic image and video retrieval.
机译:探索数字售货亭客户体验如何增强购物价值,自我心理图像和行为反应
机译:增强发现,探索性和访问数字航空影像收藏的机会
机译:另一角度看数字媒体中的“存在”体验:探索网真与心理意象的联系
机译:探索人类对数字图像的判断
机译:全球数字鸿沟:探索过去十年中国家核心计算和网络容量与人类发展进步之间的关系。
机译:与典型智障人士(ID)的人们探索精神图像和eidoiTic图像的生动性与典型发展(TD)个人相比
机译:另一点看数字媒体的经历:探索妖精与心理图像的联系