Identifying Subjective Content Descriptions among Texts

摘要

An agent pursuing a task may work with a corpus of documents as a reference library. Subjective content descriptions (SCDs) provide additional information that add value in the context of the task. On the pursuit of new documents to add to the corpus, an agent may come across new documents where content text and SCDs from another agent are interleaved and no distinction can be made unless the agent knows the content from somewhere else. Therefore, this paper presents an Hidden Markov model (HMM)-based approach to identifying SCDs based on SCD and word distributions in a previously unknown text where descriptions occur inline among content text. We present a case study evaluating the performance of identifying inline SCDs in a text based on real-world and simulated data.