Content Coding of Psychotherapy Transcripts Using Labeled TopicModels

摘要

Psychotherapy represents a broad class of medical interventions received by millions of patients each year. Unlike most medical treatments, its primary mechanisms are linguistic; i.e., the treatment relies directly on a conversation between a patient and provider. However, the evaluation of patient-provider conversation suffers from critical shortcomings, including intensive labor requirements, coder error, non-standardized coding systems, and inability to scale up to larger data sets. To overcome these shortcomings, psychotherapy analysis needs a reliable and scalable method for summarizing the content of treatment encounters. We used a publicly-available psychotherapy corpus from Alexander Street press comprising a large collection of transcripts of patient-provider conversations to compare coding performance for two machine learning methods. We used the Labeled Latent Dirichlet Allocation (L-LDA) model to learn associations between text and codes, to predict codes in psychotherapy sessions, and to localize specific passages of within-session text representative of a session code. We compared the L-LDA model to a baseline lasso regression model using predictive accuracy and model generalizability (measured by calculating the area under the curve (AUC) from the receiveroperating characteristic (ROC) curve). The L-LDA model outperforms the lassologistic regression model at predicting session-level codes with average AUCscores of .79, and .70, respectively. For fine-grained level coding, L-LDA andlogistic regression are able to identify specific talk-turns representative ofsymptom codes. However, model performance for talk-turn identification is notyet as reliable as human coders. We conclude that the L-LDA model has thepotential to be an objective, scaleable method for accurate automated coding ofpsychotherapy sessions that performs better than comparable discriminativemethods at session-level coding and can also predict fine-grained codes.