An Automated Quality Evaluation Framework of Psychotherapy Conversations with Local Quality Estimates

by   Zhuohao Chen, et al.

Computational approaches for assessing the quality of conversation-based psychotherapy, such as Cognitive Behavioral Therapy (CBT) and Motivational Interviewing (MI), have been developed recently to support quality assurance and clinical training. However, due to the long session lengths and limited modeling resources, computational methods largely rely on frequency-based lexical features or distribution of dialogue acts. In this work, we propose a hierarchical framework to automatically evaluate the quality of a CBT interaction. We divide each psychotherapy session into conversation segments and input those into a BERT-based model to produce segment embeddings. We first fine-tune BERT for predicting segment-level (local) quality scores and then use segment embeddings as lower-level input to a Bidirectional LSTM-based neural network to predict session-level (global) quality estimates. In particular, the segment-level quality scores are initialized with the session-level scores and we model the global quality as a function of the local quality scores to achieve the accurate segment-level quality estimates. These estimated segment-level scores benefit theBERT fine-tuning and in learning better segment embeddings. We evaluate the proposed framework on data drawn from real-world CBT clinical session recordings to predict multiple session-level behavior codes. The results indicate that our approach leads to improved evaluation accuracy for most codes in both regression and classification tasks.


page 3

page 10


Deep Annotation of Therapeutic Working Alliance in Psychotherapy

The therapeutic working alliance is an important predictor of the outcom...

Exploiting Session Information in BERT-based Session-aware Sequential Recommendation

In recommendation systems, utilizing the user interaction history as seq...

Automated Evaluation Of Psychotherapy Skills Using Speech And Language Technologies

With the growing prevalence of psychological interventions, it is vital ...

Automated Quality Assessment of Cognitive Behavioral Therapy Sessions Through Highly Contextualized Language Representations

During a psychotherapy session, the counselor typically adopts technique...

Speech based Depression Severity Level Classification Using a Multi-Stage Dilated CNN-LSTM Model

Speech based depression classification has gained immense popularity ove...

Leveraging Open Data and Task Augmentation to Automated Behavioral Coding of Psychotherapy Conversations in Low-Resource Scenarios

In psychotherapy interactions, the quality of a session is assessed by c...

PLDA with Two Sources of Inter-session Variability

In some speaker recognition scenarios we find conversations recorded sim...

Please sign up or login with your details

Forgot password? Click here to reset