Tone

Understand Galileo's Tone Metric

Definition: Classifies the tone of the response into 9 different emotion categories: neutral, joy, love, fear, surprise, sadness, anger, annoyance, and confusion.

Calculation: We use a pre-trained RoBERTa model fine-tuned on the GoEmotions dataset and then subsample to the 9 common emotions. Achieves an F1 score of .58 on the full 28 emotion dataset, in our internal experiments it achieves ~80% accuracy on the 9 chosen emotions.

Usefulness: Recognize and categorize the emotional tone of responses to align with user preferences, allowing for optimization by discouraging undesirable tones and promoting preferred emotional responses.