As far back as I can recall, companies have struggled with their massive volume of verbatim comments from customer satisfaction and voice of the customer studies. We struggled to find a solution that was fast, technically powerful, yet precise in nuance and sentiment. Capturing that nugget of insight was not only time-consuming, but ever elusive.
Today, ChatGPT has changed that. I used two case studies and the results were highly impressive!
Study 1: I used 100+ real verbatim comments from a survey case study about dining habits and asked ChatGPT the following:
Me: These are verbatim comments from a survey where we asked respondents to tell us how their dining habits may change after Covid. Group these verbatim comments based on sentiment and key themes. Include the proportion for each group and include a crafty quote for each group that best articulates that key theme.
This is what ChatGPT gave me in less than 5 seconds - 5 key groups, the proportion for each group, and a great quote amplifying that group's theme. Incredible!!!
Study 2: I created 10 verbatim comments that were a mix of straightforward versus sarcastic comments to determine if ChatGPT was nuanced in grouping sentiment.
Me: These are verbatim comments from a survey where we asked respondents to "Please tell us why you were satisfied or dissatisfied with your experience with us?" Please group these verbatim comments by sentiment and key themes and include the proportion for each group. If there are any sarcastic comments, please list them and tell me what their sentiment really is.
[Comment 1: Loved your retaurant sooo much, I left a $1 tip]
The below response shows ChatGPT did fairly well, except for the comment "Loved your restaurant sooo much, I left a $1 tip" - which it put under "Satisfied", even though in parenthesis, it identified it as "Sarcasm, Dissatisfied". The parenthetical notes were created by ChatGPT. So of course I pressed for clarity below... in which ChatGPT apologized and corrected its misclassification.
Also, if a comment has both negative and positive feedback, give ChatGPT very specific instructions, such as "If a verbatim comment includes a positive and negative feedback, include that comment in both Satisfied and Dissatisfied groups and treat that comment as two responses, rather than one, in determining the proportion for each group."
Wow, overall, the computing power, speed and nuanced accuracy is truly a compelling game-changer. I would pay more than $20/month to have access to this during peak hours.
Next steps would be for ChatGPT to offer services that would ensure the data that is used or uploaded in its chat is privacy protected.