Sentiment and Topic Analysis

Note: this is only available for the Professional Analytics User License and requires Credits. Data processing takes place within the EEA region and is GDPR Compliant.

To truly uncover deeper insights about the provided answers of your target group the use of Open Questions is instrumental to your surveys. The essence of the motivation for provided answers contribute to a more profound understanding of the underlying factors that are of influence. The analysis of open answers can be very time-consuming and complex. Survalyzer Next Generation offers the perfect solution: Open Text analysis through our AI integration with ChatGPT.

Within the Analyze section of your survey, the AI sentiment and topic analysis option can be checked:

This will start the Wizard which will guide you through all the steps and options:

1. It can be determined whether all interviews should be analyzed (retroactively and for new interviews) or only for new interviews. Please note that every time that you select ‘All interviews’ the analysis will be executed again and credits will be deducted for the entire data set.
2. Anonymize text: when selected, AI will detect personal identifying information like email addresses, phone numbers or postal addresses and replace them with placeholders for the transformed text version. See anonymization for more information.
3. Translate text: when selected, AI will translate the open comments to the desired language. It can be used solely to translate results from one language to another, or from multiple languages to one language. See translation for more information.

When either the anonymization or translation option is used, a custom variable will automatically be created that stores the transformed data. This will be ‘[QuestionVariable]_azure_transformed’ in the raw data.

In the next step 2: Questions one or multiple questions can be selected that should be analyzed. Only (semi) open questions are available for selection:

Per selected question it can be determined whether Sentiment and Topic analysis are desired:

Sentiment Analysis

Sentiment analysis is the process of analyzing digital text to identify whether the emotional tone of the answer is positive, negative, or neutral. These valuable insights are key to improving customer service, evaluating products and increasing brand reputation. 

When the ‘Sentiment Analysis’ option is checked for a question, this will automatically create a custom variable ‘[QuestionVariable]_azure_sentiment’. The open answer will be analyzed and this will result in three potential outcomes:

1. Positive sentiment
   Value = 1
2. Neutral sentiment
   Value = 0
3. Negative sentiment
   Value = -1

Below an example for each scenario is shown from the raw data:

Tip: On the Analyze page you can select Manage code plan to the define the labels of the values.

For instance you can add emojis (⊞ Win+. or Ctrl+⌘ Cmd+Space bar ) to make it more apparent:

In Professional Analytics the table is displayed as below:

In conclusion, the sentiment analysis allows you to easily generate a percentage distribution chart based on the textual input of your respondents in open questions:

Topic Analysis

Topic analysis detects topics in a text with advanced machine learning algorithms that involves counting words and finding and grouping similar word patterns. This allows you to gain insights in what topics are referred to most frequently related to for instance your service, product or brand. Additionally it shows whether your respondents are more positive or more negative towards specific topics, allowing you to easily identify improvements.

In step 2 Questions of the Wizard ‘Topic Analysis’ can be enabled and topics can be added. The open answers will be attributed to one or more of the defined topics based on the content of the response. For every topic that you define, a custom variable will be created in the following form: ‘[QuestionVariable]_azure_topic_[TopicName]’.

In the example below you can find some examples relating to product and service evaluation:

Tips:

1. Be as specific as possible: for example “Appearance” gives better results than “Design” for visual appearance because “Design” could also regard usability or quality of a product.
2. Topics should be mutually exclusive: for example “Appearance” and “Customer Support” are a good set example, but “Customer Support” and “Service” are closely related, which causes AI to assign a comment to both topics in the majority of cases.
3. Do not use topics like “Other”. Every topic analysis has a “No Topic” variable that is assigned when no topic from the list can be matched to the open answer.

The open answer is analyzed and if it matches specific topics this results in the value 1 in the corresponding column(s):

In Professional Analytics the results are reflected in a table with a check mark:

Based on the textual input from your respondents you can create a chart showing what percentage of answers relate to the specified topics:

And when combined with the Sentiment Analysis it provides actionable insights on where you can improve:

Anonymization

Collecting qualitative feedback enables you to understand people’s opinions, attitudes, and experiences. The freedom that is provided to respondents is very powerful, but also allows inclusion of personal data. Survalyzer offers the option to anonymize personal data. AI will detect personal identifying information like email addresses, phone numbers or postal addresses and replace them with placeholders for the transformed text version.

For instance a respondent can include a phone number and email address in the original response:

The anonymized (transformed) data will replace the personal data with the placeholder ‘[redacted]’:

Translation

In case that you work with a multilingual survey, you will collect open answers in different languages. Previously you would have to analyze these answers separately, but now you can choose to translate the input to one language and have everything in one place. In step 1 General of the Wizard you can determine to which language you would like all input to be translated.

Please note that the translation only works if the language of the open answers matches the survey language.

When you collect answers in for instance both English and German and you enabled translation to English, you will find all collected (translated) answers is the ‘[QuestionVariable]_azure_transformed]’ column:

In Professional Analytics you can for instance work with German input, translate it to English and combine it with Sentiment and Topic analysis. This would result in output as can be found below:

Summary & Analysis

When you have determined the general settings and the questions that should be analyzed, the Wizard will guide you to step 3 Summary. Here an overview is provided showing:

1. Whether ‘All interviews’ or ‘New interviews’ will be analyzed
2. The ‘Anonymization’ and/or ‘Translation’ settings that are applied.
3. The amount of credits that will be deducted based on the number of open-text answers:
   – If an open question is not answered, no credits will be charged.
   – The number of selected settings do not affect the number of charged credits.
   – When you already analyzed ‘All interviews’ and you edit and reactivate ‘All interviews’ again, credits will be deducted for all open-text answers again.
4. The costs per each new open-text answer that is analyzed. Survalyzer charges the industry standard costs of €0.08 per analyzed answer.
5. An overviewof all questions and additional settings that are selected.

When you are satisfied with the setup, you can click on the submit button and you are navigated to step 4 Analysis. This will show all variables will be created:

Depending on the number of answers that need to be analyzed, it might take a few minutes and then you will see in the right sidebar of the Analyze page:

1. View the current settings of your Sentiment and Topic analysis.
2. Edit the settings of your Sentiment and Topic analysis, this will reopen the Wizard and all steps can be edited before you reactive the analysis.
3. Stop the data analysis. This will not affect historical data, but will prevent new interviews from being analyzed.

Professional Analytics Dashboard

In conclusion, AI Sentiment and Topic analysis opens the door to a whole new world of data analysis. Just one open question can already provide you with valuable and actionable insights that enable you to keep improving your service, product or brand. Survalyzer offers you the option to save time (and money) to analyze textual input and have non-biased results.

An example of a Professional Analytics Dashboard that is created using AI Sentiment and Topic analysis can be found below: