AI Interview Quality Control

Prerequisites: one or more trap questions
The trap condition score is determined by the extent to which an interview falls into so-called trap questions. A trap condition is specifically designed to have only one correct answer. When a respondent steps into a trap once, this does not mean an immediate categorization. However, when two trap questions are triggered, an interview will score 100%.

Examples of trap questions are:

• “Select option two”. A simple trap question. A quality interview will have selected option two, as asked. But a bad interview, or a bot, can fall into this trap and select a different option.
• The second example of a good trap condition is a combination of two questions. For example: One single-choice question will be  “what is your age?” (Q1). We later in the questionnaire place another single-choice question in which we ask “what is your age group?”(Q10).

A good interview will have harmonized answers between Q1 and Q10

What is your age? à 21
What is your age group? à 18 – 25

A bad interview will have discrepancies between the two, and will therefore be flagged.

What is our age? à 21

What is your age group? à 50 – 65

Prerequisite: 20 or more completed interviews
The speeder score is calculated by taking the median of the amount of time necessary for respondents to complete the questionnaire. If an interview is completed in under 35% of this median time, it will score 100% on this indicator.

An example: the median amount of time respondents take to complete the survey is 100 seconds (about 1 and a half minutes). An interview was completed in 29.9 seconds. This interview will score 100% on the speeder indicator.

Prerequisite: at least 3 matrix / semantic differential questions with at least 3 rows and 5 columns.

Each question that meets the criteria above gets checked individually. If all rows within an individual question have the same value, then this question is internally flagged as “straight lined”. When this has happened to all questions above the score will be 100% for the straight-line indicator. If 2 out of 3 questions have been answered in a line, the score will be 66%.

Prerequisite: open questions with input size setting ‘large’ and which contains a value. Empty fields will be excluded.

The interviews meeting these conditions will be segmented in two categories:

• – category 1: open answers with 3 or less words
• – category 2: open answers with more than 3 words

Open answers with 3 or less words
If 3 or less words are present it will not be reviewed by AI text analysis and will be scored based on the values present. The software considers spaces and ignores double spaces and special signs/symbols. Scoring per question is done as follows:

• 0 words present = 100% score
• 1 word present = 100% score
• 2 words present = 80% score
• 3 words present = 60%

Open answers with more than 3 words
If more than 3 words are present and the text field is large then AI will review. It will do so as follows:

• if a language cannot be determined, then it will score a 100% for Bad open end
• if a language can be determined for the answer, but it is not supported by the survey it will score 100%
• if more than two languages can be assigned to >70% of the answers present it will score 100% for bad open end
• if more than two languages can be assigned to >70% of the answers, and at least one cannot be supported by the survey it will score 100% for bad open end.

If the average of these scores is 80% or higher, the interview will get a 100% score for Bad open end.