MaxDiff

NoteThis question type is only available for Professional Analytics users

In MaxDiff (maximum difference) analysis, survey respondents are presented with a list of items and asked to select the options that are most important and least important to them. Each item is scored based on the order of preference expressed by the respondent. MaxDiff is sometimes called best-worst scaling or maximum difference scaling and allows you to gain insight into what features or attributes are most valued by the target audience.

The purpose of a MaxDiff question is to rank a rather large number of items. The question actually consists of several automatically created subquestions, where systematically selected items can be assessed in comparison to each other.

If you want to rank fewer than 8 items we recommend using the Rank Order question type. For larger datasets (up to 50 items) a MaxDiff question is recommended.

How does MaxDiff work?

A good example of how you would use a MaxDiff question is when you want to know which attributes your audience considers when buying a new product or service. In the following example, buying a new mobile phone will be the use case.

First of all you have to determine which items or attributes of a phone should be considered in the ranking.

Based on these attributes several subquestions will be created, where the respondent will be presented with the different attributes and ranks them based on importance. In this example there are 13 items, resulting in 13 subquestions.

All respondents will see the same subquestions, but the sequence is randomized for each respondent. The respondent selects which attritute is most important and which attribute is least important and then proceeds to the next subquestion.

After all subquestions have been answered the score is automatically calculated for all attributes.

Details on generating the subquestions

The respective design of the MaxDiff question must meet a handful of requirements. It must be independent of the number of actual participants, as this is often not even known in advance of the survey. During the evaluation, it must be possible to form any number of sub-samples for comparison with each other, regardless of the design used. And in general, it should be possible to create the design with little time expenditure and few mandatory settings. For this reason, a MaxDiff question in Survalyzer shows each participant the exact same set of subquestions (the same version). A distortion of the data because dropouts or subsequently invited participants used certain versions more frequently than average is therefore ruled out.

The same set of subquestions for all respondents can only be used if all items are displayed equally frequently across all subquestions. In addition, it must be ruled out that any pair of items appears twice in the subquestions (orthogonality condition). This results in the restriction that the generated MaxDiff design has exactly as many subquestions as there are items. Depending on the number of items per subquestion, each item appears 3, 4 or 5 times. This also allows each item to appear exactly once in the first, second, third and possibly fourth or fifth position in the subquestions. Corresponding designs that fulfill the above conditions require a minimum number of items for 3, 4 or 5 items per subquestion. This means that the MaxDiff question in Survalyzer is only possible from 8 items in total (3 items per subquestion). At least 14 items are required for 4 items per subquestion and at least 23 items are required for 5 items per subquestion.

The number of theoretically existing item pairs is greater than the number that can be displayed in all subquestions together. Since a>b and b>c inevitably result in a>c, it is not necessary to display all item pairs in the subquestions. This fact makes it possible for the user to specify for a particular item pair that it is not displayed together in any subquestion. This setting is optional. If it is not set, the design is generated with randomly combined item pairs.

Analyzing results

When your respondents have assessed the attributes you will be able to see which attributes are considerd to be the most important and the least important in the decision making of your target audience.

Details on the evaluation of the MaxDiff question

The item that the respondent rates as “most important” in a subquestion is given the value 1. The item that the respondent rates as “least important” in a subquestion is given the value -1. The remaining items are given the value 0. After answering the MaxDiff question, the values are added up for each item and divided by the number of occurrences of the item (3, 4 or 5). The calculated item score in the interview is a mean value between -1 (not important) and 1 (important). If you calculate the mean value for each item across all respondents or across the respondents in a sub-group, you obtain the total score or the group score of the items.

In the raw data export, a column/variable is displayed for each MaxDiff item. The figures shown are the item scores of the individual interviews, which can be used to calculate total scores (simple averages), for example. The total scores are shown in the Basic Report. The corresponding chart sorts the items according to the total score with the most important item at the top. In the Segmented Excel Report, the group scores of the individual segments are displayed in addition to the total score.

The MaxDiff design used (see “Details on generating the subquestions”) enables simple evaluation using mean values. No further statistical knowledge is required and no special programs for e.g. Hierarchical Bayesian Estimation are necessary when analyzing the raw data. In this respect, the design actually used is not reflected in the raw data. This only becomes visible when a test interview is completed.

Additional settings

Items per subquestion

You can determine the number of items per subquestion:

  • 3 items per subquestion → minimum of 8 items required
  • 4 items per subquestion → minimum of 14 items required
  • 5 items per subquestion → minimum of 23 items required

Exclude pair of items

An optional feature for MaxDiff is that you can determine that a certain pair of items is never combined within any of the subquestions:

This particularly useful when you have two similar items where the respondent is very likely to always prefer one option over the other. If a respondent can choose between 128 GB of Memory or 256 GB of Memory it is very likely that 256 GB of Memory will be preferred.

This means that asking for the preference between the two will most likely not contribute to your survey. However comparing how important the minimum amount of memory is compared to other attributes is very valuable.

Updated on November 28, 2023

Was this article helpful?

Need Support?
Please login to your Survalyzer account and use the "Create Support Request" form.
Login to Survalyzer