This guide applies to the new version of the MaxDiff question available in Survalyzer since June 5, 2026. If you added MaxDiff before this date, it will retain its current settings. Once it is deleted, only the new version of MaxDiff can be added.
If you are using the old version of MaxDiff and need instructions, please follow this link: Here.
Introduction: When to use MaxDiff
MaxDiff helps you quickly see which items matter most and least to respondents.
Instead of asking people to rank a long list, they see smaller sets of items and simply pick the most and least important in each one. Based on these choices, Survalyzer builds a clear overall ranking.
You can use MaxDiff (also known as best‑worst scaling) when you need to research a larger number of items, like:
- product features
- purchase criteria
- service attributes
- employee benefits
The usual list length sweet spot is between 12-20 items. Longer lists are also possible, however respondent fatigue and motivation needs to be considered.
It’s especially useful when rating scales don’t work well: for example, when people tend to rate everything as important. By forcing trade-offs, MaxDiff gives you more meaningful and easier-to-read results.
Good to know:
- Survalyzer supports 8–40 items in a MaxDiff question.
- For fewer than 8 items, use Rank order question type instead.
Set up a MaxDiff question
Add the MaxDiff question type to your survey
Go to the Build section while editing your survey.
Click + Add content in the top-right corner. In the Methods subsection, select MaxDiff. You can click the tile or drag and drop it into the right place in your survey. Configure the question using the settings in the right sidebar.
Add Question text and Hint text
Enter the question text.
Use clear wording that tells respondents what they should compare. For example:
When choosing a project management tool, which of these factors is most important and which is least important to you?
You can also add optional hint text. Use it when respondents need more context, for example:
Select one item as most important and one item as least important in each set.
Adjust the Most important and Least important labels
The labels explain which side respondents should use for each selection.
By default, the labels are Most important and Least important. You can keep them or change them to match your research topic.
For example, you could use:
- Most preferred and Least preferred
- Highest priority and Lowest priority
- Most relevant and Least relevant
Use labels that are easy to understand and consistent with the question text.
Add Answer choices
Add the items you want respondents to compare.
Each item should describe one clear concept. Avoid combining two ideas in one item, because respondents may not know which part they are judging.
You can add items manually. You can also copy and paste a list from an external file into the item list.
Review the Summary
The Summary shows the main design settings for your MaxDiff question, including how many items you added, how many tasks each respondent will answer, how many items appear in each task, how many design versions are created, and the recommended number of completed responses.
In most cases, you can keep the default values selected by Survalyzer.
GototheAdvanced settings (optional)section to learn more about each item in the Summary.
Understand the recommended number of completes
The Number of completes recommended shows how many completed responses you should aim for before interpreting MaxDiff results.
Survalyzer recommendation assumes 1000 target observations per item and minimum 300 completes across whole research, which is relatively conservative approach.
Example:
If you have 12 items, show 9 tasks per respondent, and display 4 items per task, Survalyzer recommends 340 completes.
Calculation details
Recommended completes = (1000 × number of items) / (questions per respondent × items per question)Use Filters
Use filters when the MaxDiff question should only appear for specific respondents.
You can apply filters to a MaxDiff question the same way you would for other survey elements.
Learn more: Filter UI explained
Advanced settings (optional)
Most users can keep the default settings. Survalyzer automatically selects balanced settings based on the number of items.
Use Advanced Settings only when you have a specific research design requirement or when you want to adjust the balance between respondent effort and analytical precision.
| Setting | What it means | When to adjust it |
|---|---|---|
| 1. Number of items per question Range: 3–9 | How many items respondents see in each MaxDiff task. | Use fewer items to make each task easier. Use more items to collect more comparisons in fewer tasks. |
| 2. Number of questions per respondent Range: 3–20 | How many MaxDiff tasks each respondent answers. | Use fewer tasks to reduce respondent effort. Use more tasks to collect more comparisons. A range of 6–12 tasks is usually manageable. |
| 3. Number of block design versions Range: 1–10 | How many different MaxDiff tasks design versions are created. Different respondents will see different versions, randomized. With 1 version, all respondents see the same tasks. | Changing number of versions might improve design balance for your custom scenario. Adjust according to warning information if present. |
| 4. Individual items seen by respondent | How often each item appears for one respondent on average. | If the design is not perfectly balanced, aim for at least 3 times per item. A range of 3–4 is usually a good target. |
| 5. Download experimental design | Downloads the tasks design as a spreadsheet. | Use it to check which items appear together before launching the survey. |
The downloaded experimental design file shows:
- Group – design version
- Question / Task number
- Choices – items that appear together in each task (items value defined in the survey)
Read MaxDiff results
Where MaxDiff analysis is available
MaxDiff analysis is available in all Survalyzer report types:
- Basic Report
- Expert Report
- Excel Segmented Report
- Dashboard
Available analysis
In Survalyzer reports, MaxDiff results are shown in two ways:
- Chart and table: Preference Index and Preference Percentage
- Table: Best and Worst Counts
Each view helps you understand the results from a slightly different angle.
Keep in mind when analyzing results
- Interpret MaxDiff results based on the full sample or a large enough segment, not on individual respondent-level values.
- The results are statistically reliable when the recommended sample size is reached.
Preference Index chart and table
The Preference Index chart shows the MaxDiff ranking on an index scale centered around 100. Use this chart to quickly see which items performed above or below the average item in the MaxDiff set. Items are ordered from highest to lowest Preference Index, so the strongest item appears first.
The black line at 100 is the reference line. It represents the average level of preference. Values above 100 mean that an item was preferred more than average. Values below 100 mean that an item was preferred less than average.
The table helps you compare items in more detail. Use it to see which items perform above or below average and how much relative preference each item received.
Calculated metrics explanation
| Metric | What it shows | How to read it |
|---|---|---|
| Preference Index | A score that shows how strongly an item performs compared to the average item in the MaxDiff set. It shows whether an item performs above or below the average item in the MaxDiff set. Preference Index = Average rank value × number of items × 100 | 100 = average Above 100 = above average Below 100 = below average |
| Preference Percentage | The average Rank Value shown as a percentage. It shows how much relative preference each item received. | Use it to compare each item’s share of preference against the other tested items. |
Table: Best and Worst counts
This table shows the raw selection counts behind the ranking. Use it to check how often each item was directly selected as most important or least important.
Calculated metrics explanation
| Metric | What it shows | How to read it |
|---|---|---|
| Best Count | How often the item was selected as most important. | Higher value means the item was often chosen as the strongest option. |
| Worst Count | How often the item was selected as least important. | Higher value means the item was often chosen as the weakest option. |
| Difference Best-Worst | Best Count minus Worst Count. | Positive value = more best selections. Negative value = more worst selections. |
MaxDiff Raw Data
In raw data exports, a MaxDiff question creates several variables.
The exact variable names depend on the question code and answer choice codes.
For example, if the MaxDiff question code is q1, the export may include the following variables.
| Variable | Meaning |
|---|---|
| q1_group | Design version assigned to the respondent |
| q1_t1_best | Item selected as most important in task 1 |
| q1_t1_worst | Item selected as least important in task 1 |
| q1_t2_best | Item selected as most important in task 2 |
| q1_t2_worst | Item selected as least important in task 2 |
| q1_1_rankValue | Rank value for answer choice 1* |
| q1_2_rankValue | Rank value for answer choice 2* |
*The rankValue variables use the PageRank-based calculation method. For more details, see Understand the Rank Value (PageRank) metric.
Use the raw data export when you want to run advanced analysis outside Survalyzer or check the respondent-level MaxDiff selections in detail.
Understand the Rank Value (PageRank) metric
Survalyzer calculates a Rank Value score for each item in each interview.
The Rank Value score is calculated using a PageRank algorithm.
The logic is similar to the method Google uses to decide which websites should appear higher in search results. In MaxDiff, this approach is used to calculate how strongly each item performs compared to the other items in the same tasks.
The calculation does not only look at how often an item was selected as best or worst. It also considers the relationships between all items shown together in each task.
This means that an item can have a different final Rank Value even if it was rarely selected directly as best or worst. Its score also depends on which items it was compared with and how those items performed.
This PageRank-based method has been researched as an efficient alternative to Hierarchical Bayes for MaxDiff analysis. You can find more details in the white paper linked below.
For more details about the PageRank method, please refer to the white paper that explains the algorithm in detail:
Münnich, Ákos, Emese Vargáné Karsai, and Jenő Nagy. 2022. “A Real-Time Network-Based Approach for Analysing Best–Worst Data Types.” SN Business & Economics 2 (7).
Download the paper Here
Understand info, warning and error messages
Survalyzer may show messages within MaxDiff Advanced Settings section when the selected settings affect the design quality.
There are 3 types of information or warning messages present:
Type 1: Safe to continue: Design is not perfect but good enough to proceed.
Message 1: Design is not perfectly balanced, but still usable
“The design is not perfectly balanced, but items appear 3 or more times on average per respondent. Reliable analytical results can be achieved by using recommended sample size.”
Type 2: Review the setup: There are issues with the design -> Adjust the advanced settings to fix it, unless this design and it’s consequences are fully intentional.
Message 2: Items are shown too rarely
“The design is not perfectly balanced, and items appear fewer than 3 times on average per respondent. A larger sample size is required to ensure reasonable analytical precision (refer to the recommended sample size). This issue can be mitigated by increasing the number of items per question or the number of questions per respondent.”
Message 3: Items do not appear in all positions
“Sparse design: Not all items appear in all column positions. Try to increase the number of versions to mitigate this warning. ”
Type 3: Modify before launch: There are issues with the design -> Adjust the advanced settings to fix it, otherwise research results will not be reliable.
Message 4: One or more items are missing from a design version
“At least one of the items is not shown at all in one of the versions. Please adjust the settings to mitigate this issue. Otherwise, rank value calculations might be broken or unreliable.”
Message 5: The design cannot be generated
“PROCEDURE STOPPED: Insufficient design space”
This error appears when the selected settings do not provide enough design space to create a valid MaxDiff design.
Change the setup by changing the number of tasks, items per task, or design versions.