1. Home
2. Other Resources
3. Tips & Tricks
4. Monadic and Rotation Test Designs

# Monadic and Rotation Test Designs

Many concept testing surveys include two or more versions to test. The answers of the respondents could be biased just because the evaluation of the second product (or concept) happens after another evaluation, and the respondent might be primed for it. For that reason products (or concepts) should be shown in an alternating order.

There are 2 possible methods to achieve that:

The monadic test design splits participants in groups and lets the participant evaluate only one product (see also A/B Testing).

The rotation test design lets the participant evaluate all (or a few) products but in different order.

Both test designs can be conducted with a Survalyzer survey. The survey programming is different in both cases, explained below.

#### Randomisation or Equal Distribution

The easiest way is the randomisation. A random process controls what the participant will see at first. A randomisation can be programmend without much effort. However, the random process can produce uneven allocations. Not always 50% of the participants will see product A at first (and the other 50% product B), but for instance 70% of the participants will see the one product at first while the other 30% will see at first the other product. Therefore, it is more preferable to seek an equal distribution, which is difficult in a rotation test design and becomes more difficult with a bigger number of products (exponential increase in complexity).

#### How can I realize my test design within a Survalyzer survey?

The following 4 examples describe for each combination of monadic/rotation test design and random/equal distribution how to set-up the survey. All examples use 3 products A, B and C.

Variation 1: Monadic Test with Randomisation

#### Starting Position

There are 3 products to be tested. Every participant should evaluate only one product. Which product actually has to be evaluated is the result from a randomised selection.

In a value assignment the function randomInt is used to assign the product A, the product B or the product C to the participant for evaluation:

The example assigns a value between 1 and 3 (the maximum value 4 is excluded in this function) to the variable “monad” randomly. The values represent the products A, B or C.

The survey is then divided into 2 parts: First, concept presentation, and second, assessment questions.

Concept presentation is done through the use of text blocks, which are filtered by the information provided in the custom variable “monad”. Each Text Block is filtered by the consecutive value of the “monad” variable. E.g. First text element receives filter “If monad = 1”, second “If monad=2”, etc. See example below:

Then, there is a set of questions, which are the same for each tested version. Therefore they don’t need survey filter. It’s up to the user to decide, if questions should appear on the same page as the concept presentation, or on different ones (with or without possibility to go back for the respondent).

Variation 2: Monadic Test with Equal Distribution

#### Starting Position

There are 3 products to be tested. Every participant should evaluate only one product. To achieve equal numbers of evaluation for each product, the participants will be allocated to the products by the use of an equal distribution process.

The first participant will evaluate product A, the second product B, the third product C. The fourth participant again will evaluate product A, the fifth product B, the sixth product C and so on. This variant corresponds to a so-called A/B test.

Which product is to be evaluated by the participant will be defined in a value assignment. There the total number of all interviews (started and completed) is divided by 3 (=number of products), and the remainder of the division is calculated (modulus-function):

The example writes the remainder +1, (so 1, 2 or 3) into the variable “monad”. The values 1, 2 or 3 represent the products A, B or C.

This part is exactly the same as in the randomisation method, see above.

Variation 3: Rotation Test with Randomisation

#### Starting Position

There are 3 products to be tested. Every participant should evaluate all products but in different order. In which order the products actually are shown is the result of a randomisation process.

#### Products presentation and assessment

The questions regarding the three products are arranged in three separate sections, one section for product A, one for product B and one for product C:

#### Rotation Definition

The order in which the products are shown will be defined by random. The easiest way is to randomize the sections:

Please note: There is no variable that stores the information which product came first, which one was on second position and which was the last one.

Variation 4: Rotation Test with Equal Distribution

#### Starting Position

There are 3 products to be tested. Every participant should evaluate all products but in different order. To avoid evaluation biases just because of the order the products are presented, all products should be shown in equal frequency on first, second and third position.

This design/variation is the most complicated, although the most ideal one. The randomisation process from variation 3 has to be substituted by a rotation plan.

#### Products presentation and assessment

The questions regarding the three products are arranged in three separate sections, one section for product A, one for product B and one for product C:

#### Rotation Definition

To show 3 products in different order there are in total 6 combinations:

1. A-B-C
2. B-C-A
3. C-A-B
4. A-C-B
5. B-A-C
6. C-B-A

Which of these 6 combinations (rotation orders) is allocated to the participant will be defined in a value assignment. There the total number of all interviews (started and completed) is divided by 6 (=number of rotation combinations), and the remainder of the division is calculated (modulus-function):

The example writes the remainder 0, 1, 2, 3, 4 or 5 into the variable “combination”. Each value represents one of the possible rotation orders:

• 0 -> A-B-C
• 1 -> B-C-A
• 2 -> C-A-B
• 3 -> A-C-B
• 4 -> B-A-C
• 5 -> C-B-A

The sections are now put in the appropriate order with the custom variable “combination” and with the help of the value assignment function setSectionOrder (if combination=0, show section A then section B and then section C, etc.):

#### Annotations

The here shown example refers to 3 products that have 6 rotation combinations. For 2 products only 2 rotation combinations exist. For 4 products there are 24 rotation combinations, and for 5 products already 120 rotation combinations exist. The more products, the higher the complexity which increases exponentially. At the latest of 5 products it is preferable to use the variant 3 (randomisation) instead.

Updated on January 29, 2024