Many product or concept testings benchmark two or more products (or concepts). The answers of the respondents could be biased just because the evaluation of the second product (or concept) happens after another evaluation, and the respondent might be primed for it. For that reason products (or concepts) are shown in an alternating order.
Monadic or Rotation
There are always 2 procedural methods: The monadic test design lets the participant evaluate only one product. The rotation test design lets the participant evaluate all (or a few) products but in different order. Both test designs can be realised within a Survalyzer survey. The programming is most diverse. Thus, the two methods are explained separately.
Randomisation or Equal Distribution
| Randomisation | Equal Distribution (monadic) | Equal Distribution (rotation) | |
| Programming | – ordinary | – ordinary | – complex |
| Allocation | – at random – no information about rotation order | – even | – even – rotation order always known |
| Data/Analysis | – answers always in the same variables | – answers always in the same variables | – answers scattered in different variables |
| Number of products | – many feasible (no increase in complexity) | – many feasible (no increase in complexity) | – only a few (exponential increase in complexity) |
The easiest way is always the randomisation. A random process controls what the participant will see at first. A randomisation can be programmend without much effort. However, the random process can produce uneven allocations. Not always 50% of the participants will see product A at first (and the other 50% product B), but for instance 70% of the participants will see the one product at first while the other 30% will see at first the other product. Therefore, it is more preferable to seek an equal distribution, which is difficult in a rotation test design and becomes more difficult with a bigger number of products (exponential increase in complexity).
How can I realize my test design within a Survalyzer survey?
The following 4 examples describe for each combination of monadic/rotation test design and random/equal distribution how to set-up the survey. All examples use 3 products A, B and C.
| Randomisation | Equal Distribution | |
| Monadic Test Design | Variation 1 | Variation 2 |
| Rotation Test Design | Variation 3 | Variation 4 |
Starting Position
There are 3 products to be tested. Every participant should evaluate only one product. Which product actually has to be evaluated is the result from a randomised selection.
Even though this variation is a very seldom case (the equal distribution from variation 2 should be preferred), it is described here to have all variations documented.
Allocation of Monads
In a value assignment the function randomInt is used to assign the product A, the product B or the product C to the participant for evaluation:
The example assigns a value between 1 and 3 (the maximum value 4 is excluded in this function) to the variable “monad” randomly. The values represent the products A, B or C.
Interrogation of Monads
The questions regarding the three products are arranged in three separate sections, one section for product A, one for product B and one for product C. The sections are filtered by the information provided in the custom variable “monad” (“show section A, if monad=1”, and so on):
Starting Position
There are 3 products to be tested. Every participant should evaluate only one product. To achieve equal numbers of evaluation for each product, the participants will be allocated to the products by the use of an equal distribution process.
The first participant will evaluate product A, the second product B, the third product C. The fourth participant again will evaluate product A, the fifth product B, the sixth product C and so on.
Allocation of Monads
Which product is to be evaluated by the participant will be defined in a value assignment. There the total number of all interviews (started and completed) is divided by 3 (=number of products), and the remainder of the division is calculated (modulus-function):
The example writes the remainder 0, 1 or 2 into the variable “monad”. The values 0, 1 or 2 represent the products A, B or C.
Interrogation of Monads
The questions regarding the three products are arranged in three separate sections, one section for product A, one for product B and one for product C. The sections are filtered by the information provided in the custom variable “monad” (“show section A, if monad=0”, and so on):
Starting Position
There are 3 products to be tested. Every participant should evaluate all products but in different order. In which order the products actually are shown is the result of a randomisation process.
Interrogation of Products
The questions regarding the three products are arranged in three separate sections, one section for product A, one for product B and one for product C:
Rotation Definition
The order in which the products are shown will be defined by random. The easiest way is to randomize the sections:
Please note: There is no variable that stores the information which product came first, which one was on second position and which was the last one.
Starting Position
There are 3 products to be tested. Every participant should evaluate all products but in different order. To avoid evaluation biases just because of the order the products are presented, all products should be shown in equal frequency on first, second and third position.
This design/variation is the most complicated, although the most ideal one. On the one hand, the same questions have to be created (copied) several times. On the other hand, a rotation plan has to be generated.
Interrogation of Products
The questions regarding the three products are initially arranged in three separate sections, one section for product A, one for product B and one for product C. After everything is finished, tested and corrected the sections have to be copied. In the end there must be 3 sections for each product (because 3 is the number of products). To have a clear arrangement name the sections from A1 to C3. Since the sections will be filtered, the participants will actually see only one section of each product. Die sections should be arranged in this order: A1, B1, C1, A2, B2, C2, A3, B3, C3.
Rotation Definition
To show 3 products in different order there are in total 6 combinations:
- A1 B2 C3
- B1 C2 A3
- C1 A2 B3
- A1 C2 B3
- B1 A2 C3
- C1 B2 A3
Which of these 6 combinations (rotation orders) is allocated to the participant will be defined in a value assignment. There the total number of all interviews (started and completed) is divided by 6 (=number of rotation combinations), and the remainder of the division is calculated (modulus-function):
The example writes the remainder 0, 1, 2, 3, 4 or 5 into the variable “combination”. Each value represents one of the possible rotation orders:
- 0 -> A1 B2 C3
- 1 -> B1 C2 A3
- 2 -> C1 A2 B3
- 3 -> A1 C2 B3
- 4 -> B1 A2 C3
- 5 -> C1 B2 A3
The sections now will be filtered with the information from the variable “combination” (“show section A1, if combination=0”, “show section B2, if combination=0”, “show section C3, if combination=0”, and so on):
Please note: The answers regarding each product are stored in different variables (in different sections that contain the same questions). The variables have to be consolidated prior to analyzing the data.
Annotations
The here shown example refers to 3 products that have 6 rotation combinations. For 2 products only 2 rotation combinations exist. For 4 products there are 24 rotation combinations and for 5 products already 120 rotation combinations exist. The more products, the higher the complexity which increases exponentially. At the latest of 5 products it is preferable to use the variant 3 (randomisation) instead.