Many product or concept testings benchmark two or more products (or concepts). The answers of the respondents could be biased just because the evaluation of the second product (or concept) happens after another evaluation, and the respondent might be primed for it. For that reason products (or concepts) are shown in an alternating order.
Monadic or Rotation
There are always 2 procedural methods: The monadic test design lets the participant evaluate only one product. The rotation test design lets the participant evaluate all (or a few) products but in different order. Both test designs can be realised within a Survalyzer survey. The programming is most diverse. Thus, the two methods are explained separately.
Randomisation or Equal Distribution
| Randomisation | Equal Distribution (monadic) | Equal Distribution (rotation) | |
| Programming | – ordinary | – ordinary | – complex |
| Allocation | – at random – no information about rotation order | – even | – even – rotation order always known |
| Number of products | – many feasible (no increase in complexity) | – many feasible (no increase in complexity) | – only a few (exponential increase in complexity) |
The easiest way is always the randomisation. A random process controls what the participant will see at first. A randomisation can be programmend without much effort. However, the random process can produce uneven allocations. Not always 50% of the participants will see product A at first (and the other 50% product B), but for instance 70% of the participants will see the one product at first while the other 30% will see at first the other product. Therefore, it is more preferable to seek an equal distribution, which is difficult in a rotation test design and becomes more difficult with a bigger number of products (exponential increase in complexity).
How can I realize my test design within a Survalyzer survey?
The following 4 examples describe for each combination of monadic/rotation test design and random/equal distribution how to set-up the survey. All examples use 3 products A, B and C.
| Randomisation | Equal Distribution | |
| Monadic Test Design | Variation 1 | Variation 2 |
| Rotation Test Design | Variation 3 | Variation 4 |
Starting Position
There are 3 products to be tested. Every participant should evaluate only one product. Which product actually has to be evaluated is the result from a randomised selection.
Even though this variation is a very seldom case (the equal distribution from variation 2 should be preferred), it is described here to have all variations documented.
Allocation of Monads
In a value assignment the function randomInt is used to assign the product A, the product B or the product C to the participant for evaluation:
The example assigns a value between 1 and 3 (the maximum value 4 is excluded in this function) to the variable “monad” randomly. The values represent the products A, B or C.
Interrogation of Monads
The questions regarding the three products are arranged in three separate sections, one section for product A, one for product B and one for product C. The sections are filtered by the information provided in the custom variable “monad” (“show section A, if monad=1”, and so on):
Starting Position
There are 3 products to be tested. Every participant should evaluate only one product. To achieve equal numbers of evaluation for each product, the participants will be allocated to the products by the use of an equal distribution process.
The first participant will evaluate product A, the second product B, the third product C. The fourth participant again will evaluate product A, the fifth product B, the sixth product C and so on.
Allocation of Monads
Which product is to be evaluated by the participant will be defined in a value assignment. There the total number of all interviews (started and completed) is divided by 3 (=number of products), and the remainder of the division is calculated (modulus-function):
The example writes the remainder 0, 1 or 2 into the variable “monad”. The values 0, 1 or 2 represent the products A, B or C.
Interrogation of Monads
The questions regarding the three products are arranged in three separate sections, one section for product A, one for product B and one for product C. The sections are filtered by the information provided in the custom variable “monad” (“show section A, if monad=0”, and so on):
Starting Position
There are 3 products to be tested. Every participant should evaluate all products but in different order. In which order the products actually are shown is the result of a randomisation process.
Interrogation of Products
The questions regarding the three products are arranged in three separate sections, one section for product A, one for product B and one for product C:
Rotation Definition
The order in which the products are shown will be defined by random. The easiest way is to randomize the sections:
Please note: There is no variable that stores the information which product came first, which one was on second position and which was the last one.
Starting Position
There are 3 products to be tested. Every participant should evaluate all products but in different order. To avoid evaluation biases just because of the order the products are presented, all products should be shown in equal frequency on first, second and third position.
This design/variation is the most complicated, although the most ideal one. The randomisation process from variation 3 has to be substituted by a rotation plan.
Interrogation of Products
The questions regarding the three products are arranged in three separate sections, one section for product A, one for product B and one for product C:
Rotation Definition
To show 3 products in different order there are in total 6 combinations:
- A-B-C
- B-C-A
- C-A-B
- A-C-B
- B-A-C
- C-B-A
Which of these 6 combinations (rotation orders) is allocated to the participant will be defined in a value assignment. There the total number of all interviews (started and completed) is divided by 6 (=number of rotation combinations), and the remainder of the division is calculated (modulus-function):
The example writes the remainder 0, 1, 2, 3, 4 or 5 into the variable “combination”. Each value represents one of the possible rotation orders:
- 0 -> A-B-C
- 1 -> B-C-A
- 2 -> C-A-B
- 3 -> A-C-B
- 4 -> B-A-C
- 5 -> C-B-A
The sections are now put in the appropriate order with the custom variable “combination” and with the help of the value assignment function setSectionOrder (if combination=0, show section A then section B and then section C, etc.):
Annotations
The here shown example refers to 3 products that have 6 rotation combinations. For 2 products only 2 rotation combinations exist. For 4 products there are 24 rotation combinations, and for 5 products already 120 rotation combinations exist. The more products, the higher the complexity which increases exponentially. At the latest of 5 products it is preferable to use the variant 3 (randomisation) instead.