DM p20-21 Opinion Dec08.qxp 02/12/2008 13:09 Page 20
Opinion
A
s the old saying goes, and being attributed to nique found its way into US rather than UK DM test-
King Solomon, it is a very old saying, ‘What ing disciplines. A large scale UK campaign would be
has been will be again, what has been done anything in excess of 250,000, whereas in the US it is
will be done again; there is nothing new under the more in the region of 1,000,000. However, many UK
sun’. So when something new does turn up, it is very banks, credit card providers, insurers, and utility
giants, to name but a few, regularly execute cam-
Is multi-variable
paigns of 500,000 or more, which can facilitate MVT.
So, what exactly is MVT and why is it so much bet-
ter than traditional AB split testing that we have all
testing the
grown up with? Because it allows the testing of many
different variables all at the same time.
future?
Traditional AB split testing dictates that one vari-
able only is altered while everything else stays con-
stant. Hence, any difference in response between two
MVT – is it a TV
packs can be attributed to the change in the test vari-
channel, a heart
able. However, MVT is not bounded by this limita-
tion. Consequently, insight is gained that much faster
condition, or the way and paradoxically more cheaply in comparison to
forward for DM testing?
conventional testing methods.
For example, suppose we have identified that there
Ian Robinson argues are ten components of a pack that we could vary to
that it is one of the most
improve response. We may have two versions of the
covering letter to test to see which generates a higher
important pieces of
response; we may be interested in testing visuals on
innovation in the field in
the envelop vs plain envelope, DL vs C5 format, 10%
off vs free widget and so on.
recent years.
In the end, if we have ten different factors that
have two alternatives each, the total number of per-
exciting. And I talk of the testing methodology, mutations is 1024. Our task is to find which permuta-
‘multi-variable testing’ or MVT. tion is the one that will generate the highest
In truth, the approach hit the UK about 18 months response; clearly, a tall order for conventional testing
ago as part of the website optimisation offerings arriv- methods.
ing from the US, though closer inspection shows that However, rather than having to test every permuta-
not all of them are the real McCoy. But let’s not be tion to find the best one, MVT substantially reduces
picky!! the number of permutations to test by using an
The methodology is a superior form of testing that obscure mathematical theorem that generates some-
delivers learnings as to the most effective execution thing called orthogonal arrays. These apparently
of a direct mail pack, email template, newsletter lay- identify which permutations need to be tested to
out, landing page design etc, etc. reveal the optimal permutation that will produce the
The slight drawback is that it is only applicable in best result. In the case of the example above, the
large scale direct mail/email campaigns or websites orthogonal array told us that only twelve permuta-
that generate high hit rates, which is why the tech- tions needed to feature in the test if we were able to
% res
1.2
1.0
.8
.6
.4
.2
0
Letter Outer Proposition Offer Headline
% res
1.2
1.0
.8
.6
.4
.2
0
Visual 1 Visual 2 Body Text CTA Format
20 Database Marketing December 2008
www.dmarket.co.uk
Page 1 |
Page 2 |
Page 3 |
Page 4 |
Page 5 |
Page 6 |
Page 7 |
Page 8 |
Page 9 |
Page 10 |
Page 11 |
Page 12 |
Page 13 |
Page 14 |
Page 15 |
Page 16 |
Page 17 |
Page 18 |
Page 19 |
Page 20 |
Page 21 |
Page 22 |
Page 23 |
Page 24 |
Page 25 |
Page 26 |
Page 27 |
Page 28 |
Page 29 |
Page 30 |
Page 31 |
Page 32 |
Page 33 |
Page 34 |
Page 35 |
Page 36 |
Page 37 |
Page 38 |
Page 39 |
Page 40 |
Page 41 |
Page 42 |
Page 43 |
Page 44