Multivariate testing is a process by which more than one component of a website may be tested in a live environment. It can be thought of in simple terms as numerous A/B tests performed on one page at the same time. A/B tests are usually performed to determine the better of two content variations; multivariate testing can theoretically test the effectiveness of limitless combinations. The only limits on the number of combinations and the number of variables in a multivariate test are the amount of time it will take to get a statistically valid sample of visitors and computational power.

Multivariate testing is different from A/B testing in a few small but important ways. First of all, A/B testing uses one original and one variable version of a feature, whereas multivariate testing takes into account a number of variables tested simultaneously. Another big difference, albeit debatable, is that A/B testing is reserved for major changes, such as an entirely different landing page or design, while multivariate testing makes many smaller adjustments to an already existing page.

For example, your checkout process may be causing a lot of trouble for visitors, and there are a number of different issues with it that aren’t determinable with just one test. In this case you might decide to implement a multivariate test that takes into account the mandatory form fields of the checkout page, the prominence of the shipping and handling policy, the colour of the backdrop, and a new progress bar above each page of the checkout. The results come in with the combination that caused the most successful checkouts, and this is the one you choose to implement.

A/B Multivariate Testing
Image from www.webseoanalytics.com




I'm Katrina McKinnon, creator of eCommerce University, founder of McKinnon Group and Small Revolution. I'm using my 20 years' experience in building and operating online businesses to create engaging educational materials that helps others become successful online workers. Find me on LinkedIn and Twitter.