Business owners who engage in online sales want the option of testing out different versions of a web page to see which one has a higher conversion rate. Most auto dealers have heard of, or currently utilize, “split” A/B testing. In split A/B testing, traffic is evenly divided between two variations of the same web page until a traffic cap is reached, and a statistically significant victor is chosen.

A slightly different approach, known as “multi-armed bandit” testing, adjusts traffic as the test progresses. This method works extremely well with responsive web design. It ensures that dealers and their web providers are creating pages with user interfaces that make sense — on every device. Google has recently been integrating this type of testing within its Analytics API, along with new analytics features and report data.

Two Testing Types
To illustrate the difference between split A/B testing and the multi-armed bandit approach, let’s imagine that a dealership has opened up showroom space for new vehicles. The dealer wants to try out two different models to see which one will generate the most interest. She performs an experiment on the first 200 customers who walk in. A sales rep greets customers and directs half of them into an area of the room dominated by Vehicle A and the other half into an area featuring Vehicle B. The dealer decides which vehicle to display prominently by counting how many customers from each group took a serious look at the vehicle they were directed toward.

If the dealer used a multi-armed bandit approach, however, she would tell the sales rep to analyze his data twice a day, checking customer behavior and gradually adjusting incoming traffic to favor the more popular vehicle. By the end of the experiment, most of the traffic would be flowing to that vehicle, thus saving on conversions that might otherwise be lost. Hypothetically, due to the dynamic traffic adjustments, the sales rep could have directed 60 people into the area with the lower conversion rate and 140 into the area with the higher conversion rate.

This example has become slightly muddled by Internet marketing terminology, but it should still serve to clarify the difference between the two approaches. The logic appeals to many marketers who wish to maintain higher conversion rates during testing. Google has several articles on the subject of multi-armed bandit testing, all of which can be found at www.support.google.com.

The company claims that the new approach is just as statistically valid — and more efficient, faster, cheaper and more appropriate for website testing — than the split A/B approach, since it optimizes throughout the testing process. “If one of your variations performs much better than the others, the optimal arm will be found very quickly,” Google’s FAQ states. “If one or more variations perform much worse than the others, they will be down-weighted very quickly, so the experiment can focus on finding the best arm.”

Content Experiments
It should come as no surprise that Google now implements multi-armed bandit testing in Google Analytics, which has become a fully functional A/B experimentation platform. Multiple variations can be tested simultaneously. Google adjusts traffic flow twice a day based on the user’s testing criteria, until the traffic cap is reached, with a three-month maximum timeframe.

Google Analytics’ Content Experiments API rolled out earlier this year, and is designed to facilitate analytics of the website content and content experimentation. In-page tracking, in-site search tracking, site speed analysis and AdSense data tracking are a few of the features offered by the service.

With multi-armed bandit testing integrated into the Analytics API, developers and marketers now have a new toolbox to test multiple web page variations and view the results as they occur in their analytics reports. This new, highly programmable analytics interface provides new A/B experimentation solutions that are flexible, responsive and customizable. As time goes on, the API will undoubtedly continue to evolve and offer new functions for dealers and other business owners who depend on online sales.

Adam Dennis is vice president of product development for Dominion Dealer Services and is an expert in project management and user testing.
ADennis@AutoDealerMonthly.com