Recently, a former Facebook Engineer Tan Chao on 2010 to 2015 Facebook revision of the site of the home page of the article is a great concern. According to his record, in 2012, when the vice president of Facebook products to lead a company's internal top expert team, which lasted six months to develop a new website home page, a major revision and update. Subsequently, Facebook on the home of the A/B test: when the test to 5 user base, the data indicators are in decline, when the test to 12 user base, the decline is still evident. So, Facebook had to give up the results of the more than 30 people in the team for nearly a year of effort.
A/B is a ghost?
A/B testing is a mature test, development method, Google development team is generally used. Industry insiders told reporters that the titanium media, Google advertising each new change will also go through a rigorous testing to verify the results, in ensuring that the search experience while improving the conversion rate of advertising, which is based on the data of Internet product development and operations, can greatly accelerate innovation, improve user experience, play a multiplier effect. Former Google advertising quality control department engineer Wang Ye introduction, A/B test optimization can bring $10000000000 in revenue growth of Google.
In simple terms, the A/B test is a A version of the page and the B version of the test, which version of the higher click rate. The most common form of A/B testing is called gray scale test, which is a company's own Internet users to 2, 5, 10, 1 and other scale of user testing, and then select the best test results page to the full user group.
Virtually all successful Internet Co, to varying degrees, the use of A/B testing for the optimization of the web and the effect of the click rate. Wang Ye said, Google's products have hundreds of different test versions of the same time running, the test data determines only a few to change the kinetic energy of the final line, these better changes can achieve a monthly income of about 2 to upgrade, and ultimately reached 20 of annual growth.
And through Facebook, it shows that the in the world of the Internet have an ever victorious general, experience and intuition are not Internet world rules of the game. Only through continuous testing and repeated data analysis, to prove a point of view or to judge whether the right. From the example of Facebook can be seen, but not fast breaking and behind the trial and error, is actually
Technical support and data thinking cross-border
A/B testing sounds simple, but in fact is a combination of technical support and data thinking cross-border areas.
Reporters in Silicon Valley interview with the United States A/B Optimizely Product Manager Jones Byron, he introduced the A/B test generic five step method: analysis, hypothesis, build test, run test, effect evaluation. The analysis, assumptions, and build test belongs to the stage of data analysis, the operational test and evaluation belongs to the category of technical support, the two complement each other, complement each other.
In the data analysis, the A/B test is mainly about the mathematical hypothesis of the sample user group: according to the statistical principle and algorithm, the most statistically significant sampling user group characteristics, scale, the number of experimental groups and the number of times. In the technical support, the A/B test is based on the large scale Internet traffic segmentation technology, different web pages for different user groups, and then through the server and network control to complete the traffic segmentation, to different users to send different web pages.
In data analysis, there is a very interesting Simpson paradox. The image of that is when 50 of female users to implement a promotion, sales have increased; when the same promotion to 50 of male users, sales have also increased; but when the two groups of people, and then promote the same way, sales fell. Therefore, the need to pass the relevant mathematical processing algorithms, as far as possible to eliminate the Simpson paradox.
A/B test cloud services, has become a vertical direction of entrepreneurship
Internet Co's A/B test are mostly from the internal team, with Google as the representative of the scale and professional A/B testing gradually mature. With the rapid development of the Internet, more and more companies need to carry out Internet services, all the use of self built A/B test team is clearly a high cost, so there is a SaaS based model of the third party specialized A/B testing company.
Google engineer background of the two teams, respectively, are doing a similar service Venture Company: one, in the United States to create a Optimizely company, has completed the C round of financing, the company was founded in five years to complete a total of $145000000 in financing. Optimizely's investors, including the famous Silicon Valley venture A16Z, Capital Ventures Bain, Ventures Salesforce, etc., the company has more than 400 employees, business development is very rapid.
While another team returned to the China, created at science and technology in the last year, this is the Wang Ye team. Wang Ye has been involved in the Chinese next generation Internet () and the network APP traffic optimization (ALTO) and other large projects, such as the Chinese next generation Internet (CNGI) and network traffic optimization (NEC) and other major projects in the Yale, Microsoft Research Institute and the North American laboratories involved in a number of technical innovation. 2012 to 2014, Wang Ye in the United States Google headquarters advertising quality control department. In 2014, he saw the domestic Internet boom after deciding to return to entrepreneurship, the mature A/B test technology to provide domestic enterprises and developers in the form of public cloud.
Yell technical product called AppAdhoc optimizer is, which is optimizing platform based on an A / B test, in April 2015 comprehensive on-line, currently has more than 100 customers in trial. Yell technical support dynamic test traffic control, multi variable combination test, a large number of parallel test, according to specific populations of directional test, the SDK not only support HTML5 web end a / B test, also supports native mobile app (IOS, Android) end. At present science and technology investment institutions including forright capital, geeks and other states.
2016 will become the key to the Internet operation of the year, regardless of the traditional enterprise Internet service, or a large number of O2O and electricity provider Venture Company, can be tested through A/B such professional tools to study online user behavior, improve operational efficiency, to avoid the pitfalls of empirical and intuitive thinking.
A/B test tells us that the Internet Era
Text / titanium media. ITValue reporter Wu Ningchuan