At veepee.com, we develop recommender systems to rank the 200+ sales banners of the website homepage. The Data Science team is continuously improving the recommendation algorithms on various perimeters by either adding / removing / refactoring features or modifying the model’s architecture and optimization methods. Thus, implementing A/B testing is an essential part of the online evaluation process to assert every move is made in the right direction. This introductory document aims to summarize the basic concepts of the Bayesian A/B testing approach and gives an example of application process.

In certain cases, the Bayesian approach may provide useful results faster than the frequentist method. It may also be relevant to reach conclusions with small volumes. In addition, if the theory behind the method is more complex then for the frequentist approach, the main results are easier to understand for the business:

• Probability to chose the best variation.
• Expected loss linked to the choice of one or another variant.

#data-science #python #testing #machine-learning #developer