The A’s and B’s of A/B Testing — A Beginner’s guide to experimentation

I recently completed Udacity’s course on A/B testing. The course started with a high-level understanding of what a typical A/B test entails before diving into specifics of each stage in the process of experimentation. Needless to say, it was a great learning experience! In this post, I am going to summarize my key learning from the course and explain how it benefits a lot of companies which are focused on improving user experience.

So, let us dive in!

So, tell me what is an A/B Test?

A/B testing is a method of experimentation to understand how user experiences change following any changes/variations made in the way they interact with a website, a mobile app etc. It is a very popular method to understand the level of existing customer experience and improve it based on certain relevant metrics.

Possible changes that companies may bring about are changing the layout of the website, changing the appearance/position of certain buttons on a particular webpage, change the ranking of options on the website etc. A/B testing helps the companies to evaluate whether these changes would be successful or not post-launch. This done by testing these changes on a subset of viewers for a certain period of time.

Okay, gotcha! But how will I evaluate these tests?

Evaluation takes place on the basis of select metrics.

1. Sanity Metrics: These are called invariant metrics. These are the metrics which should not change across control and experiment groups during the course of the experiment. If they change then there is something fundamentally wrong in the experiment setup.
2. Evaluation metrics: These can be chosen on the basis of business needs. These can be any of the following listed below:

#product-analytics #data-science #a-b-testing #product-management #experimentation #big data

What is GEEK

Buddha Community

The A’s and B’s of A/B Testing — A Beginner’s guide to experimentation

I recently completed Udacity’s course on A/B testing. The course started with a high-level understanding of what a typical A/B test entails before diving into specifics of each stage in the process of experimentation. Needless to say, it was a great learning experience! In this post, I am going to summarize my key learning from the course and explain how it benefits a lot of companies which are focused on improving user experience.

So, let us dive in!

So, tell me what is an A/B Test?

A/B testing is a method of experimentation to understand how user experiences change following any changes/variations made in the way they interact with a website, a mobile app etc. It is a very popular method to understand the level of existing customer experience and improve it based on certain relevant metrics.

Possible changes that companies may bring about are changing the layout of the website, changing the appearance/position of certain buttons on a particular webpage, change the ranking of options on the website etc. A/B testing helps the companies to evaluate whether these changes would be successful or not post-launch. This done by testing these changes on a subset of viewers for a certain period of time.

Okay, gotcha! But how will I evaluate these tests?

Evaluation takes place on the basis of select metrics.

1. Sanity Metrics: These are called invariant metrics. These are the metrics which should not change across control and experiment groups during the course of the experiment. If they change then there is something fundamentally wrong in the experiment setup.
2. Evaluation metrics: These can be chosen on the basis of business needs. These can be any of the following listed below:

#product-analytics #data-science #a-b-testing #product-management #experimentation #big data

The A’s and B’s of A/B Testing — A Beginner’s guide to experimentation

I recently completed Udacity’s course on A/B testing. It offers a high-level understanding of what a typical A/B test entails before diving into specifics of each stage in the process of experimentation. Needless to say, it was a great learning experience! In this post, I am going to summarize my key learning from the course and explain how it benefits a lot of companies which are focused on improving user experience.

So, let us dive in!

So, tell me what is an A/B Test?

A/B testing is a method of experimentation to understand how user experiences change following any changes/variations made in the way they interact with a website, a mobile app etc. It is a very popular method to understand the level of existing customer experience and improve it based on certain relevant metrics.

Possible changes that companies may bring about are changing the layout of the website, changing the appearance/position of certain buttons on a particular webpage, change the ranking of options on the website etc. A/B testing helps the companies to evaluate whether these changes would be successful or not post-launch. This done by testing these changes on a subset of viewers for a certain period of time.

Okay, gotcha! But how will I evaluate these tests?

#product-analytics #data-science #a-b-testing #product-management #experimentation

The Ultimate Guide to Multiclass A/B Testing

One essential skill that certainly useful for any data analytics professional to comprehend is the ability to perform an A/B testing and gather conclusions accordingly.

Before we proceed further, it might be useful to have a quick refresher on the definition of A/B testing in the first place. As the name suggests, we can think of A/B testing as the act of testing two alternatives, A and B, and use the test result to choose which alternative is superior to the other. For convenience, let’s call this type of A/B testing as the binary A/B testing.

Despite its name, A/B testing in fact can be made more general, i.e. to include more than two alternatives/classes to be tested. To name a few, analyzing click-through rate (CTR) from a multisegment digital campaign and redemption rate of various tiers of promos are two nice examples of such multiclass A/B testing.

The difference in the number of classes involved between binary and multiclass A / B testing also results in a slight difference in the statistical methods used to draw conclusions from them. While in binary testings one would straightforwardly use a simple t-test, it turns out that an additional (preliminary) step is needed for their multiclass counterparts.

In this post, I will give one possible strategy to deal with (gather conclusions from) multiclass A/B testings. I will demonstrate the step-by-step process through a concrete example so you can follow along. Are you ready?

#hypothesis-testing #a-b-testing #click-through-rate #t-test #chi-square-test #testing

What is Bitcoin Cash? - A Beginner’s Guide

Bitcoin Cash was created as a result of a hard fork in the Bitcoin network. The Bitcoin Cash network supports a larger block size than Bitcoin (currently 32mb as opposed to Bitcoin’s 1mb).

Later on, Bitcoin Cash forked into Bitcoin SV due to differences in how to carry on its developments.

That’s Bitcoin Cash in a nutshell. If you want a more detailed review watch the complete video. Here’s what I’ll cover:

0:50 - Bitcoin forks
2:06 - Bitcoin’s block size debate
3:35 - Big blocks camp
4:26 - Small blocks camp
5:16 - Small blocks vs. big blocks arguments
7:05 - How decisions are made in the Bitcoin network
10:14 - Block size debate resolution
11:06 - Bitcoin cash intro
11:28 - BTC vs. BCH
12:13 - Bitcoin Cash (ABC) vs. Bitcoin SV
13:09 - Conclusion
📺 The video in this post was made by 99Bitcoins
The origin of the article: https://www.youtube.com/watch?v=ONhbb4YVRLM
🔺 DISCLAIMER: The article is for information sharing. The content of this video is solely the opinions of the speaker who is not a licensed financial advisor or registered investment advisor. Not investment advice or legal advice.
Cryptocurrency trading is VERY risky. Make sure you understand these risks and that you are responsible for what you do with your money
🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
⭐ ⭐ ⭐The project is of interest to the community. Join to Get free ‘GEEK coin’ (GEEKCASH coin)!
☞ **-----CLICK HERE-----**⭐ ⭐ ⭐
Thanks for visiting and watching! Please don’t forget to leave a like, comment and share!

#bitcoin #blockchain #bitcoin cash #what is bitcoin cash? - a beginner’s guide #what is bitcoin cash #a beginner’s guide

Testing Microservices Applications

The shift towards microservices and modular applications makes testing more important and more challenging at the same time. You have to make sure that the microservices running in containers perform well and as intended, but you can no longer rely on conventional testing strategies to get the job done.

This is where new testing approaches are needed. Testing your microservices applications require the right approach, a suitable set of tools, and immense attention to details. This article will guide you through the process of testing your microservices and talk about the challenges you will have to overcome along the way. Let’s get started, shall we?

A Brave New World

Traditionally, testing a monolith application meant configuring a test environment and setting up all of the application components in a way that matched the production environment. It took time to set up the testing environment, and there were a lot of complexities around the process.

Testing also requires the application to run in full. It is not possible to test monolith apps on a per-component basis, mainly because there is usually a base code that ties everything together, and the app is designed to run as a complete app to work properly.

Microservices running in containers offer one particular advantage: universal compatibility. You don’t have to match the testing environment with the deployment architecture exactly, and you can get away with testing individual components rather than the full app in some situations.

Of course, you will have to embrace the new cloud-native approach across the pipeline. Rather than creating critical dependencies between microservices, you need to treat each one as a semi-independent module.

The only monolith or centralized portion of the application is the database, but this too is an easy challenge to overcome. As long as you have a persistent database running on your test environment, you can perform tests at any time.

Keep in mind that there are additional things to focus on when testing microservices.

• Microservices rely on network communications to talk to each other, so network reliability and requirements must be part of the testing.
• Automation and infrastructure elements are now added as codes, and you have to make sure that they also run properly when microservices are pushed through the pipeline
• While containerization is universal, you still have to pay attention to specific dependencies and create a testing strategy that allows for those dependencies to be included

Test containers are the method of choice for many developers. Unlike monolith apps, which lets you use stubs and mocks for testing, microservices need to be tested in test containers. Many CI/CD pipelines actually integrate production microservices as part of the testing process.

Contract Testing as an Approach

As mentioned before, there are many ways to test microservices effectively, but the one approach that developers now use reliably is contract testing. Loosely coupled microservices can be tested in an effective and efficient way using contract testing, mainly because this testing approach focuses on contracts; in other words, it focuses on how components or microservices communicate with each other.

Syntax and semantics construct how components communicate with each other. By defining syntax and semantics in a standardized way and testing microservices based on their ability to generate the right message formats and meet behavioral expectations, you can rest assured knowing that the microservices will behave as intended when deployed.

Ways to Test Microservices

It is easy to fall into the trap of making testing microservices complicated, but there are ways to avoid this problem. Testing microservices doesn’t have to be complicated at all when you have the right strategy in place.

There are several ways to test microservices too, including:

• Unit testing: Which allows developers to test microservices in a granular way. It doesn’t limit testing to individual microservices, but rather allows developers to take a more granular approach such as testing individual features or runtimes.
• Integration testing: Which handles the testing of microservices in an interactive way. Microservices still need to work with each other when they are deployed, and integration testing is a key process in making sure that they do.
• End-to-end testing: Which⁠—as the name suggests⁠—tests microservices as a complete app. This type of testing enables the testing of features, UI, communications, and other components that construct the app.

What’s important to note is the fact that these testing approaches allow for asynchronous testing. After all, asynchronous development is what makes developing microservices very appealing in the first place. By allowing for asynchronous testing, you can also make sure that components or microservices can be updated independently to one another.

#blog #microservices #testing #caylent #contract testing #end-to-end testing #hoverfly #integration testing #microservices #microservices architecture #pact #testing #unit testing #vagrant #vcr