Vern  Greenholt

Vern Greenholt

1598523540

The A’s and B’s of A/B Testing — A Beginner’s guide to experimentation

I recently completed Udacity’s course on A/B testing. The course started with a high-level understanding of what a typical A/B test entails before diving into specifics of each stage in the process of experimentation. Needless to say, it was a great learning experience! In this post, I am going to summarize my key learning from the course and explain how it benefits a lot of companies which are focused on improving user experience.

So, let us dive in!

So, tell me what is an A/B Test?

A/B testing is a method of experimentation to understand how user experiences change following any changes/variations made in the way they interact with a website, a mobile app etc. It is a very popular method to understand the level of existing customer experience and improve it based on certain relevant metrics.

Possible changes that companies may bring about are changing the layout of the website, changing the appearance/position of certain buttons on a particular webpage, change the ranking of options on the website etc. A/B testing helps the companies to evaluate whether these changes would be successful or not post-launch. This done by testing these changes on a subset of viewers for a certain period of time.

Okay, gotcha! But how will I evaluate these tests?

Image for post

Evaluation takes place on the basis of select metrics.

  1. Sanity Metrics: These are called invariant metrics. These are the metrics which should not change across control and experiment groups during the course of the experiment. If they change then there is something fundamentally wrong in the experiment setup.
  2. Evaluation metrics: These can be chosen on the basis of business needs. These can be any of the following listed below:

#product-analytics #data-science #a-b-testing #product-management #experimentation #big data

What is GEEK

Buddha Community

The A’s and B’s of A/B Testing — A Beginner’s guide to experimentation
Mike  Kozey

Mike Kozey

1656151740

Test_cov_console: Flutter Console Coverage Test

Flutter Console Coverage Test

This small dart tools is used to generate Flutter Coverage Test report to console

How to install

Add a line like this to your package's pubspec.yaml (and run an implicit flutter pub get):

dev_dependencies:
  test_cov_console: ^0.2.2

How to run

run the following command to make sure all flutter library is up-to-date

flutter pub get
Running "flutter pub get" in coverage...                            0.5s

run the following command to generate lcov.info on coverage directory

flutter test --coverage
00:02 +1: All tests passed!

run the tool to generate report from lcov.info

flutter pub run test_cov_console
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
 print_cov_constants.dart                    |    0.00 |    0.00 |    0.00 |    no unit testing|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

Optional parameter

If not given a FILE, "coverage/lcov.info" will be used.
-f, --file=<FILE>                      The target lcov.info file to be reported
-e, --exclude=<STRING1,STRING2,...>    A list of contains string for files without unit testing
                                       to be excluded from report
-l, --line                             It will print Lines & Uncovered Lines only
                                       Branch & Functions coverage percentage will not be printed
-i, --ignore                           It will not print any file without unit testing
-m, --multi                            Report from multiple lcov.info files
-c, --csv                              Output to CSV file
-o, --output=<CSV-FILE>                Full path of output CSV file
                                       If not given, "coverage/test_cov_console.csv" will be used
-t, --total                            Print only the total coverage
                                       Note: it will ignore all other option (if any), except -m
-p, --pass=<MINIMUM>                   Print only the whether total coverage is passed MINIMUM value or not
                                       If the value >= MINIMUM, it will print PASSED, otherwise FAILED
                                       Note: it will ignore all other option (if any), except -m
-h, --help                             Show this help

example run the tool with parameters

flutter pub run test_cov_console --file=coverage/lcov.info --exclude=_constants,_mock
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

report for multiple lcov.info files (-m, --multi)

It support to run for multiple lcov.info files with the followings directory structures:
1. No root module
<root>/<module_a>
<root>/<module_a>/coverage/lcov.info
<root>/<module_a>/lib/src
<root>/<module_b>
<root>/<module_b>/coverage/lcov.info
<root>/<module_b>/lib/src
...
2. With root module
<root>/coverage/lcov.info
<root>/lib/src
<root>/<module_a>
<root>/<module_a>/coverage/lcov.info
<root>/<module_a>/lib/src
<root>/<module_b>
<root>/<module_b>/coverage/lcov.info
<root>/<module_b>/lib/src
...
You must run test_cov_console on <root> dir, and the report would be grouped by module, here is
the sample output for directory structure 'with root module':
flutter pub run test_cov_console --file=coverage/lcov.info --exclude=_constants,_mock --multi
---------------------------------------------|---------|---------|---------|-------------------|
File                                         |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|
---------------------------------------------|---------|---------|---------|-------------------|
File - module_a -                            |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|
---------------------------------------------|---------|---------|---------|-------------------|
File - module_b -                            |% Branch | % Funcs | % Lines | Uncovered Line #s |
---------------------------------------------|---------|---------|---------|-------------------|
lib/src/                                     |         |         |         |                   |
 print_cov.dart                              |  100.00 |  100.00 |   88.37 |...,149,205,206,207|
lib/                                         |         |         |         |                   |
 test_cov_console.dart                       |    0.00 |    0.00 |    0.00 |    no unit testing|
---------------------------------------------|---------|---------|---------|-------------------|
 All files with unit testing                 |  100.00 |  100.00 |   88.37 |                   |
---------------------------------------------|---------|---------|---------|-------------------|

Output to CSV file (-c, --csv, -o, --output)

flutter pub run test_cov_console -c --output=coverage/test_coverage.csv

#### sample CSV output file:
File,% Branch,% Funcs,% Lines,Uncovered Line #s
lib/,,,,
test_cov_console.dart,0.00,0.00,0.00,no unit testing
lib/src/,,,,
parser.dart,100.00,100.00,97.22,"97"
parser_constants.dart,100.00,100.00,100.00,""
print_cov.dart,100.00,100.00,82.91,"29,49,51,52,171,174,177,180,183,184,185,186,187,188,279,324,325,387,388,389,390,391,392,393,394,395,398"
print_cov_constants.dart,0.00,0.00,0.00,no unit testing
All files with unit testing,100.00,100.00,86.07,""

Installing

Use this package as an executable

Install it

You can install the package from the command line:

dart pub global activate test_cov_console

Use it

The package has the following executables:

$ test_cov_console

Use this package as a library

Depend on it

Run this command:

With Dart:

 $ dart pub add test_cov_console

With Flutter:

 $ flutter pub add test_cov_console

This will add a line like this to your package's pubspec.yaml (and run an implicit dart pub get):

dependencies:
  test_cov_console: ^0.2.2

Alternatively, your editor might support dart pub get or flutter pub get. Check the docs for your editor to learn more.

Import it

Now in your Dart code, you can use:

import 'package:test_cov_console/test_cov_console.dart';

example/lib/main.dart

import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        // This is the theme of your application.
        //
        // Try running your application with "flutter run". You'll see the
        // application has a blue toolbar. Then, without quitting the app, try
        // changing the primarySwatch below to Colors.green and then invoke
        // "hot reload" (press "r" in the console where you ran "flutter run",
        // or simply save your changes to "hot reload" in a Flutter IDE).
        // Notice that the counter didn't reset back to zero; the application
        // is not restarted.
        primarySwatch: Colors.blue,
        // This makes the visual density adapt to the platform that you run
        // the app on. For desktop platforms, the controls will be smaller and
        // closer together (more dense) than on mobile platforms.
        visualDensity: VisualDensity.adaptivePlatformDensity,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key? key, required this.title}) : super(key: key);

  // This widget is the home page of your application. It is stateful, meaning
  // that it has a State object (defined below) that contains fields that affect
  // how it looks.

  // This class is the configuration for the state. It holds the values (in this
  // case the title) provided by the parent (in this case the App widget) and
  // used by the build method of the State. Fields in a Widget subclass are
  // always marked "final".

  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      // This call to setState tells the Flutter framework that something has
      // changed in this State, which causes it to rerun the build method below
      // so that the display can reflect the updated values. If we changed
      // _counter without calling setState(), then the build method would not be
      // called again, and so nothing would appear to happen.
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    // This method is rerun every time setState is called, for instance as done
    // by the _incrementCounter method above.
    //
    // The Flutter framework has been optimized to make rerunning build methods
    // fast, so that you can just rebuild anything that needs updating rather
    // than having to individually change instances of widgets.
    return Scaffold(
      appBar: AppBar(
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text(widget.title),
      ),
      body: Center(
        // Center is a layout widget. It takes a single child and positions it
        // in the middle of the parent.
        child: Column(
          // Column is also a layout widget. It takes a list of children and
          // arranges them vertically. By default, it sizes itself to fit its
          // children horizontally, and tries to be as tall as its parent.
          //
          // Invoke "debug painting" (press "p" in the console, choose the
          // "Toggle Debug Paint" action from the Flutter Inspector in Android
          // Studio, or the "Toggle Debug Paint" command in Visual Studio Code)
          // to see the wireframe for each widget.
          //
          // Column has various properties to control how it sizes itself and
          // how it positions its children. Here we use mainAxisAlignment to
          // center the children vertically; the main axis here is the vertical
          // axis because Columns are vertical (the cross axis would be
          // horizontal).
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headline4,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

Author: DigitalKatalis
Source Code: https://github.com/DigitalKatalis/test_cov_console 
License: BSD-3-Clause license

#flutter #dart #test 

The A’s and B’s of A/B Testing — A Beginner’s guide to experimentation

I recently completed Udacity’s course on A/B testing. It offers a high-level understanding of what a typical A/B test entails before diving into specifics of each stage in the process of experimentation. Needless to say, it was a great learning experience! In this post, I am going to summarize my key learning from the course and explain how it benefits a lot of companies which are focused on improving user experience.

So, let us dive in!

So, tell me what is an A/B Test?

A/B testing is a method of experimentation to understand how user experiences change following any changes/variations made in the way they interact with a website, a mobile app etc. It is a very popular method to understand the level of existing customer experience and improve it based on certain relevant metrics.

Possible changes that companies may bring about are changing the layout of the website, changing the appearance/position of certain buttons on a particular webpage, change the ranking of options on the website etc. A/B testing helps the companies to evaluate whether these changes would be successful or not post-launch. This done by testing these changes on a subset of viewers for a certain period of time.

Okay, gotcha! But how will I evaluate these tests?

Image for post

#product-analytics #data-science #a-b-testing #product-management #experimentation

Vern  Greenholt

Vern Greenholt

1598523540

The A’s and B’s of A/B Testing — A Beginner’s guide to experimentation

I recently completed Udacity’s course on A/B testing. The course started with a high-level understanding of what a typical A/B test entails before diving into specifics of each stage in the process of experimentation. Needless to say, it was a great learning experience! In this post, I am going to summarize my key learning from the course and explain how it benefits a lot of companies which are focused on improving user experience.

So, let us dive in!

So, tell me what is an A/B Test?

A/B testing is a method of experimentation to understand how user experiences change following any changes/variations made in the way they interact with a website, a mobile app etc. It is a very popular method to understand the level of existing customer experience and improve it based on certain relevant metrics.

Possible changes that companies may bring about are changing the layout of the website, changing the appearance/position of certain buttons on a particular webpage, change the ranking of options on the website etc. A/B testing helps the companies to evaluate whether these changes would be successful or not post-launch. This done by testing these changes on a subset of viewers for a certain period of time.

Okay, gotcha! But how will I evaluate these tests?

Image for post

Evaluation takes place on the basis of select metrics.

  1. Sanity Metrics: These are called invariant metrics. These are the metrics which should not change across control and experiment groups during the course of the experiment. If they change then there is something fundamentally wrong in the experiment setup.
  2. Evaluation metrics: These can be chosen on the basis of business needs. These can be any of the following listed below:

#product-analytics #data-science #a-b-testing #product-management #experimentation #big data

The Ultimate Guide to Multiclass A/B Testing

One essential skill that certainly useful for any data analytics professional to comprehend is the ability to perform an A/B testing and gather conclusions accordingly.

Before we proceed further, it might be useful to have a quick refresher on the definition of A/B testing in the first place. As the name suggests, we can think of A/B testing as the act of testing two alternatives, A and B, and use the test result to choose which alternative is superior to the other. For convenience, let’s call this type of A/B testing as the binary A/B testing.

Despite its name, A/B testing in fact can be made more general, i.e. to include more than two alternatives/classes to be tested. To name a few, analyzing click-through rate (CTR) from a multisegment digital campaign and redemption rate of various tiers of promos are two nice examples of such multiclass A/B testing.

The difference in the number of classes involved between binary and multiclass A / B testing also results in a slight difference in the statistical methods used to draw conclusions from them. While in binary testings one would straightforwardly use a simple t-test, it turns out that an additional (preliminary) step is needed for their multiclass counterparts.

In this post, I will give one possible strategy to deal with (gather conclusions from) multiclass A/B testings. I will demonstrate the step-by-step process through a concrete example so you can follow along. Are you ready?

#hypothesis-testing #a-b-testing #click-through-rate #t-test #chi-square-test #testing

Abigail betty

Abigail betty

1624226400

What is Bitcoin Cash? - A Beginner’s Guide

Bitcoin Cash was created as a result of a hard fork in the Bitcoin network. The Bitcoin Cash network supports a larger block size than Bitcoin (currently 32mb as opposed to Bitcoin’s 1mb).

Later on, Bitcoin Cash forked into Bitcoin SV due to differences in how to carry on its developments.

That’s Bitcoin Cash in a nutshell. If you want a more detailed review watch the complete video. Here’s what I’ll cover:

0:50 - Bitcoin forks
2:06 - Bitcoin’s block size debate
3:35 - Big blocks camp
4:26 - Small blocks camp
5:16 - Small blocks vs. big blocks arguments
7:05 - How decisions are made in the Bitcoin network
10:14 - Block size debate resolution
11:06 - Bitcoin cash intro
11:28 - BTC vs. BCH
12:13 - Bitcoin Cash (ABC) vs. Bitcoin SV
13:09 - Conclusion
📺 The video in this post was made by 99Bitcoins
The origin of the article: https://www.youtube.com/watch?v=ONhbb4YVRLM
🔺 DISCLAIMER: The article is for information sharing. The content of this video is solely the opinions of the speaker who is not a licensed financial advisor or registered investment advisor. Not investment advice or legal advice.
Cryptocurrency trading is VERY risky. Make sure you understand these risks and that you are responsible for what you do with your money
🔥 If you’re a beginner. I believe the article below will be useful to you ☞ What You Should Know Before Investing in Cryptocurrency - For Beginner
⭐ ⭐ ⭐The project is of interest to the community. Join to Get free ‘GEEK coin’ (GEEKCASH coin)!
☞ **-----CLICK HERE-----**⭐ ⭐ ⭐
Thanks for visiting and watching! Please don’t forget to leave a like, comment and share!

#bitcoin #blockchain #bitcoin cash #what is bitcoin cash? - a beginner’s guide #what is bitcoin cash #a beginner’s guide