Machine Learning for Automation Testing

Machine Learning for Automation Testing

The goals we are trying to achieve here by using Machine Learning for automation in testing are to dynamically write new test cases based on user interactions by data-mining their logs and their behavior on the application / service for which tests are to be written, live validation so that in case if an object is modified or removed or some other change like “modification in spelling” such as done by most of the IDE’s in the form of Intelli-sense like Visual Studio or Eclipse.

The goals we are trying to achieve here by using Machine Learning for automation in testing are to dynamically write new test cases based on user interactions by data-mining their logs and their behavior on the application / service for which tests are to be written, live validation so that in case if an object is modified or removed or some other change like “modification in spelling” such as done by most of the IDE’s in the form of Intelli-sense like Visual Studio or Eclipse.

Machine Learning in “Test Automation” can help prevent some of the following but not limited cases:

  1. Saving on Manual Labor of writing test cases
  2. Test cases are brittle so when something goes wrong a framework is most likely to either drop the testing at that point or to skip some steps which may result in wrong / failed result.
  3. Tests are not validated until and unless that test is run. So, if a script is written to check for an “OK” button then we wouldn’t know about its existence until we run the test.

The Machine Shall help recover from tests on the fly by applying fuzzy matching, that means if an object gets modified or removed then the program then the script must be able to find the closest object to the one it was looking for and then continue the test. For example, if a web services has options “small, medium, large” at first and the script was written according to that and if another choice i.e. “extra-large” is added then the script must be able to adapt to that and anticipate that change so that the test run can continue running without fail.

Know more about cross browser testing

HP Unified Function Testing is one of the well-known tools available in the market used for automated testing. It has a GUI interface for the same, other than that there are other tools like selenium libraries (implemented in several languages like Java, C++, C#, Python etc.) and Cucumber. Tools like these allow one to write their scripts and let the scripts take the job of testing from there by running through several cases.

Ideas for Implementation

Before starting the tests the system needs to learn the cases. We need to give it something to begin with. Before triggering the training part, we had to setup the system where some ads where shown to the normal users in a customized window of time, and during that time logs were collected and recorded which helped us in generating a gender ratio and age groups of people who looked at specific ads. The goal was to find out which age group and gender of users were stimulated to purchase something from the website after looking at certain ads presented to them. The results were saved as training data so that tests could be performed on them.

The machine was trained to write test cases based on the collected information. Training data was updated every once in a while so that tests could be run on the latest data based on different demographics and relevant ads could be delivered to potential customers. Had this been done manually one might have to make modifications in the script or add test cases manually each time trends changes or the website changes.

ML algorithms for automated testing

SVM: It belongs to the “linear classifier” family of ML algorithms that attempt to find a (linear) hyperplane that separates examples from different classes. In the learning phase, SVM treats the training data as a vector of k-1 dimensions. And the goal is to find maximum margin (distance). SVM technique is mostly used for the binary classification. Other than this we have MartiRank, a ranking algorithm, in the learning phase. It takes a number of rounds and during each round/iteration data is broken down into N sub-lists, each sub-list containing 1/n of the total number of device / app failures.

In case of Regression testing a suite of test cases needed to be developed, development of MartiRank is an ongoing process and has been used to detect new bugs. Ex: A dev might have refactored some code in the application and put into a new function. Regression testing showed us that the resulting models were different from the previous ones.

When we write test cases we test how the software is supposed to behave theoretically and there’s no real data with us, some of the test cases might never be used in real life and some that missed the test cases might be the most important ones and that is why data-mining the logs and letting the machine write test cases according to those logs automatically saves a lot of man-hours and helps in practical testing. Services like HockeyApp & TestFlight are providing automated mobile app testing as a service.

As for GUI tests there are some research papers out there that talk about DeepLearning and ReinforcementLearning for automation of the test. The systems that were being tested were first data-mined to get the meaningful clicks, texts and button pushes on the GUI interface which generated a good amount of training data. That data was then used to perform tests on the software for a few hours. Best part was that there were no need for models or test cases to be written and the bugs were being found as the time passed by, but some of the cases were not being tested which can be due to lack of training data. The reinforcement approach improved the testing as they were running through multiple iterations.

Intel and Nvidia have been investing heavily in hardware solutions that can aid Deep Learning and related algorithms to achieve results more quickly. Moving from a mobile first world to an AI first world. We know that for testing a certain product whether that be a small calculator there can never be enough number of right test cases and that is why developers and testers are encouraged to write more and more test cases in order to make their product more stable. Paul Graham once suggested the used of Bayesian Filter for filtering out spam emails, thousands of emails were fed to the system and it was made to learn then tests were performed on that training data to make sure that the filter was fool-proof. Web crawlers move through different websites looking for 404 or other errors all the time, updating their indexes and updating their test cases in real time.

Originally Published at LambdaTest

JavaScript Testing using Selenium WebDriver, Mocha and NodeJS

JavaScript Testing using Selenium WebDriver, Mocha and NodeJS

In case you are looking to write a functional test in JavaScript, the following tutorial provides UI automation engineers with the perfect structural reference material for JavaScript testing with Selenium WebDriver 3, Mocha and NodeJS.

In case you are looking to write a functional test in JavaScript, the following tutorial provides UI automation engineers with the perfect structural reference material for JavaScript testing with Selenium WebDriver 3, Mocha and NodeJS.

These days, JavaScript is a ubiquitous web language which seems to overcome its ‘notorious’ past and has become a more solid platform not only for client-side, but for server domains too. Mocha.js, or simply Mocha, is a feature-rich JavaScript test framework running on Node.js, which provides the platform and the API for building standalone applications server-side using Google’s V8 JavaScript engine at its base.

*Note: to get started with this JavaScript tutorial, you’ll need to be familiar with the basics of NodeJS and the JavaScript programming language.

Tutorial Overview:

1. Mocha Test Construction
  • Introduction
  • Installation
  • Installing Chai Assertion Module
  • Test suite and Test Case Structure
  • Constructing Tests with Mocha
  • Running Mocha’s Test Suite and Test Cases
  • Managing Syncing of Async Testing Code
2. Using Javascript Selenium 3 API Integrated with MochaJS
  • Selenium Introduction
  • Selenium Installation
  • WebDriver Construction
  • Integrating MochaJS with Selenium WebDriver 3

Versions used:

  • Node version used: 6.10.1 (LTS)
  • Mocha: 2.5.3
  • WebDriverJS: 3.3.0
1. Constructing Tests with Mocha

Introduction to Mocha

As mentioned, Mocha is a JavaScript test framework that runs tests on Node. Mocha comes in the form of a Node package via npm, allowing you to use any library for assertions as a replacement to Node’s standard ‘assert’ function, such as ChaiJS.

Mocha provides an API, which specifies a way to structure the testing code into test suites and test case modules for execution, and later on to produce a test report. Mocha provides two modes for running: either by command line (CLI) or programmatically (Mocha API).

Install Mocha

If Mocha is to be used in CLI, then it should be installed globally as Node.js.

npm install -g mocha 

Install Chai Assertion Module

npm install --save chai 

The –save option is used to install the module in the project’s scope and not globally.

Test Suite and Test Case Structure

In Mocha, a test suite is defined by the ‘describe’ keyword which accepts a callback function. A test suite can contain child / inner test suites, which can contain their own child test suites, etc. A test case is denoted by the ‘it’ function, which accepts a callback function and contains the testing code.

Mocha supports test suite setup and test case setup functions. A test suite setup is denoted by before while a test case setup applies beforeEach. beforeEach is actually a common setup for every case in the suite, and will be executed before each case.

As with the setup, Mocha supports test suite and test case teardown functions. A test suite teardown is denoted by after, while a test case teardown is implemented with afterEach, functions that are executed after a test suite and after each test case respectively.

Create a file that will ‘host’ the test suite, e.g. test_suite.js, and write the following to it;

describe("Inner Suite 1", function(){

    before(function(){

        // do something before test suite execution
        // no matter if there are failed cases

    });

    after(function(){

        // do something after test suite execution is finished
        // no matter if there are failed cases

    });

    beforeEach(function(){

        // do something before test case execution
        // no matter if there are failed cases

    });

    afterEach(function(){

        // do something after test case execution is finished
        // no matter if there are failed cases

    });

    it("Test-1", function(){

        // test Code
        // assertions

    });

    it("Test-2", function(){

        // test Code
        // assertions

    });

    it("Test-3", function(){

        // test Code
        // assertions

    });

});

Running Mocha Test Suite and Test Cases

Mocha supports execution of tests in three ways: Whole Test Suite file, tests filtered by “grep” patterns and tests grep filtering looking in a directory tree (recursive option)

Run whole Test Suite file:

mocha /path/to/test_suite.js 

Run a specific suite or test from a specific suite file.

If a suite is selected then all the child suites and/or tests will be executed.

mocha -g “Test-2” /path/to/test_suite.js 

Run a specific suite or test file by searching recursively in a directory tree.

mocha --recursive -g “Test-2” /directory/ 

For extensive CLI options:

mocha –-help 

Managing Syncing of Async Testing Code

In case async functions are used with Mocha and not handled properly, you may find yourself struggling. If asyncing code (e.g. http requests, files, selenium, etc.) is to be used in a test case, follow these guidelines to overcome unexpected results:

1. **done** Function

In your test function (it) you need to pass the done function down the callback chain — this ensures it is executed after your last step.

The example below emphasizes the done functionality. In this case three seconds of timeout will occur at the end of the test function.

it(‘Test-1’, function(done){

    setTimeout(function(){

        console.log(“timeout!”);

  // mocha will wait for done to be called before exiting function.
        done();     
    }, 3000);

});

2. Return Promise

Returning a promise is another way to ensure Mocha has executed all code lines when async functions are used (‘done’ function is not needed in this case.)

it(‘Test-1’, function(done){

    var promise;
    promise = new Promise(function(resolve, reject){
        setTimeout(function(){

            console.log("Timeout");
            resolve();

        }, 3000);

    });
    // mocha will wait for the promise to be resolved before exiting
    return promise;  
});

2. Javascript Selenium 3 Integration with MochaJS

Selenium Introduction

Selenium is a library that controls a web browser and emulates the user’s behavior. More specifically, Selenium offers specific language library APIs called ‘bindings’ for the user. These ‘bindings’ act as a client in order to perform requests to intermediate components and acting as servers in order to finally control a Browser.

The intermediate components could be the actual webdriver, found natively in each Selenium package, the selenium-standalone-server, as well as vendor native browser controlling drivers — such as Geckodriver for Mozilla, chromedriver for Chrome, etc. Moreover, Selenium webdriver communicates with browser drivers via ‘JsonWired Protocol’ and becomes a W3C Web Standard.

Selenium Installation

Before diving any deeper into Selenium integration with MochaJS, we will take a quick look into Selenium implementation with NodeJS.

In order to use the Selenium API for JavaScript (or Selenium JavaScript bindings), we should install the appropriate module:

npm install selenium-webdriver 

At this point, it should be clarified that Javascript Selenium WebDriver can also be referred to as Webdriverjs (although not in npm). Webdrivejs is different than other libs/modules, such as WebdriverIO, Protractor, etc. selenium-webdriver is the official open-source base JavaScript Selenium library while the others are wrapper libraries/frameworks that are built on top of webdriverjs API, claiming to enhance usability and maintenance.

In NodeJS code, the module is used by:

require(‘selenium-webdriver’) 

WebDriver Construction

In order to be able to use Selenium, we should build the appropriate ‘webdriver’ object which will then control our browser. Below, we can see how we use the “Builder” pattern to construct a webdriver object by chaining several functions.

Builder with Options

var webdriver = require('selenium-webdriver')
var chrome = require('selenium-webdriver/chrome'),
var firefox = require('selenium-webdriver/firefox');

var driver = new webdriver.Builder()
    .forBrowser(‘firefox’)
    .setFirefoxOptions( /* … */)
    .setChromeOptions( /* … */)
    .build();

In the code above, we have managed to build a WebDriver object which aggregates configuration for more than one browser (notice the ‘options’ methods), despite the fact that the forBrowser() method explicitly sets firefox.

The user can set the SELENIUM_BROWSER environmental variable on runtime to set the desired browser. It will override any option set by forBrowser, since we have already configured multiple browser capabilities by set Options.

The browser properties can have several types of information depending on the browser under test. For example, in Mozilla’s properties we can set the desired ‘profile’ configuration as follows:

var profile = new firefox.Profile( /* … path to firefox local profile … */);
var firefoxOptions = new firefox Options().setProfile(profile);

Then, in the above Builder snippet we can add:

‘setFirefoxOptions( firefoxOptions )’ 

Builder with Capabilities

Selenium WebDriver JavaScript API documents several ways that a webdriver could be built. One more possible way is by setting all the required driver configurations in capabilities:

var driver = new webdriver.Builder().
    .withCapabilities( { ‘browserName’ : ‘firefox’ } )
    .build();

Note that if setOptions are set after withCapabilities, the configurations will be overridden (e.g. proxy configurations).

Selenium WebDriver Control Flow and Promise Management

Since JavaScript and NodeJS are based on asynchronous principles, Selenium WebDriver behaves in a similar way. In order to avoid callback pyramids and to assist a test engineer with the scripting experience as well as code readability and maintainability, Selenium WebDriver objects incorporate a promise manager that uses a ‘ControlFlow’. ‘ControlFlow’ is a class responsible for the sequential execution of the asynchronous webdriver commands.

Practically, each command is executed on the driver object and a promise is returned. The next commands do not need to be nested in ‘thens’, unless there is a need to handle a promise resolved value as follows:

driver.get("http://www.google.com");
driver.getTitle().then(function( title ) {

    // google page title should be printed 
    console.log(title)

});

driver.quit();

Pointers for JavaScript Testing with Selenium WebDriver and Mocha

  1. driver is a webdriver object, not a promise object
  2. driver.getTitle() or driver.get(url), or any other Selenium command, returns a promise object!

This means that we can perform the following:

var titlePromise = driver.getTitle();
titlePromise.then(function(title){

    console.log(title);

});
  1. Additionally, since driver is asyncing in its base, the following will not work:
var title = driver.getTitle();
expect (title).equals("Google");

Note: title is a promise object and not an actual resolved value.

MochaJS + Selenium WebDriver

Generally speaking, Selenium WebDriver can be integrated with MochaJS since it is used in any plain NodeJS script. However, since Mocha doesn’t know when an asynchronous function has finished before a done() is called or a promise is returned, we have to be very careful with handling.

Promise Based

Selenium commands are registered automatically, to assure webdriver commands are executed in the correct sequential order a promise should be returned.

The code below shows Mocha’s (before, beforeEach, after, afterEach) or test case body it hooks.

describe( 'Test Suite' , function(){

    before(function(){

        driver.get( my_service );
        driver.findElement(webdriver.By.id(username)).sendKeys(my_username);

        // a promise is returned while ‘click’ action
        // is registered in ‘driver’ object
        return driver.findElement(webdriver.By.id(submit)).click();
    });

    after(function(){

        return driver.quit();

    });

    it( 'Test Case', function(){

        driver.getTitle().then(function(title){
            expect(title).equals(my_title);
        })

The following actions will be executed:

  1. Browser page of “my_service” is loaded
  2. Text Field with id ‘username’ is located
  3. Text Field with id ‘username’ is filled with ‘my_username’
  4. Page title is retrieved and checked for equality against ‘my_title’
  5. WebDriver quits and browser window is closed. Browser process is terminated.

Selenium Webdriver Support for MochaJS

In order to perform JavaScript testing with Selenium WebDriver and Mocha in a simple way, WebDriver facilitates usage with MochaJS by wrapping around MochaJS test functions (before, beforeEach, it, etc.) with a test object. This creates a scope that provides awareness that WebDriver is being used. Therefore, there is no need for promise returns.

First, the corresponding module should be loaded:

var test = require('selenium-webdriver/testing'); 

All the function of Mocha are preceded by ‘test.’ as follows:

test.before()
test.describe()

And so on. Then, the above code is fully re-written as:

test.describe( 'Test Suite' , function(){

    test.before(function(){

        driver.get( my_service );
        driver.findElement(webdriver.By.id(username)).sendKeys(my_username);
        driver.findElement(webdriver.By.id(submit)).click();
    });

    test.after(function(){
        driver.quit();
    });

    test.it( 'Test Case' , function(){

        driver.getTitle().then(function(title){
            expect(title).equals(my_title);
        })

        driver.sleep();
    });

});
Conclusion

In this tutorial we got a chance to experience JavaScript testing with Selenium WebDriver and MochaJS. We should keep in mind the main difference when comparing to other programming language bindings, due to the asynchronous nature of NodeJS, MochaJS and Selenium WebDriver.

As long as we keep returning promises in any function which creates a promise (either a custom test lib function or a MochaJS hook/testcase), Mocha will execute them in the correct order.

Other frameworks such as WebdriverIO, Protractor and CodeseptJS provide wrapper solutions that hide some configurations from the user, and provide some promise-enhanced handling for a better scripting experience that many test automation experts might find helpful.

Machine Learning Full Course - Learn Machine Learning

Machine Learning Full Course - Learn Machine Learning

This complete Machine Learning full course video covers all the topics that you need to know to become a master in the field of Machine Learning.

Machine Learning Full Course | Learn Machine Learning | Machine Learning Tutorial

It covers all the basics of Machine Learning (01:46), the different types of Machine Learning (18:32), and the various applications of Machine Learning used in different industries (04:54:48).This video will help you learn different Machine Learning algorithms in Python. Linear Regression, Logistic Regression (23:38), K Means Clustering (01:26:20), Decision Tree (02:15:15), and Support Vector Machines (03:48:31) are some of the important algorithms you will understand with a hands-on demo. Finally, you will see the essential skills required to become a Machine Learning Engineer (04:59:46) and come across a few important Machine Learning interview questions (05:09:03). Now, let's get started with Machine Learning.

Below topics are explained in this Machine Learning course for beginners:

  1. Basics of Machine Learning - 01:46

  2. Why Machine Learning - 09:18

  3. What is Machine Learning - 13:25

  4. Types of Machine Learning - 18:32

  5. Supervised Learning - 18:44

  6. Reinforcement Learning - 21:06

  7. Supervised VS Unsupervised - 22:26

  8. Linear Regression - 23:38

  9. Introduction to Machine Learning - 25:08

  10. Application of Linear Regression - 26:40

  11. Understanding Linear Regression - 27:19

  12. Regression Equation - 28:00

  13. Multiple Linear Regression - 35:57

  14. Logistic Regression - 55:45

  15. What is Logistic Regression - 56:04

  16. What is Linear Regression - 59:35

  17. Comparing Linear & Logistic Regression - 01:05:28

  18. What is K-Means Clustering - 01:26:20

  19. How does K-Means Clustering work - 01:38:00

  20. What is Decision Tree - 02:15:15

  21. How does Decision Tree work - 02:25:15 

  22. Random Forest Tutorial - 02:39:56

  23. Why Random Forest - 02:41:52

  24. What is Random Forest - 02:43:21

  25. How does Decision Tree work- 02:52:02

  26. K-Nearest Neighbors Algorithm Tutorial - 03:22:02

  27. Why KNN - 03:24:11

  28. What is KNN - 03:24:24

  29. How do we choose 'K' - 03:25:38

  30. When do we use KNN - 03:27:37

  31. Applications of Support Vector Machine - 03:48:31

  32. Why Support Vector Machine - 03:48:55

  33. What Support Vector Machine - 03:50:34

  34. Advantages of Support Vector Machine - 03:54:54

  35. What is Naive Bayes - 04:13:06

  36. Where is Naive Bayes used - 04:17:45

  37. Top 10 Application of Machine Learning - 04:54:48

  38. How to become a Machine Learning Engineer - 04:59:46

  39. Machine Learning Interview Questions - 05:09:03

Machine Learning Tutorial - Learn Machine Learning - Intellipaat

Machine Learning Tutorial - Learn Machine Learning - Intellipaat

This Machine Learning tutorial for beginners will enable you to learn Machine Learning algorithms with python examples. Become a pro in Machine Learning.

Mastering the Machine Learning Course would easily develop one's career. This is the reason why studying Machine Learning Tutorial becomes so important in the career of a particular student.
Making a part of the machine learning course would enact and studying the Machine Learning Tutorial would make one carve out a new niche.