What you need to know about automation testing in CI/CD

“If things seem under control, you’re just not going fast enough.” —Mario Andretti Test automation means focusing continuously on detecting defects, errors, and bugs as early and quickly as possible in the software development process. This is done using tools that pursue quality as the highest value and are put in place to ensure quality—not just pursue it.

#what #testing #ci #cd

What is GEEK

Buddha Community

Top 10 Automation Testing Tools: 2020 Edition

The demand for delivering quality software faster — or “Quality at Speed” — requires organizations to search for solutions in Agile, continuous integration (CI), and DevOps methodologies. Test automation is an essential part of these aspects. The latest World Quality Report 2018–2019 suggests that test automation is the biggest bottleneck to deliver “Quality at Speed,” as it is an enabler of successful Agile and DevOps adoption.

Test automation cannot be realized without good tools; as they determine how automation is performed and whether the benefits of automation can be delivered. Test automation tools is a crucial component in the DevOps toolchain. The current test automation trends have increased in applying artificial intelligence and machine learning (AI/ML) to offer advanced capabilities for test optimization, intelligent test generation, execution, and reporting. It will be worthwhile to understand which tools are best poised to take advantage of these trends.****

#automation-testing #automation-testing-tools #testing #testing-tools #selenium #open-source #test-automation #automated-testing

How To Build CI/CD Pipeline With TeamCity For Selenium Test Automation

Continuous Integration/Continuous Deployment (CI/CD) has become an essential part of modern software development cycles. As a part of continuous integration, the developer should ensure that the Integration should not break the existing code because this could lead to a negative impact on the overall quality of the project. In order to show how the integration process works, we’ll take an example of a well-known continuous integration tool, TeamCity. In this article, we will learn TeamCity concepts and integrate our test suites with TeamCity for test automation by leveraging LambdaTest cloud-based Selenium grid.

Source

There are numerous best CI/CD tools available for building high-quality code and narrowing the gap between development and impacted teams. Besides establishing a DevOps culture in the organizations, teams enhance it by implementing best CI/CD practices throughout the Software Development Life Cycle(SDLC). These practices help the teams accelerate product development, automate the processes, and improve overall productivity.

#ci/cd #automation #how to build ci/cd pipeline with teamcity for selenium test automation #selenium test automation #teamcity

Where To Learn Test Programming — July 2020 Edition

What do you do when you have lots of free time on your hands? Why not learn test programming strategies and approaches?

When you’re looking for places to learn test programming, Test Automation University has you covered. From API testing through visual validation, you can hone your skills and learn new approaches on TAU.

We introduced five new TAU courses from April through June, and each of them can help you expand your knowledge, learn a new approach, and improve your craft as a test automation engineer. They are:

These courses add to the other three courses we introduced in January through March 2020:

• IntelliJ for Test Automation Engineers (3 hrs 41 min)
• Cucumber with JavaScript (1 hr 22 min)
• Python Programming (2 hrs)

Each of these courses can give you a new set of skills.

Let’s look at each in a little detail.

Mobile Automation With Appium in JavaScript

Orane Findley teaches Mobile Automation with Appium in JavaScript. Orane walks through all the basics of Appium, starting with what it is and where it runs.

“Appium is an open-source tool for automating native, web, and hybrid applications on different platforms.”

In the introduction, Orane describes the course parts:

• Setup and Dependencies — installing Appium and setting up your first project
• Working with elements by finding them, sending values, clicking, and submitting
• Creating sessions, changing screen orientations, and taking screenshots
• Timing, including TimeOuts and Implicit Waits
• Collecting attributes and data from an element
• Selecting and using element states
• Reviewing everything to make it all make sense

The first chapter, broken into five parts, gets your system ready for the rest of the course. You’ll download and install a Java Developer Kit, a stable version of Node.js, Android Studio and Emulator (for a mobile device emulator), Visual Studio Code for an IDE, Appium Server, and a sample Appium Android Package Kit. If you get into trouble, you can use the Test Automation University Slack channel to get help from Orane. Each subchapter contains the links to get to the proper software. Finally, Orane has you customize your configuration for the course project.

Chapter 2 deals with elements and screen interactions for your app. You can find elements on the page, interact with those elements, and scroll the page to make other elements visible. Orane breaks the chapter into three distinct subchapters so you can become competent with each part of finding, scrolling, and interacting with the app. The quiz comes at the end of the third subchapter.

The remaining chapters each deal with specific bullets listed above: sessions and screen capture, timing, element attributes, and using element states. The final summary chapter ensures you have internalized the key takeaways from the course. Each of these chapters includes its quiz.

When you complete this course successfully, you will have both a certificate of completion and the code infrastructure available on your system to start testing mobile apps using Appium.

Selenium WebDriver With Python

Andrew Knight, who blogs as The Automation Panda, teaches the course on Selenium WebDriver with Python. As Andrew points out, Python has become a popular language for test automation. If you don’t know Python at all, he points you to Jess Ingrassellino’s great course, Python for Test Programming, also on Test Automation University.

In the first chapter, Andrew has you write your first test. Not in Python, but Gherkin. If you have never used Gherkin syntax, it helps you structure your tests in pseudocode that you can translate into any language of your choice. Andrew points out that it’s important to write your test steps before you write test code — and Gherkin makes this process straightforward.

The second chapter goes through setting up a pytest, the test framework Andrew uses. He assumes you already have Python 3.8 installed. Depending on your machine, you may need to do some work (Macs come with Python 2.7.16 installed, which is old and won’t work. Andrew also goes through the pip package manager to install pipenv. He gives you a GitHub link to his test code for the project. And, finally, he creates a test using the Gherkin codes as comments to show you how a test runs in pytest.

In the third chapter, you set up Selenium Webdriver to work with specific browsers, then create your test fixture in the pytest. Andrew reminds you to download the appropriate browser driver for the browser you want to test — for example, chromedriver to drive Chrome and geckodriver to drive Firefox. Once you use pipenv to install Selenium, you begin your test fixture. One thing to remember is to call an explicit quit for your webdriver after a test.

Chapter 4 goes through page objects, and how you abstract page object details to simplify your test structure. Chapter 5 goes through element locator structures and how to use these in Python. And, in Chapter 6, Andrew goes through some common webdriver calls and how to use them in your tests. These first six chapters cover the basics of testing with Python and Selenium.

Now that you have the basics down, the final three chapters review some advanced ideas: testing with multiple browsers, handling race conditions, and running your tests in parallel. This course gives you specific skills around Python and Selenium on top of what you can get from the Python for Test Programming course.

#tutorial #performance #testing #automation #test automation #automated testing #visual testing #visual testing best practices #testing tutorial

Measure Your Test Automation Maturity

I’m a Developer Advocate and one of the things I love most about my role is that I travel all of over the world — meeting and consulting with engineering teams, and discussing the challenges that they face.

One thing that I’ve realized about building quality software is…the struggle is real!

Everyone is trying to figure out how to rapidly-produce software that is yet of high quality. So we did some research (shout out to Moshe Milman who helped with this effort) and gathered best practices from some of the top companies in software, financial services, healthcare, gaming, and entertainment verticals and I’ll share with you what these innovative development teams are doing to achieve great levels of success with their test automation initiatives.

As I go through the points of research, feel free to grade your team’s maturity in that respective area. By the end of the article, you’ll have your Test Automation Maturity Level.

Automated Tests

100% of companies researched automate their tests

For starters, 100% of the companies we researched employ automated tests to expedite their release cycles. When the goal is to release software on a continuous cycle, test automation is a must-have. There simply isn’t enough time to manually test every new feature as well as manually execute regression tests to make sure existing functionality isn’t broken. So these teams invest an extensive amount of effort into automating their tests so that they are confident in their product each time they deploy.

Percentages showing who on the team is responsible for writing tests

I know from personal experience how difficult it is for developers to find the time to write tests and also how difficult it is to have test teams write the code to automate tests, so we inquired about this a bit more to determine how are teams overcoming these challenges.

Every single one of these companies has its developers involved in writing tests. Many of them said their developers take care of the unit tests, while the QA team is responsible for writing the integration and end-to-end tests.

A whopping 60% of the teams shared that they no longer have the distinction between development and QA engineers, and instead have hybrid engineers. Their goal here is to have developers own ALL the testing of their code, as well as the triaging and maintenance of those tests.

What they discovered is what I already know — developers aren’t the best at this. There’s not much time, and frankly not much interest from developers to go beyond writing their unit tests. So, many of these teams have had to bring in qualified experts to help out.

I dug a bit more to learn how exactly the Quality Advocates are assisting here. We got a variety of answers but here were some of the common ones:

• Write test infrastructure
• Coach developers on how to write better tests
• Develop a testing strategy

Let’s discuss each of these…

Write Test Infrastructure

The Quality Advocates find the best testing libraries, create the test automation codebase, and all of the utility functionality the developers will need to write their tests. That way it’s not much overhead for the developers. The developers can just focus on cranking their tests out.

Coach Developers

Unfortunately, many of the Computer Science and Bootcamp programs that graduated your developers did not teach them how to test. This is a huge hurdle for developers who may have good intentions and want to test their code. They may not ever share this with you, but a LOT of the developers that I speak with simply don’t know how to test. These quality advocates specialize in this stuff and can help the developers think of scenarios, as well as teach them how to write good tests. If you think this may be a problem for your developers and you don’t have a quality advocate just yet, send them to Test Automation University which is an online learning platform that provides free courses on this very thing.

Develop Testing Strategies

Finally, quality advocates develop testing strategies for the team. They help them assess risk and come up with a plan of attack on what should be tested and how thoroughly.

They also have a big picture view which is greatly needed because your developers are zoned in on their features and their tests. Someone needs to consider how these features interact with one another so that more sophisticated tests can be developed.

Someone also needs to strategize on which tests automatically run given certain pull requests. The advocate can help with that.

The quality advocates also help keep the test suites relevant by pruning out tests that are no longer of high business value.

Criteria: Does Your Team Automate Any Test?

If your team automates _any _tests at all, go ahead and give yourself 10 points!

Types of Automated Tests

We wanted to make sure that we were talking about more than unit tests here, so we inquired about which tests the companies automated.

• Every one of these companies automated their unit, web, and API tests.
• 80% of the companies who develop mobile apps automated their mobile tests.
• 80% of companies create reusable web design components and automated tests for those.
• However, there was very little effort by the core development teams to automate non-functional tests. Areas like security, performance, and accessibility testing were mostly handled by separate groups, like the Center of Excellences.

Criteria: Which Type of Tests Do You Automate?

Give yourself 10 points for each of the types you automate: unit, web/mobile, API, security, performance, and accessibility. If your team does not develop mobile apps, just give yourself 10 points so that you don’t have a deficit. The same goes for if your company does not develop web apps or APIs.

Programming Languages

I did some research last year on the top programming languages used in test automation. This included all sorts of companies, not just the mature top dogs. The vast majority of teams are using Java (44%) and while JavaScript was on the rise and came in at #2, it was still only 15%.

This was interesting compared to what the top dogs are using. ALL of them use JavaScript! Some have teams that use other languages in addition to JavaScript for things like native mobile testing and APIs, but for their web apps, it’s JavaScript.

Programming languages used for test automation

I inquired why this is and they explained that their web developers are JavaScript programmers. Some companies even said they had legacy test frameworks built in other languages and their devs wouldn’t touch it! When they switched to JavaScript, the developers became more engaged.

I found this interesting because it aligns with what some thought leaders have been preaching for years. I’ll admit, I’ve been a bit stubborn over my career and I tend to go for the language that the automation will be most comfortable in but it seems perhaps I need to rethink that when I want the developers to contribute.

And to be fair, JavaScript automation tools have gotten much better in recent years…which brings me to the next point of research…

#tutorial #performance #testing #programming #automation #test automation #automated testing #qa #automated testing best practices

Review — Automated Visual Testing With WebdriverIO

I took Nyran Moodie’s course on Test Automation University: Automated Visual Testing with WebdriverIO.

If you want the explicit link to the course, here it is https://testautomationu.applitools.com/automated-visual-testing-javascript-webdriverio/index.html.

Course Introduction

If you use WebdriverIO regularly, and you are unfamiliar with the basics of using Applitools for automated visual testing, you will appreciate this course. Nyran focuses you on how to add visual test automation with Applitools to your WebdriverIO tests.

Nyran expects you to know WebdriverIO. If you want to learn WebdriverIO, check out Julia Pottinger’s excellent course — UI Automation with WebdriverIO on Test Automation University.

Nyran uses JavaScript for his examples and Visual Studio Code as his IDE. In case you didn’t know, you can use Applitools with a range of test languages. No matter what your test language of choice or IDE, I think you will find Nyran’s code and coding approach fairly intuitive.

Course Structure

Nyran breaks the course into eight main chapters. He gives this description in the course overview:

• 1 — I am going to start by giving you an introduction to visual testing.
• 2 — We are going to look at how we can get our environment set up to start using Applitools eyes.
• 3 — We are going to create and run our first visual test using Applitools
• 4 — I want to introduce you to the different match levels that Applitools has and the concept of viewport sizes
• 5 — I will be talking about checkpoints.
• 6 — I will be looking at how we can organize our visual tests using batches
• 7 — We will look at how we can analyze our test results using the Test Manager.
• 8 — We will be looking at the integration and collaborations that Applitools provides.

Each of these chapters provides a methodical approach to getting going with Applitools. I’ll give a quick overview of each.

If you read this review or take the course, you know why visual testing. Your UI and end-to-end tests result in the rendered output. You can write all the functional tests that grab locators, enter data, effect action, and cause the appropriate output locators to have the appropriate values. But, until you look at the result, you cannot tell if the input and output conform to design and usability expectations.

Nyran did not explain the most frequently-experienced reason for visual testing — unintended consequences of code changes over time. Our experience shows us that most expected application changes get tested, but unintended changes cause problems.

Nyran does a nice job explaining how to get started. You need an Applitools API key, which you can get from the Applitools console. Nyran explains why you set up a local environment variable for your API key (so you do not need to include your API key in your test code directly). He also points to the GitHub repo he uses for all the examples in the course.

Chapter 3 involves the first coding examples for setting up Applitools. With a simple:

npm install @applitools/eyes.webdriverio

You get the the node instructions for installing the Applitools Eyes to your WebdriverIO setup. After this, you can install the Applitools Eyes service to your tests. He shows code examples of what test code looks like when calling Applitools:

#performance #testing #automation #automated testing #webdriver #visual testing #automated testing best practices #webdriver io #ui automation tools