Riley Lambert

Riley Lambert

1564823394

Cypress.io: The Selenium killer

Originally published by Paul Cowan at https://blog.logrocket.com

Introduction

Before I start, I want to emphasize that this post is not about one particular project or any automation testers that I have worked with. I have seen this behavior in three recent projects, and nearly every automation tester that I have worked with has busted a gut to make this faulty machine work.

I am fairly sure that a memo has gone out to every contract that I have worked on recently stipulating that a million automation tests are required to guarantee success. We must not stop to question the worth of these tests. We must protect them like our children.

These tests must be written in selenium despite nearly everyone having a pretty grim experience due to the inherent known issues that I will state later.

Selenium tests are insanely challenging to write, but we won’t let that hold us back, and instead, we will get our testers who have maybe come into programming late or are new to development, and we will get these less experienced developers to write these difficult tests.

Selenium tests might be difficult to write, but they are straightforward to copy and paste, which will, of course, lead to all sorts of problems.

We often hear, “If it moves, write a selenium test for it”. Automation tests must be written for the API, the frontend, the backend, the middle-end, the happy path, the sad path, the upside-down path, etc.

We won’t have any time for manual testing, how could we? We have all these flakey selenium tests to write and maintain. We are already late for this sprint, and every story must have an automation test.

After a year or so and an insanely long build, we will decide that this was a bit silly and delete them all or, worse, start again.

Why does everyone still use Selenium despite the inherent problems?

I think I would be closer to understanding the true nature of our existence if I could answer the above question but joking aside, why is the use of selenium so widespread? It does stagger me, but here are a few suggestions:

To be fair, the sudden surge of writing a million acceptance tests is not selenium’s fault. For my money, the correct number of automation tests is one happy path test, no sad paths or upside-down paths. This one test is a smoke test to ensure that our system is open for business.

Unit tests and integration tests are cheaper to run, implement and maintain and should be the bulk of our tests, has everyone forgotten about the test pyramid?

Selenium is not fit for purpose and here is why

The problems with selenium can be expressed in one word, timing.

Before we can even start writing code to assert that our test is correct, we need to ensure that whatever elements we need to interact with are visible and are in a state to accept simulated input. Remote APIs calls will need to have resolved, animations and spinners need to have concluded. The dynamic content that now makes up the majority of our apps will need to have finished rendering from the currently retrieved data of the API calls.

So what do we do while this macabre pantomime of asynchronicity is occurring? How do we stop our tests from just finishing or bottoming out because a particular text input is disabled until an API call has finished or a beautiful SVG spinner overlay has put a veil of darkness over our virtual world?

In layman’s terms, we wait for the HTML elements to be in a ready state, in selenium speak, we write many custom waitForXXXXX code helpers, e.g.

waitForTheFullMoonAndTheWereWolvesHaveFinishedEating or more realistically…

wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//input[@id='text3']")));

One of the worst crimes to commit is to use Thread.sleep. This is a heinous crime where a random number is plucked from thin air and used as a wild guess for when we think the UI is in a ready state. Please, never do this.

Below are my all-time favorite selenium exceptions that I have found while wading through a CI build report:

  • NoSuchElementException – move along, you’ll not find your input here
  • ElementNotVisibleException – this cheeky scamp means you are tantalizingly close but not close enough, it is in the DOM, but you can’t do a single thing with it
  • StaleElementReferenceException – the element has finished work for the day and gone to the pub. Please try again tomorrow
  • TimeoutException – you could wait until the end of time and whatever you are trying to do is just not going to happen. You just rolled a seven

Behold the flake

One of the most soul-destroying moments that I have experienced is having a build fail due to a failing automation test only for it to magically pass by just rerunning the build again. This phenomenon or zombie automation test is often referred to as a flake. The main problem with the flake is that it is non-deterministic which means that a test can exhibit different behavior when executed with the same inputs at different times. You can watch the confidence in your regression test suite go up in smoke as the number of non-deterministic tests rises.

A flakey test is more than likely down to timing, latency and the macabre opera of asynchronicity that we are trying to tame with our Thread.sleep and waitForAHero helpers that we need to keep writing to try and keep sane.

Just think how much easier this would be if we could somehow make all this asynchronous programming go away and if our world started to behave linearly or synchronously. What a natural world to test we would have.

Cypress.io sets out to do just that.

Cypress.io – The ghost in the machine

One of the main differences between cypress.io and selenium is that selenium executes in a process outside of the browser or device we are testing. Cypress executes in the browser and in the same run loop as the device under test.

Cypress executes the vast majority of its commands inside the browser, so there is no network lag. Commands run and drive your application as fast as its capable of rendering. To deal with modern JavaScript frameworks with complex UI’s, you use assertions to tell Cypress what the desired state of your application is.

Cypress will automatically wait for your application to reach this state before moving on. You are completely insulated from fussing with manual waits or retries. Cypress automatically waits for elements to exist and will never yield you stale elements that have been detached from the DOM.

This is the main take away, and cypress has eliminated the main problem with selenium by executing in the same run loop as the device. Cypress takes care of waiting for DOM elements to appear. I repeat, Cypress takes care of all this waiting business. No Thread.sleep, no waitForTheMoon helper. Don’t you see what this means?

To know how good this is, you have to have experienced the pain.

Below are a few examples of cypress tests. One thing synonymous by their absence is any timing or obscene waitFor helpers:

context("Login", () => {
  beforeEach(() => {
    cy.visit("localhost:8080/login");
  });
 
  it("can find and type in email", () => {
    cy.get("#email")
      .type("fake@email.com")
      .should("have.value", "fake@email.com");
  });
 
  it("can find and type in password", () => {
    cy.get("#password")
      .type("fakepassword")
      .should("have.value", "fakepassword");
  });
 
  it("will fail when type invalid user credentials", () => {
    cy.get("#email").type("fake@email.com");
 
    cy.get("#password").type("fakepassword");
 
    cy.get("input[type=submit]").click();
 
    cy.get("#login-message").should("have.text", "Login failed");
  });
});

I like these tests; they clearly state their purpose and are not obfuscated by code that makes up for the limitations of the platform.

Below are some tests I wrote to run the axe accessibility tool through cypress:

import { AxeConfig } from "../support/axeConfig";
 
describe("Axe violations", () => {
  beforeEach(() => {
    cy.visit("/");
    cy.injectAxe();
  });
 
  it("home page should have no axe violations", () => {
    cy.configureAxe(AxeConfig);
    cy.checkA11yAndReportViolations();
  });
});

And here is a similar test using webdriver:

// in e2e/home.test.js
import assert from 'assert';
import { By, until } from 'selenium-webdriver';
import {
    getDriver,
    analyzeAccessibility,
} from './helpers';
 
describe('Home page', () => {
    let driver;
 
    before(() => {
        driver = getDriver();
    });
 
    it('has no accessibility issues', async () => {
        await driver.get(`http://localhost:3000`);
 
        // The dreaded wait until.  Abandon hope
        await driver.wait(until.elementLocated(By.css('h1')));
 
        const results = await analyzeAccessibility();
        assert.equal(results.violations.length, 0);
    });
});

The main striking difference and the worrying thing to me is the latency, and there are two await calls and the dreaded wait(until.elementLocated). This is a simple test, but the more interactions you have, the more waitFor helpers you will need, and the flakiness starts spreading.

JavaScript all the way down

Cypress is clearly aimed at the frontend developer, installing cypress is a breeze and performed via your favorite package manager choice of npm or yarn.

npm install cypress --save-dev

It really could not be any easier. Compare that with downloading the chrome webdriver and friends in the world of selenium.

There is no multi-language support like selenium. You can have any programming language you like as long as it is JavaScript or TypeScript.

Cypress cons

Of course, there are drawbacks, and some of them are notable so it would be remiss of me not to list these.

  • Cypress is relatively new, and it does not have the vast community that selenium does
  • No cross-browser testing, this is huge and will cause less adoption until it can be cured
  • As stated earlier, it’s JavaScript or bust. You won’t write cypress tests in the tired old static languages of C# and java
  • Because it runs inside the browser, you won’t be able to support multiple tabs
  • There is no cross-browser support other than Chrome and Electron
  • At this time of writing, there is no shadow DOM support

The above items are, in some cases, insurmountable and will not be overcome.

Will Cypress replace Selenium?

As much as I would like to say yes, I have my doubts. There is an army of automation testers who have not known any other world than selenium, and it may be difficult to move away from soon.

Conclusion

As I stated at the start of this article, my experience of automation testing is not a good one where a lot of money, time and pain are spent keeping thousands of hard to maintain tests afloat for not great payout. Automation testing has only ever guaranteed a long CI build in my experience.

We, as developers, need to be better at automation testing. We need to write fewer tests that do more and are useful. We’ve left some of the most difficult code to write to some of the least experienced developers. We’ve made manual testing seem outdated when, for my money, this is still where the real bugs are found.

We need to be sensible about what automation testing can achieve.

Cypress is great because it makes thing synchronous, this eliminates a whole world of pain, and for this, I am firmly on board. This, however, is not the green light to write thousands of cypress tests. The bulk of our tests are unit tests with a layer of integration tests before we get to a few happy path automation tests.

This, of course, is far too sensible a strategy ever to happen.

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow us on Facebook | Twitter


#selenium #java #testing #web-development

What is GEEK

Buddha Community

Cypress.io: The Selenium killer
Ilene  Jerde

Ilene Jerde

1596801420

Selenium Tool Suite - Components and Features

Selenium is an automation testing tool; it is primarily used to test websites and web applications; it is an open-source tool. With the help of Selenium, test cases can run directly in web browsers, just like a person operating the web browsers. It supports many web browsers such as Opera, Safari, Chrome, Firefox, IE, etc. There are several different sub tools to support different automation test approaches. In this article, we will learn about selenium tool suite, its components and features. So let’s start!!!

#selenium tutorials #selenium grid #selenium ide #selenium rc #selenium tool suite #selenium webdriver

iOS App Development Company in Singapore

iOS app development in Singapore

iOS has become the first priority for most smartphone users because of the security it offers compares to the Android operating system. Due to this reason, it is suggested to launch an app in iOS before other platforms.

Want to develop an iOS app in Singapore?

WebClues Infotech with its worldwide reach has already offered its iOS app development services to customers in Singapore. With a highly-skilled development team of 120+ members, WebClues Infotech has got the required resources an agency needs to fulfil client requirements around the world.

Want to know more about our iOS app development services in Singapore?

Visit: https://www.webcluesinfotech.com/iphone-app-development/

Share your requirements https://www.webcluesinfotech.com/contact-us/

View Portfolio https://www.webcluesinfotech.com/portfolio/

#ios app development in singapore #ios app development company #ios app development #ios #ios app #hire ios developer

iOS App Development in the United Arab Emirates

iOS App Development in the United Arab Emirates

Developed and Developing nations have seen a rapid rise in the demand for iOS app development and the United Arab Emirates is no exception. The use of on-demand apps has increased drastically in the last decade and business is leveraging this demand with launching iOS mobile apps.

Want to develop the iOS app in the United Arab Emirates?

WebClues Infotech after serving multiple clients in UAE has become well aware of the people’s needs in the region. With a highly experienced development team that has completed more than 950+ projects, we are prepared to serve you with your iOS app development needs.

Want to know more about our iOS App Development Services in UAE?

Visit: https://www.webcluesinfotech.com/iphone-app-development/

Share your requirements https://www.webcluesinfotech.com/contact-us/

View Portfolio https://www.webcluesinfotech.com/portfolio/

#ios app development in the united arab emirates #ios app development #ios app #ios #ios app development company #hire ios developer

iOS App Development Company in India

iOS app development company in India

India is considered the IT hub of the world because of n number of IT infrastructure development services offering companies. In this whole market iOS app development is the leading service offered by agencies across India

Want to develop the iOS app in India

WebClues Infotech with its head office in India has created a huge presence across the world over time and has served clients in all of the major countries around the world. WebClues Infotech with a highly skilled development team of 120+ members can help you deliver a better result at a reasonable cost.

Want to know more about our iOS app development services in India?

Visit: https://www.webcluesinfotech.com/iphone-app-development/

Share your requirements https://www.webcluesinfotech.com/contact-us/

View Portfolio https://www.webcluesinfotech.com/portfolio/

#ios app development company in india #ios app development company #ios app development #ios app #ios #hire ios app developer

iOS App Development Agency in the USA

iOS App Development Agency in the USA

Whenever a start-up in the USA launch an MVP version of the app they prefer to launch it only for iPhone user because the US has a large market of iPhone users in comparison to Android. The recent phenomenon of Clubhouse is the biggest example.

Want to develop an iOS app in the USA?

With 2 office locations across the USA and 6 worldwide, WebClues Infotech has the experience of serving a huge client base of 600+. After such a satisfied client base, WebClues Infotech is prepared to serve you with and iOS App Development Services in the USA.

Want to know more about our iOS App Development Services in the USA?

Visit: https://www.webcluesinfotech.com/iphone-app-development/

Share your requirements https://www.webcluesinfotech.com/contact-us/

View Portfolio https://www.webcluesinfotech.com/portfolio/

#ios app development agency in the usa #ios app development agency #ios app development #ios app #ios #hire ios app developer