Kyle  M. Farish

Kyle M. Farish

1564215018

Performance Testing: How to Load Test

Objective

The purpose of this article is to help you design and run load test 👍

Why Learn?

As developers we already have the foundational knowledge and with a little effort we could expand our skillset.

  • Your company cannot afford to hire performance engineers
  • Not enough testers compared to developers
  • The skill & knowledge could help you write better and scalable code
  • Less dependant on other’s expertise

Why Load Testing?

While unit & integration tests ensure code is functionally correct, load testing measures its performance which is equally important.

Only a Load Test can shed light into concurrency issues, whether the database queries make good use of indexes instead of a full table scans, where are the bottlenecks, does the application scale efficiently, what is the application’s response time and throughput and so on.

Getting Started Apache JMeter

In this section we will design and run Apache JMeter load tests.

Environment Setup

For environment, either find a suitable online resource (not recommended), come up with your own simple service (node, python, whatever) or simply use the web service provided in this article.

We will be using a simple java-based spring boot web service that exposes four (4) endpoints. The requirement are Java 1.8 and Apache Maven.

git clone https://github.com/rhamedy/tinyservice4loadtest.git
cd tinyservice4loadtest
mvn spring-boot:run

Apache JMeter

Please go ahead and install Apache JMeter from download site, unzip and execute the following command

apache-jmeter/bin/jmeter.sh       // Linux & MacOS
apache-jmeter/bin/jmeter.bat      // Windows

Design a Test Plan for Apache JMeter

Let’s load test the following apis

http://localhost:8080/students/list  - [GET] List students
http://localhost:8080/students/id    - [GET] Get student by id
http://localhost:8080/students       - [POST] Create student
http://localhost:8080/students/id    - [DELETE] Delete student by id

All the sample tests for above endpoints are available in the GitHub repository.

Step 1

Right click on test plan and pick Add > Threads (Users) > Thread Group. A test plan must have at least 1 Thread Group.

  1. The Number of Threads (users)
  2. The Ramp-Up period (in seconds). How long to reach the max users?
  3. How many times or for how long to run the test

Step 2

Let’s specify what to test. Right Click on ThreadGroup and select Add > Config Elements > HTTP Request Defaults option.

Config Elements are useful if you wish to share configuration among one or more requests for example i.e. server address, port number, token, etc.

Let’s fill out the HTTP Request Defaults Config element

Also let’s add an HTTP Header Manager via Add > Config Elements > HTTP Header Manager for Content-Type header

Step 3

Let’s configure the http requests, Right Click on the ThreadGroup and select Add > Sampler > HTTP Request

  1. What is the method type i.e.GET or POST etc
  2. What is the api path i.e. /students/list or /students etc

Right click to duplicate a request and update it.

Step 4

The Listeners are used to collect results. Right Click on ThreadGroup and select Add > Listener > Summary Report option.

The test plans in words, we create an Apache JMeter Test Plan to Load Testtwo apis with 50 Users with a ramp-up for a duration of 30 seconds.

If you have the tinyservice4loadtest (or your own) running, then let’s hit the play button and see the results in Summary Report

Run Apache JMeter Test Using CLI

The GUI is not recommended for running complex tests. Let’s open a complex sample test from here.

The above test plan has more elements

  • The Random Variable generates a value between x and y
  • The Loop Controller executes the content of loop x times

To generate a meaningful report we need user.properties (Source) in the directory where .jmx test is.

jmeter.reportgenerator.report_title=Apache JMeter Dashboard
jmeter.reportgenerator.overall_granularity=60000
jmeter.reportgenerator.graph.responseTimeDistribution.property.set_granularity=100
jmeter.reportgenerator.apdex_satisfied_threshold=1500
jmeter.reportgenerator.apdex_tolerated_threshold=3000
jmeter.reportgenerator.exporter.html.property.output_dir=/tmp/test-jmeter.reportgenerator.exporter.html.filters_only_sample_series=true

Run the test script using command (output-directory should be empty)

jmeter.sh -n -t loadtest.jmx -l log.jtl -e -o output-directory

In the above command the -n stands for no gui the -t indicates the scripts loadtest.jmx the -l is for log.jtl where -e and -o are for reporting.

The output-directory will contain a bunch of files including an index.htmlthat opens the graphical results of the test as shown below.

In this graph the left side is APDEX and the right side is Request summary. The red indicates all our 404 errors and the green is 200 successful requests.

Some numbers relating to Number of samples, Response time, Throughput

Most importantly the Response Time and Throughput

Lastly, it’s worth-mentioning that Apache JMeter can be configured to listen to a browser’s activity and capture the network request.

Getting Started with Taurus

Now that we know how to use Apache JMeter to run a basic load test, let’s explore open-source framework Taurus. In short, one of the reasons for birth of Taurus was because Apache JMeter has a steep learning curve and Taurusmake things a lot simpler.

Taurus is an abstraction layer (or a wrapper) on top of Apache JMeter and that means you can run an Apache JMeter script using Taurus. So go ahead and install Taurus using the easy to install instruction

The Taurus scripts can be written in YAML or JSON using follow blocks

Scenarios is basically where one or more requests are defined. For each scenario an execution is defined with props such as no of usersdurationramp-up period and so on. The modules allow us to configure executor that could be Apache JMeterSelenium and so on. Likewise, the reporting allows configuring how the report should be generated i.e. csv, live reporting in console, or push the result to the blazemeter website.

scenarios: 
 StudentList: 
  requests: 
   - url: http://localhost:8080/students/list
     label: Student List
execution: 
 - scenario: StudentList 
   concurrency: 15
   ramp-up: 2s
   hold-for: 10s
reporting:
 - module: console 
 - module: final-stats 
   summary: true 
   percentiles: true
   test-duration: true 
   dump-csv: single_scenario_single_req.csv

Load test /students/list api reaching 15 users within 2s (ramp-up) for a duration of 10s and display live result in the console as well as csv file.

To run the Taurus test, simply run the command bzt test.yaml

In a Taurus test you can also configure a scenario to point to an Apache JMeter script and override execution and other parameters.

scenarios:
JMeterExample:
script: student_crud_jmeter.jmx

Taurus seems to be a very interesting framework and it is worth checking it out. It is very well-documented here.

Conclusion

I highly recommend checking out Apache JMeter and Taurus documentations if you wish you learn more techniques and tricks.

Thanks, Keep Visiting. If you liked this post, share it with all of your programming buddies!

Modern Web Apps Performance Tricks with PWA and Vue

The performance benchmarks of Redis and MySQL

Optimize the Performance of a Vue App with Async Components

#java #testing #web-development

What is GEEK

Buddha Community

Performance Testing: How to Load Test
Rahul  Hickle

Rahul Hickle

1598457456

Is API Load Testing Right for Your App? - DZone Integration

API load testing has been around for decades. There are lots of robust tools you can choose from, both commercial and open-source, and many of these tools have large communities and extensive documentation around how to script the most common causes. It’s a far cry from the browser-level testing space, which is relatively new and sparsely populated by comparison.
API load testing is one of the most cost-efficient ways you can get started with load testing, allowing you to scale up your load relatively cheaply while getting immediate results.
How to Get Started With API Load Testing
API load testing isn’t for every application, but depending on your test scenario, it may be the easiest way to test applic.

#integration #performance testing #load testing #jmeter #puppeteer #api load testing #load testing difficulties

Dejah  Reinger

Dejah Reinger

1599859380

How to Do API Testing?

Nowadays API testing is an integral part of testing. There are a lot of tools like postman, insomnia, etc. There are many articles that ask what is API, What is API testing, but the problem is How to do API testing? What I need to validate.

Note: In this article, I am going to use postman assertions for all the examples since it is the most popular tool. But this article is not intended only for the postman tool.

Let’s directly jump to the topic.

Let’s consider you have an API endpoint example http://dzone.com/getuserDetails/{{username}} when you send the get request to that URL it returns the JSON response.

My API endpoint is http://dzone.com/getuserDetails/{{username}}

The response is in JSON format like below

JSON

{
  "jobTitle": "string",
  "userid": "string",
  "phoneNumber": "string",
  "password": "string",
  "email": "user@example.com",
  "firstName": "string",
  "lastName": "string",
  "userName": "string",
  "country": "string",
  "region": "string",
  "city": "string",
  "department": "string",
  "userType": 0
}

In the JSON we can see there are properties and associated values.

Now, For example, if we need details of the user with the username ‘ganeshhegde’ we need to send a **GET **request to **http://dzone.com/getuserDetails/ganeshhegde **

dzone.com

Now there are two scenarios.

1. Valid Usecase: User is available in the database and it returns user details with status code 200

2. Invalid Usecase: User is Unavailable/Invalid user in this case it returns status with code 404 with not found message.

#tutorial #performance #api #test automation #api testing #testing and qa #application programming interface #testing as a service #testing tutorial #api test

Mikel  Okuneva

Mikel Okuneva

1596793726

Where To Learn Test Programming — July 2020 Edition

What do you do when you have lots of free time on your hands? Why not learn test programming strategies and approaches?

When you’re looking for places to learn test programming, Test Automation University has you covered. From API testing through visual validation, you can hone your skills and learn new approaches on TAU.

We introduced five new TAU courses from April through June, and each of them can help you expand your knowledge, learn a new approach, and improve your craft as a test automation engineer. They are:

These courses add to the other three courses we introduced in January through March 2020:

  • IntelliJ for Test Automation Engineers (3 hrs 41 min)
  • Cucumber with JavaScript (1 hr 22 min)
  • Python Programming (2 hrs)

Each of these courses can give you a new set of skills.

Let’s look at each in a little detail.

Mobile Automation With Appium in JavaScript

Orane Findley teaches Mobile Automation with Appium in JavaScript. Orane walks through all the basics of Appium, starting with what it is and where it runs.

javascript

“Appium is an open-source tool for automating native, web, and hybrid applications on different platforms.”

In the introduction, Orane describes the course parts:

  • Setup and Dependencies — installing Appium and setting up your first project
  • Working with elements by finding them, sending values, clicking, and submitting
  • Creating sessions, changing screen orientations, and taking screenshots
  • Timing, including TimeOuts and Implicit Waits
  • Collecting attributes and data from an element
  • Selecting and using element states
  • Reviewing everything to make it all make sense

The first chapter, broken into five parts, gets your system ready for the rest of the course. You’ll download and install a Java Developer Kit, a stable version of Node.js, Android Studio and Emulator (for a mobile device emulator), Visual Studio Code for an IDE, Appium Server, and a sample Appium Android Package Kit. If you get into trouble, you can use the Test Automation University Slack channel to get help from Orane. Each subchapter contains the links to get to the proper software. Finally, Orane has you customize your configuration for the course project.

Chapter 2 deals with elements and screen interactions for your app. You can find elements on the page, interact with those elements, and scroll the page to make other elements visible. Orane breaks the chapter into three distinct subchapters so you can become competent with each part of finding, scrolling, and interacting with the app. The quiz comes at the end of the third subchapter.

The remaining chapters each deal with specific bullets listed above: sessions and screen capture, timing, element attributes, and using element states. The final summary chapter ensures you have internalized the key takeaways from the course. Each of these chapters includes its quiz.

When you complete this course successfully, you will have both a certificate of completion and the code infrastructure available on your system to start testing mobile apps using Appium.

Selenium WebDriver With Python

Andrew Knight, who blogs as The Automation Panda, teaches the course on Selenium WebDriver with Python. As Andrew points out, Python has become a popular language for test automation. If you don’t know Python at all, he points you to Jess Ingrassellino’s great course, Python for Test Programming, also on Test Automation University.

Se

In the first chapter, Andrew has you write your first test. Not in Python, but Gherkin. If you have never used Gherkin syntax, it helps you structure your tests in pseudocode that you can translate into any language of your choice. Andrew points out that it’s important to write your test steps before you write test code — and Gherkin makes this process straightforward.

first test case

The second chapter goes through setting up a pytest, the test framework Andrew uses. He assumes you already have Python 3.8 installed. Depending on your machine, you may need to do some work (Macs come with Python 2.7.16 installed, which is old and won’t work. Andrew also goes through the pip package manager to install pipenv. He gives you a GitHub link to his test code for the project. And, finally, he creates a test using the Gherkin codes as comments to show you how a test runs in pytest.

In the third chapter, you set up Selenium Webdriver to work with specific browsers, then create your test fixture in the pytest. Andrew reminds you to download the appropriate browser driver for the browser you want to test — for example, chromedriver to drive Chrome and geckodriver to drive Firefox. Once you use pipenv to install Selenium, you begin your test fixture. One thing to remember is to call an explicit quit for your webdriver after a test.

Chapter 4 goes through page objects, and how you abstract page object details to simplify your test structure. Chapter 5 goes through element locator structures and how to use these in Python. And, in Chapter 6, Andrew goes through some common webdriver calls and how to use them in your tests. These first six chapters cover the basics of testing with Python and Selenium.

Now that you have the basics down, the final three chapters review some advanced ideas: testing with multiple browsers, handling race conditions, and running your tests in parallel. This course gives you specific skills around Python and Selenium on top of what you can get from the Python for Test Programming course.

#tutorial #performance #testing #automation #test automation #automated testing #visual testing #visual testing best practices #testing tutorial

Carmen  Grimes

Carmen Grimes

1595505720

How to Test Mobile App Performance: 3 Key Components - DZone Performance

You’ve probably interacted with an app on your phone or tablet that’s slow, takes a long time to load, freezes, or even crashes on you altogether.

Frustrating, right?

On the flip side, you can probably think of an app that you love to use because from day one, it’s never given you any trouble.

Or maybe you never paid any mind to an app that works quickly, because isn’t that how it’s supposed to be?

So, what causes one app to be crash-prone and another, fast, and reliable?

Whether an app has good or bad performance depends on three factors: the backend, the network, and the app itself running on the device.

A developer or mobile tester can measure the performance of an application in different scenarios.

For example, they can test for when there’s a concurrency of users on the app at the same time, on different devices (which vary in hardware resources and screen sizes), and multiple networks such as 3G, 4G, Wifi, and more.

The reality is that many variables affect the performance of a mobile application. Moreover, a user may have a very bad experience with your app and the cause might not even have anything to do with the code or its implementation.

But, by running performance tests for each of these three factors, you’ll be able to identify problems and optimize your app for the best user experience possible.

Keep reading as we’ll cover the different types of tests for each factor, what to measure, and what tools are available to help you along the way.

1st Mobile Performance Factor: The Backend

A mobile app’s backend architecture is generally based on an application server, a web server, and a database.

**When it comes to the backend, the things related to performance that are important to know when an app is under load are the server’s response times, database queries times, and the server’s resource usage. **

Using this information, it’s easier to detect issues such as:

  • High server response times
  • Bottlenecks or breakpoints in the database and application server resources
  • Poor implementation of escalation policies

So what kind of tests are normally run to check the app’s backend performance? Load tests.

This is when you simulate load on the backend in different ways, whether it be through stress testing, peak testing, endurance testing, load testing, etc.

In general, the objective of these tests is to understand how the backend systems of an app behave and handle a certain volume of concurrent users.

Several tools allow you to load test your mobile app. The most commonly used ones include:

Apache JMeter – the number one open-source load testing tool

Gatling– a developer-friendly, open-source load testing tool with scripts written in Scala

BlazeMeter – a cloud performance testing platform that scales your JMeter or Gatling tests for a greater amount of concurrent users

2nd Mobile Performance Factor: The Network

With regards to the network that the device is connected to, there are two key things to measure: latency and bandwidth.

  • Latency is the time that elapses when information is sent on the network (measured in milliseconds).
  • Bandwidth is the maximum capacity (the amount of data) that can be transmitted through the network (measured in bits per second).

For mobile performance, the lower the latency and the higher the bandwidth, the better.

An app’s performance can vary depending on, for example, whether it’s connected to a 3G network or a 4G network, and unfortunately, this is beyond an app developer or tester’s control.

But, it is possible to incorporate the network during the mobile app performance testing process, simulating the different types of networks and measuring their impact on the response times, both on the server-side and the client-side.

#tutorial #performance #mobile apps #load testing #mobile testing #mobile app performance #client side performance

Top 10 Mobile Performance Testing Tools

Challenge for brands: how to offer a seamless, fast, and user-friendly mobile experience?

App users have a low tolerance for slowness, with a reported 43% of users unhappy if they have to wait longer than three seconds for an app to load. ([App Samurai])

It’s not enough to ensure that your mobile app functions properly, but also to test how it behaves on different devices, under heavy user load, different network connections, etcetera. It’s equally important to test different metrics on both the client-side as well as the server-side. This is where finding the right tool or set of tools for mobile performance testing is essential.

After extensively researching, I’ve put together a list of top-rated mobile performance testing tools and provided an overview of each below.

#testing #load testing tool #testing tools #performance #mobile testing tools