One popular way of grouping test cases in terms of test granularity is the testing pyramid model.

But when trying to mapping other testing categories into this model, things get a bit messy.

Some QA engineers consider UI automation at the E2E level, API testing at the Integration level.

Others would argue the UI testing sitting at both E2E and Integration levels, and API testing is Integration and Unit levels. When they talk to each other, sometimes they actually refer to different things by mixing these concepts up.

Who is right?

If we go back to the definition, things are simple and clear. All of them can be correct answers. It depends on the actual architecture of your system and the piece your team is working on.

Unit tests are test cases working on the method level, integration tests are performing on the service level, end-to-end tests are checking the workflow from beginning to end.

For a full stack team, UI automation usually is the end-to-end testing, since it usually involves a set of planned actions as a workflow, and requests travel from browser to web services, other back-end middle-wares, back-end services and databases; API testing plays the integration testing roles, since it is the de facto way front-end calling the back-end, and how some back-end services talk to each other.

For an organization that split teams into front-end and back-end, or some teams only provide API services as the client-facing application, the API testing now becomes the end-to-end testing. If the team is processing data transformation jobs or managing data pipelines, data quality check test cases are the end-to-end testing for this team. Or QA from different teams can get together to define and write end-to-end test suites that cover applications or services from different teams.

On the unit tests level, we also have UI tests and API tests, but these test cases use mock techniques instead of actually talking to browsers , making the HTTP calls, or connecting real databases. If a test case has methods talking to another component outside the current code base, then it is a non-unit test, meaning it is an integration test or end-to-end test.

Below is an illustration of different components that consists of a unit test framework vs components that consists of an end-to-end /integration test automation framework.

Usually in a team, the developers handle unit tests, and QAs deal with the rest, writing code for test automation frameworks and automation scripts, performing manual tests, etc. But again, it depends on the team size and complexity of the product. For example, in a small size team, one QA engineer touches all kinds of testing or no QA role at all, the developers or product owners do all testing. In a middle size team or a large company, there are different types of QA with different titles, e.g. Software Development Engineer in test (SET), Test Engineer(TE), or Software Quality Engineer, Quality Assurance Analyst, which focus on different types of tests.

In the book how Google tests software, the way Google groups the tests is based on size. Google uses the language of small,medium,and large tests, emphasizing scope over form. Size is a more defined concept to measure the granularity, but this way, it loses the ability to link the test cases back to the physical testing objects intuitively.

Obviously, there is no one perfect model that fits all conditions. Out of many ways to categorize test cases, Testing pyramid is a nice and simple model for illustrating test strategies as long as the team maps its owned test cases into these three layers clearly.

#software-engineering #automation #software-testing #software-development