Different approaches to testing: what is their essence and which one to choose for your projects
What are the tests

There are various classifications of testing types: by object, knowledge of the internal structure, degree of automation, degree of isolation, etc.

I will talk about the types of testing according to the degree of isolation, that is, the volume of what we are testing - a separate function or the entire object as a whole. There are three types of testing: unit testing, integration testing, and end-to-end testing.

There is no clear definition of which tests belong to which type. Different testing schools can classify the same test as a unit or integration test. I will talk about my experience and approach to this classification.

testing integration unit

Unit Testing

Unit testing is completely isolated tests that cover classes or individual functions. With utilities and pure functions, everything is simple: we supply test data as input, get the result and compare it with expectations.

But what about components that have dependencies in the form of other components or even stores and an external API? Gotta get wet!

Mock (mock) - object-imitation. It replicates the desired component with the required precision and implements its interface or API. Used only in a test environment. For example, for backend tests, you can lock the repository so that it writes and reads data from RAM, while the real repository works with the database.

Stub (stub) - a stub object with an interface of related components or API, but without logic. For example, an object whose method returns the same result on its call. Or with a predefined set of parameters: with a unit at the input, it returns one object, with a deuce, another.

Usually stubs are used for API stubs. For example, a GET api/user/1 request will return the same user without the need to run a real backend.

An important feature: you can read data from both mocks and stubs. But to write down - only in the mock. In most cases, stubs are preferable because their logic does not change and they are faster to write than mocks.

How to write unit tests

Step 1. We replace with mocks and stubs everything that does not apply to the tested component. This can be done through dependency injection or using tools like jest.mock.

Step 2. We check the default scenario and boundary conditions, both positive and negative cases with an error.

Not worth:

  • test trivial cases, such as getters and setters, which Robert Martin gives as an example in his article;
  • chasing the percentage of code coverage - after reaching 70-80% coverage, it is difficult to write a useful test.

The unit tests the public methods of the class/component. An important clarification: you only need to test the public interface, since the internals can be refactored in the future. In addition, if you get stuck on the internals, it's easy to get bogged down in tautological tests that only say that the code works exactly as it is written.

If there is a desire to test the internal part of the component, then most likely the testing did not go according to plan and the component needs to be decomposed. For example, pull out complex logic into a separate utility and test it separately, and do not pay attention to it in tests of the entire component.

If the component has several complex methods that you want to test, then they definitely need to be taken out, since combinatorics comes into play here. And if one method has 5 possible outcomes and the second one too, then if we leave them in the component, we will have to write 25 tests for them instead of 10. As an example, consider a button that can be set to one of five colors and one of five sizes.

Integration testing

In integration tests, we check the operation of several components or classes in a bundle. Mocks and stubs in it close only interservice interaction, for example, API calls to the backend or between microservices. On the front, it's convenient to mock API requests using MSW.

There are no clear rules here, they come from the task of the component, but there are a few tips. For example, you should not test the logic of external libraries (react and its state, databases) and other things beyond our control. In addition, in these tests, you should not look at the visual display, we are interested in making the component work.

It remains to test what often breaks: business logic, data model and boundary situations. Here, first of all, we focus on user scenarios.

For example, there is an authorization page. We are trying to guess the user's actions: we enter the login and password, press the "Login" button and see that a request with the necessary parameters has come to the backend mock. In the next test, we check if an error occurs if the user entered only the login.

There is a rule: one test - one scenario. If tests have common parts, they are taken out into a separate function and executed in beforeEach, or simply by hand where it is needed. This is in good agreement with the Single Responsibility Principle.

End-to-end (e2e), or end-to-end testing

End-to-end testing emulates user actions in an environment identical to the product. If we are talking about a client-server web application, then to test it, you need to raise a full-fledged front and back and write a bot that repeats the user's behavior. He will visit the site, press buttons, try to log in, etc.

e2e helps to avoid regression testing, automate the work of testers and catch bugs that the lower levels of tests could not catch.

How to choose tests for code

We figured out the types of tests, now we need to correlate them with each other. At the moment, the testing pyramid concept is popular, which gained worldwide fame in 2009 after the publication of Mike Cohen's book "Succeeding with Agile". This concept implies that there should be more unit tests, fewer integration tests, and very little e2e. The author talks about this ratio in detail in the article The Forgotten Layer of the Test Automation Pyramid.

testing pyramid unit integration

In the pyramid, e2e tests are given the least attention. This seems illogical, because such tests emulate user actions and, it would seem, they need as many as possible.

In fact, everything is not so simple. Tests not only look for errors, but also speed up development: they simplify code refactoring, find where the logic has broken, and automatically check the code. But to ensure time savings, tests must be fast.

Emulation in e2e requires tens of seconds per test. And a thousand tests will take 3 hours, which greatly increases the lead time. Especially if an error is found: it will be fixed and the tests will have to be started again.

e2e has one more problem: after the discovery of the error itself, it is not clear where exactly it occurred. For example, authorization does not work. Where is this problem? On the front, on the back, in which module? Localization of the problem will take additional time. More about this is described in the article by Google engineers Just Say No to More End-to-End Tests.

Lower-level tests can narrow the scope and reproduce the error in isolation. They are faster and less bloated, which helps with manual debugging.

But there is also a downside. The lower to the base of the pyramid, the more the tests are divorced from the actual user actions. As a result, sometimes it turns out that all unit tests are successfully passed, but the program still does not work:

testing door secret

What to Consider When Choosing a Test Ratio

Testing everything does not make sense, it is very expensive:

  • each component needs to be wetted;
  • if the code changes, then the tests will have to change;
  • when changing a class, you need to change the mocks of this class;
  • The test code itself must also be checked for errors.

Ultimately, we come to the conclusion that the number of tests depends on the budget and the stage of product development.

Budget. How much money will the time saved on tests bring to the business? And how much will a mistake cost on the sale? If the first is greater than the second, fewer tests can be done. Otherwise, it is better to observe the pyramid.

Product development stage. At the start, when a company writes an MVP, and the concept changes regularly, covering everything with tests is pointless and even harmful. And in a large project, it is impossible to expand the functionality without the risk of breaking the existing one. Therefore, without tests, development will simply stop.

Depending on the priorities, several main strategies can be distinguished:

  • Quality. We use the testing pyramid. We write unit tests for components and integration tests for groups of components, do not forget about e2e. Multi-level exhaustive testing will require a lot of time and resources, but will catch critical defects.
  • Speed. We write e2e for the main user scenarios. In this way, we will detect critical defects in advance, and we will fix the rest if we find a breakdown. And as a result, we quickly deliver functionality.
  • Balance. We cover all the components with tests, but we only mock the API and everything related to dynamically changing parameters. For example, getting the current date. Each component of the higher level is tested based on the fact that the components of the lower level have already been tested and do not require additional verification. This means that they can be safely used, ignoring their internal cyclomatic complexity. Requires fewer resources, but takes a little more time.

It seems logical to follow the path of balance. But the question arises: how to localize a bug if several tests turn red at once? In fact, it’s simple: set everything up so that first the tests of the low-level components are launched and only then the tests of the higher levels.

This whole story contradicts the testing pyramid, because we get more integration tests than unit tests. But that's not bad - it's just a different approach, which is called the testing trophy.

Key rules for writing tests
  • First of all, we check the default scenario and boundary conditions. Both positive and negative cases need to be checked.
  • It is necessary to test only the external behavior of the class / component, without going into its structure. Otherwise, it will be a tautological test.
  • If there is a desire to test the logic inside the component, then it should be moved to a separate utility. This way you prevent the growth of the cyclomatic complexity of testing the entire component.
  • You should not test trivial cases and chase the percentage of code coverage.
  • First of all, you need to test business logic and user use cases. If we talk about the front, then the task of the tests remains the same: first of all, we test the logic, and not the visual display.
  • Tests must be atomic. Do not cram multiple scenarios into one test.
  • To maintain balance, it is best to write unit tests for common utilities and components. And integration - for entire pages, forms, modals and other independent things. At the same time, we consider that all downstream components have already been tested and work.