InnerWeb
Skip to main content

Case study: 13 defects found by E2E tests, fixed in 2 days

Scenarios from manual testers' recorded sessions from summer 2026 were rewritten as automated tests. Porting them exposed 13 defects in the application, and seven of them were fixed within two days. This case shows how the deployment team turned repetitive tester work into a package that guards the work safety system after every change.

13
defects found while porting scenarios
Source: product documentation, E2E browser tests
7
defects fixed in 2 days
Source: product documentation, E2E browser tests
242
tests in the E2E package
Source: product documentation, E2E browser tests

Starting point: manual tests from recorded sessions

From June to September 2026 testers went through the system step by step, and their sessions were recorded. Such recordings are valuable because they show the real way people work with the application: the order of clicks, the places where a tester hesitates, and errors that appear only after a specific sequence of actions. They have one drawback, though: they cannot be replayed on demand after every code change.

The deployment team therefore asked a practical question: how much of this knowledge can be turned into scenarios that run by themselves? The answer was a package of browser tests with 242 tests in 31 files. The package starts a real browser against a real backend, not a mock, so it follows the same path as the user.

How the scenarios were ported

Porting means rewriting each recorded scenario as an automated test: logging in, opening the admin panel, planning work, work permits, reports, documentation, maps and analytics. The scope matches what a person does at the plant, not what is convenient to check from the code.

An important technical detail: the test database is rebuilt from migrations once per test run. As a result the outcome does not depend on data left by a previous run, and a test that passed yesterday does not pass or fail today only because someone changed a record.

The hard failure rule

A test fails automatically when the browser shows an uncaught JavaScript exception or the server responds with a 5xx status. The tester does not have to notice it on the screen: an error that a user could skip by refreshing the page still stops the run.

Defects from porting scenarios: status after 2 days

Defects from porting scenarios: status after 2 daysBar chart: 13 defects found, 7 fixed in two days, 6 known with a ticket number.051015defectsFound: 13 defects13FoundFixed in 2 days: 7 defects7Fixed in 2 daysKnown, with ticket number: 6 defects6Known, with ticket number
Source: Product documentation: E2E browser tests; number of known defects = 13 minus 7

What was found: 13 defects

Porting the scenarios revealed 13 defects in the application. The most valuable part of this number is when they were found: the defects came up while the tests were being prepared, before anyone saw them on the plant floor or in the office. For a system that handles work permits and alarms, the difference between catching a defect at the vendor and catching it at the operator is significant.

The defects did not look like headline failures. They were inconsistencies at the boundary between modules, errors visible only after a particular sequence of steps, and behaviour that unit tests do not catch because they run against a mocked API. This is the typical profile of defects found by browser tests: not a bug in a single function, but a problem in the flow.

  • 13 defects found while porting scenarios
  • 7 of them fixed within two days
  • the remaining 6 marked in the tests as known, with a ticket number
  • every fixed defect left a regression test behind

Two-day fixes and turning a defect into a regression test

Seven of the 13 defects were fixed within two days. The speed came from the fact that each defect arrived with a reproducible scenario: there was no need to ask how to trigger it. The developer received a test that failed, and the task came down to making it pass.

After the fix the test does not disappear. It becomes an ordinary regression test that checks at every later run that the same error has not come back. In this way the list of fixed defects turns into lasting protection rather than an entry in the ticket history.

Six known bugs, not hidden

Defects that were not fixed right away were not skipped. A known, still unfixed bug is marked in the test as an expected failure with a ticket number, never as a skipped test. The difference is essential: a skipped test stays silent, while a test with an expected failure stays in the package and keeps watching the area.

After a fix the test itself reports that the annotation has to be removed. Nobody has to remember to clean up, and the list of known bugs in the package always matches reality. For an auditor or the plant's IT manager, this is clear evidence that the vendor keeps a register of known defects and does not make them disappear by quietly switching tests off.

Conclusions for a plant that accepts the system

First conclusion: recorded manual sessions are good raw material, but only converting them into automated scenarios brings repeatability. Second: a package that runs in a real browser against a real backend catches user-visible errors that unit tests will not detect. Third: an explicit register of known bugs is safer than an empty list.

The full set takes about 2 to 2.5 hours, so it does not block the daily merging of changes. It runs on request and automatically at midnight on weekends. For the client this means that before an update there is a fresh result from the last run, which can be requested for review.

Checklist

Video

Frequently asked questions

Do E2E tests replace unit tests?

No. Unit tests check single functions quickly and cheaply. Browser tests check the flow the user sees and catch errors that a test on a mocked API will not see. The two kinds complement each other.

Why not skip known bugs in tests?

A skipped test stops guarding the area and nobody knows the bug still exists. An expected failure with a ticket number stays in the package, and after a fix the test itself reports that the annotation has to be removed.

How often is the full test set run?

The full set takes about 2 to 2.5 hours. It runs on request and automatically at midnight on weekends, without blocking the merging of changes.

See how we test changes before deployment

Let's talk about which regression tests and which register of known bugs you want to see from the vendor of your safety system.

Let's talk