InnerWeb
Skip to main content

Vendor review checklist: maintenance and development

An annual review of a safety system vendor lets the plant check whether maintenance and development follow the contract. The checklist covers tests, change control, error monitoring, reports and feature development, and the comment on each group explains what proof to ask for.

242
tests in the E2E package
5,000
errors per month in the shared budget
2
change control review rounds

Purpose and preparation of the review

A vendor review should answer one question: does the plant have evidence that the system is maintained and developed in a controlled way. It is worth inviting the IT department, the health and safety manager and purchasing to the review, because each of these roles looks at a different area.

Before the meeting, ask the vendor for three documents: the list of releases with version numbers, the list of reports with response times and the result of the last full test run. The system is offered on a subscription basis with a one-time implementation fee, so the review is also a good occasion to compare the scope of services in the subscription with what is actually used.

Tests and acceptance

The first group concerns how the vendor checks its own software. The browser end-to-end test package includes 242 tests in 31 files and runs a real Chromium browser against a real backend. The plant should ask how many tests the package has now, when the last full run took place and what its result was.

The full suite takes about 2 to 2.5 hours and runs manually on request and automatically at midnight on weekends. Ask for the history of runs from recent months: regularity of runs says more about the vendor's discipline than a single green result.

Known bugs and regression tests

A known, not yet fixed bug is marked in the test as an expected failure with a ticket number, never by skipping the test. Ask for the list of such annotations: its length and age show how quickly the vendor removes known problems.

Figures to compare in the review

Figures to compare in the reviewBar chart: 242 E2E tests, 31 test files and a limit of 21 events per hour per error type.050100150200250valueE2E tests: 242 value242 valueE2E testsTest files: 31 value31 valueTest filesLimit of events per hour per error type: 21 value21 valueLimit of events per hour pererror type
Source: and

Change control

The second group concerns the path from a change to a release. Every change in the repository goes through static analysis, style checks and unit tests, and merging is blocked until the result is green. Ask whether this rule applies without exceptions and how the vendor documents exceptions if they occur.

The documentation also describes an independent review agent that assesses the change against its description and the domain rules, with three classes of comments (blocking, weighted warnings, suggestions) and a limit of two rounds. This mechanism is described as designed and built, but, as of August 2026, not yet deployed to production, so ask the vendor about its current state instead of assuming it works.

Error monitoring and reports

The third group concerns what the vendor sees after deployment. All instances report uncaught exceptions to a single error monitoring project, and the monthly budget of 5,000 errors is a shared resource. Ask how the vendor keeps this budget under control and what it does when one instance generates an excess of events.

The limit of error events is 21 per hour per error type. On days without incidents the worst day gave 305 events (about 12.7 per hour), and a loop failure reached 180 per hour. Ask for data from the last year: the number of days when the limit was exceeded and how each of them was handled.

Review the plant's list of reports: response times, resolution times and recurring topics. A recurring topic is a signal that a fix should be ordered at the source, not handled through reports.

Feature development and summary

The fourth group concerns new features. Check which orders from the last year were deployed, in which version and with what test result. Ask how the vendor assesses the risk of further changes and what it plans for the next year.

At the end, compare cost with value: the subscription, transmitter maintenance, mobile device rental and development. Record the conclusions of the review: what the vendor should improve, by when, and who in the plant will check it. An update on the InnerWeb side does not replace readers or cards, so in the review do not look for access control hardware costs linked to the software.

Checklist

Video

Frequently asked questions

How often should a vendor review be held?

Once a year, and additionally after a serious failure. An annual rhythm fits the billing of the subscription and the planning of development.

Who should take part in the review?

The IT department, the health and safety manager and purchasing. IT assesses tests and monitoring, health and safety assesses functions and alarms, and purchasing assesses costs and the contract.

Must the vendor share test results?

It is worth writing such an obligation into the contract. The result of a full E2E run is the basic proof that quality is maintained.

Carry out the review with us

We are happy to go through the annual review with you: tests, changes, errors and development. Book a conversation with InnerWeb.

Let's talk