InnerWeb
Skip to main content

Checklist before the maintenance window

The maintenance window for a container system update is best prepared the day before. The service team and the IT department then check the migrations, the pre-production, the backups and the state of the pipeline. The list below collects the points that shorten the window and reduce the risk of surprises on production.

3-32 s
migration time on production copies of six installations
Source: card
6
installations in the migration trial
Source: card

The rule: the day before, not an hour before

Most problems of a maintenance window can be detected the day before, if there is a pre-production with the same version and data close to production. An hour before the window is enough to confirm, not to diagnose.

The list is divided into four groups: change and migrations, pre-production, backups and rollback, communication and on-call. Each group is ticked off by the person named in the window plan.

Group 1: change and migrations

The change must pass the migration chain on an empty database in CI, and the database must match the code. The migrations were additionally checked on copies of production data of six installations with the code of the new branch; each finished successfully, and the run time ranged from 3 to 32 seconds.

For an installation of our own, the service team estimates the migration time from the trial on pre-production. The result goes into the window plan as an indicative time, with no promise of accuracy.

Migration time on production copies (range)

Migration time on production copies (range)Two bars: shortest migration 3 seconds, longest 32 seconds010203040sShortest: 3 s3 sShortestLongest: 32 s32 sLongest
Source: InnerWeb migration chain documentation

Group 2: pre-production

The twin pre-production gets the same change first. We check the start of the containers, the execution of the migrations and the login. Only a green result on pre-production allows the production to be scheduled.

If pre-production has not passed, we move the window. Moving it is cheaper than stopping the work permits during a change on production.

Group 3: backups and rollback

Before the window we confirm that a fresh backup is on the client's disks and that the way back to the previous image version is known. The data does not leave the plant, so the backup is available locally.

We keep in mind one property of the deployment: the deployment to the host is a task started manually and cannot be interrupted. A new commit will not stop an update that has started, so we begin it only when the team is ready to carry it to the end.

Group 4: communication and on-call

The IT department informs the users about the window, and about the fact that the mobile app queues data offline and that the server will accept the backlogged events after the connection returns. The managers of dangerous work agree a substitute process for the duration of the window.

We name the on-call person on the client's side and on the service side, and the time of the decision: we continue, or we return to the previous version.

Checklist

Video

Frequently asked questions

When is the best time to run through the list?

The day before the window, to leave time for removing errors. On the day of the window only a confirmation is done.

What if pre-production does not pass the trial?

We move the window. The cost of moving it is lower than stopping the system on production.

Can the deployment be interrupted during the window?

A deployment to the host that has started cannot be interrupted, so the decision to start should be deliberate.

Do mobile app users lose data during the window?

No. The app queues data offline and sends it to the server once the connection returns.

Let's talk about the installation in your server room

We will present the requirements and the split of responsibilities for your plant based on the installation documentation. Book a conversation with the InnerWeb team.

Let's talk