The importance of implementing strict processes for software development, testing and deployment were emphasized once more when earlier in March 2022, domestic kitchen chefs in the Netherlands suddenly found that their AEG hot air / microwave kitchen appliance, had transformed itself into a quite different tool. The appliance became inoperable and complained to the user with error messages.
The cause of this behavior was an over-the-air (OTA) software update released by AEG on March 2, 2022. The appliance received the update, followed its hard coded instructions to flash the new firmware and performed a restart. Sadly, the update also affected the WiFi connection making it impossible to remotely correct the problem and leaving AEG no other choice than to have a repair man making house calls.
According to a spokesman of Electrolux (owner of AEG), the cause stems from an operator who applied a wrong number during the update procedure. This activated a wrong software update to be deployed.
It is a too simple statement to blame an operator for applying a wrong number, allowing an erroneous update of this magnitude. There are many options to implement checkpoints in or before various process steps in a modern software development environment. Some of them can be as simple as a peer review or multi operator activation, others can be fully automated checks. Without knowing the details of this particular case, it is evident that this error could have been avoided by implementing at least these checks:
- Cross check of the model number and firmware version by operator, combined with process automation tooling
- Safety check performed by the deployment software
- Firmware / model check performed by the appliance before the update
- Firmware backup / Restore architecture within the appliance
Errors like these are a companies’ nightmare and it’s not only happening in household equipment. In another recent example, the American plane manufacturer Boeing saw it’s 737 MAX airplanes grounded for 20 months, leading to enormous financial damages. This is obviously a worse example but it’s definitely food for thought – are you sure this could not happen to your organization?
At Logic Technology we help developers and engineers by providing tools and services supporting the entire SDLC. This includes solutions for enriched collaboration, online reviews by stakeholders and workflows automation to configure your IT Process. We provide a configurable framework to define, monitor and orchestrate any software process, whether you’re performing Product Development, Service or Software Maintenance activities.
Using a graphical workflow designer combined with a powerful workflow engine can save costs and prevent mistakes by human interference. With ALM and DevOps solutions you can define and execute processes spanning across multiple tools and teams within you’re organization.
Additional Project management functionality like a Task scheduler, project calendar, resource availability, and workload monitor assists in planning and controlling the application development process from beginning to end. This end-to-end visibility helps organizations to make the development process faster with better quality.
If testing is a weak spot in your organization we recommend reading our blog on regression testing or contact us to discuss your use-case and let us help to work on prevention, as it is better and cheaper than to work on a cure. Do you have any tips on how to prevent costly errors like these? Comment below!