Automation Bias is a cognitive bias in which people over-rely on automated systems, following their recommendations even when those recommendations are wrong or when available evidence suggests a different conclusion. As we increasingly depend on technology for decision support, from GPS navigation to medical diagnostics to AI assistants, automation bias becomes an increasingly consequential cognitive vulnerability.
This bias manifests in two main ways: errors of commission, where people follow automated advice that is incorrect; and errors of omission, where people fail to notice problems because they expected the automated system to catch them. Both reflect an excessive trust in technology that overrides human judgment and vigilance. The more sophisticated and usually reliable a system is, the more likely we are to defer to it uncritically.
Automation bias has led to serious accidents in aviation, medicine, and other domains. Pilots have followed malfunctioning autopilots into dangerous situations. Medical professionals have accepted incorrect computer-generated diagnoses despite contradictory patient symptoms. The more automation handles routine operations correctly, the more difficult it becomes to maintain the vigilance needed to catch the rare cases when it fails.
Managing automation bias requires maintaining appropriate skepticism and engagement with automated systems rather than passive acceptance of their outputs. Training should emphasize that automation can fail and develop skills for detecting such failures. System designs that keep humans actively engaged rather than merely monitoring can also help maintain the vigilance needed to catch automation errors.
