n o ren
AI & Technology

Does AI Automation Kill Expert Judgment?

In 2009, Air France 447's pilots lost manual flying skill to autopilot reliance, then couldn't recover from a stall.

On June 1, 2009, Air France Flight 447 flew into a tropical storm over the Atlantic with the captain asleep in the crew rest area and two co-pilots at the controls. Ice crystals blocked the plane's pitot tubes and the autopilot disconnected, handing the Airbus A330 to pilots who had spent years monitoring flight computers rather than hand-flying at cruise altitude. Within seconds, the pilot flying pulled the nose up into a steep climb, and a stall warning sounded for most of a minute. Neither co-pilot diagnosed the stall, because neither had practiced manual recovery from one at high altitude since training.

The BEA's investigation found a deeper design problem behind the human error. Airbus's fly-by-wire system enforces stall protection automatically under normal law, but that protection vanishes the moment the automation drops out, so the pilots lost their safety net at the exact moment they were suddenly asked to fly manually. The jet fell for three and a half minutes, engines at climb power and nose still pitched up, before it struck the ocean and killed all 228 people aboard. Cockpit recordings later showed the pilots calling out contradictory instructions, unable to agree on what the aircraft was doing.

Investigators named the underlying pattern automation dependency: the more a system absorbs routine control, the less practiced its operators become at the exact skill retained for emergencies. Regulators and airlines responded within a few years by mandating recurrent manual-handling and stall-recovery training worldwide, an admission that heavy automation had quietly hollowed out a core piloting skill it was never meant to replace. The lesson generalizes past aviation: any system that removes the thinking step from routine work must deliberately reintroduce moments where the human practices it, or that skill atrophies exactly when it's needed most.

Automation replaces the routine reps that keep a skill sharp, so the skill is weakest exactly when the automation fails.
Envelope-protection systems that vanish the moment automation drops out can hand operators an unfamiliar system at the worst possible time.

Ignoring automation dependency lets teams retain fewer manual-override skills exactly when a system fails outside its training envelope, raising the stakes of every edge case.

Skills that atrophy quietly are invisible until a crisis, so leaders discover the gap only after a costly failure has already happened.

1
Pull up the last five incidents where a human overrode an automated system in your workflow, and count how many overrides succeeded without a runbook or documentation to consult.
2
Ask your team how many hours of unassisted, no-AI practice on the core task they've logged in the past month, and compare the count across senior versus junior staff.

The BEA's 2012 final report on Flight 447 is one of the most detailed public studies of automation dependency ever produced, and it directly shaped how aviation regulators think about startle effect and skill decay. Journalist William Langewiesche's account, 'The Human Factor,' argued that increasingly capable autopilots had made basic airmanship a rarely used skill well before the accident. The same pattern shows up in medicine, where clinicians who lean heavily on decision-support software can lose the habit of independently double-checking its recommendations.

Software teams see a milder version of the same effect: continuous-integration pipelines that auto-merge passing code reduce how often engineers manually review changes, which can let subtle defects slip through until the pipeline itself has a blind spot. The fix in every domain is structurally similar: deliberately preserve moments of unassisted practice or review, rather than assuming automation will always catch what a human once would have.