Catastrophic Forgetting

Catastrophic forgetting (also called catastrophic interference) is when a machine learning model forgets what it learned before as it learns something new. You can also call it a failure of stability, where new experience overwrites old experience.

It looks like this.

• A neural network "forgets" a task it learned before, or gets much worse at it, after learning a new task
• An artificial neural network suddenly and drastically loses old information as it takes in new information
• A network ends up working with data too far from its basic training
• A model focuses only on task B and steps in the direction of that task's gradient
• The model gets worse at telling apart data from different tasks

The current fixes for forgetting ignore the problem of catastrophic remembering.