Reversible checkpointing automatic differentiation is a technique that allows to save and restore the state of an automatic differentiation computation at specific checkpoints. This enables pausing a training process, performing other operations (e.g., hyperparameter optimization), and then resuming the training from the last checkpoint. The reversibility provides the flexibility to modify the computation graph or perform sensitivity analysis without retraining the model.
