Skip to content

Do not try step if there is invalid direction (DDP and FDDP)

@nmansard found out that our solvers were trying step even though the direction wasn't computed (or invalid). In this PR, I implemented a very simple rule: increasing regularization up to finding a valid direction.

This definitely avoids to try any invalid direction. If this approach is reasonable for @nmansard, I will improve the efficiency of the code by avoiding to recompute stuff in the backwardPass.

Edited by Carlos Mastalli

Merge request reports