Prevent cancelling updates from stopping FISTA early - #701

FISTA can stop after two iterations on a positive, well-conditioned least-squares problem because abs(sum(x - x_new)) lets opposing coordinate updates cancel. With default tolerance, the example below returns (2.125, 3.125, 1.5) instead of (1, 3.5, 1.5), leaving a stationarity residual of about 1.352.

Use sum(abs(x - x_new)), the documented L1 iterate-change criterion. This preserves the relative stopping rule and avoids squaring large updates. It applies to both matrix normal equations and tensor-core equations used by HALS Tucker decomposition.

With the default NumPy backend:

import numpy as np
from tensorly.solvers.nnls import fista

gram = np.diag([1.0, 2.0, 4.0])
rhs = np.array([[1.0], [7.0], [6.0]])
initial = np.array([[3.0], [2.0], [2.0]])
result = fista(rhs, gram, x=initial)  # Default tolerance and step size.
print(result.ravel())
print("stationarity residual:", np.linalg.norm(gram @ result - rhs))
np.testing.assert_allclose(result, np.linalg.solve(gram, rhs), rtol=1e-6, atol=1e-6)

The Gram matrix is positive definite with condition number 4; its minimizer is strictly positive, so the NNLS KKT condition here is simply gram @ result == rhs. The second update has nonzero changes (0.375, -0.375, 0): their signed sum is zero while their L1 norm is 0.75. The corrected call returns approximately (1, 3.5, 1.5) (residual 9.7e-8 in the NumPy CPU replay); the unchanged implementation also reaches it when early stopping is disabled with tol=0.

CPU validation:

• The complete NNLS and Tucker test modules pass 23 tests on each of NumPy and PyTorch, including two new matrix/tensor regression cases that fail on BASE.
• A separate analytic/SciPy NNLS/dense Kronecker reference passes 54 cases per backend across float32/float64. Independent review passes 56 additional scenarios/272 checks, including coupled SPD and tensor equations, KKT conditions, active boundaries, regularization and unchanged iteration-cap/tol=0 controls.
• The complete configured NumPy pytest/coverage command passes 979 tests, with 12 skips and one existing xpass (BASE: 977 passes with the same skips/xpass and 35 identical warning diagnostics). Configured Black passes all 156 files on both states.
• Both unmodified HTML documentation builds execute all 13 selected gallery examples successfully. They retain the same 48 baseline Sphinx warnings; no warning suppression or warning-free claim.

Validation used source-bound CPU imports. The full backend/interpreter CI matrix, GPU, general AD/convergence guarantees and performance claims are outside this change's proof scope.

CI follow-up, 5 November 2026: made every tensor in the new regression use one explicit float64 context, including the auxiliary tensor-core factor. The data, tolerance, two regression cases and production correction are unchanged. The latest hosted matrix passes NumPy, PyTorch, Paddle, JAX and TensorFlow, plus Black and coverage checks. The Paddle job passes on the submitted head 58c84b8.

The fixture also retains BASE mathematical failures and candidate passes in qualified NumPy/PyTorch/current Paddle 3.3.1 CPU checks. That extra local full Paddle 3.3.1 suite has eight tensor-ring sampling failures also reproduced on the unchanged previous submitted head; it is not a full-green result or an exact local replay of the hosted legacy environment.

I directed this work and take responsibility for the contribution and its follow-up. AI assistance was used in producing the change.