Alexander Todoran: Advancing Dimensionality Reduction: The Potential of Spectral Density-Based Stopping Criteria

Abstract.

In the rapidly evolving field of data analytics, Principal Component Analysis (PCA) remains a fundamental technique for dimensionality reduction, particularly in high-dimensional datasets common in machine learning and signal processing. The performance of PCA is critically influenced by its stopping criterion, which determines the extent of dimensionality reduction. Building upon the work of Ubaru et al. (2018)—who introduced a randomized algorithm based on Krylov subspace methods and the Lanczos algorithm, employing an information criterion as the stopping rule—this study explores an alternative strategy: Spectral Density Estimation. We demonstrate that this approach yields lower approximation error compared to the original criterion, reduces the risk of overfitting, and improves computational efficiency by decreasing runtime. These results position Spectral Density Estimation as a promising stopping criterion and pave the way for the development of adaptable, domain-specific dimensionality reduction techniques in deep learning and related fields.