Spectral Gap Dynamics in Large-Cap Equity Correlation Matrices: Evidence of Regime-Dependent Factor Structure, 2010–2024

Question

Does the eigenvalue spectrum of a large-cap equity correlation matrix exhibit time-varying structure consistent with market regime shifts, and how does the empirical density of eigenvalues deviate from the Marchenko-Pastur random-matrix null to signal the presence of statistically significant common factors?

Method

We computed the eigenvalue decomposition of the return correlation matrix for 11 large-cap U.S. equities (AAPL, AMZN, BAC, CVX, GOOGL, JNJ, JPM, META, MSFT, NVDA, PFE, XOM) using daily adjusted-close returns from yfinance over the window 2010-01-01 to 2024-12-31 (3,772 observations). The data source is yfinance daily adjusted-close returns; the method is PCA eigenvalue spectrum of the return correlation matrix versus the Marchenko-Pastur null, where eigenvalues above the MP upper bound are statistically distinguishable from random-matrix noise.

The Marchenko-Pastur distribution provides a null hypothesis for the eigenvalue spectrum of a correlation matrix constructed from uncorrelated Gaussian returns. For a matrix with n = 11 assets and T = 3,772 observations, the ratio q = n/T = 0.003 yields theoretical bounds [λ_min, λ_max] = [0.8949, 1.1109] under the null of no common structure. Eigenvalues exceeding λ_max = 1.1109 indicate the presence of genuine common factors beyond random noise.

To assess temporal variation, we recomputed the eigenvalue spectrum and factor count on a per-calendar-year basis (in-sample within each year) using the same data and method, producing a rolling time series of the number of significant factors from 2010 through 2024.

Result

Full-sample eigenvalue spectrum (2010–2024)

The top 10 eigenvalues of the 11×11 correlation matrix are: 5.1473, 1.5929, 0.9728, 0.7274, 0.5580, 0.5024, 0.4561, 0.3919, 0.3428, 0.1652. Against the Marchenko-Pastur upper bound of 1.1109, exactly 2 eigenvalues exceed the threshold, establishing n_significant_factors = 2.

The dominant eigenvalue λ₁ = 5.1473 accounts for 46.79% of total variance (variance_explained_top1 = 0.4679). The two significant factors together explain 61.28% of variance (variance_explained_significant = 0.6128). The remaining 9 eigenvalues lie below the MP upper bound, consistent with random noise or idiosyncratic variation.

Factor loadings and economic interpretation

Factor 1 (λ₁ = 5.1473): The top three loadings are JPM (−0.343), MSFT (−0.335), and BAC (−0.331). This factor exhibits broad, near-uniform loadings across the universe, consistent with a market-wide common mode. The negative sign is an arbitrary rotation; the magnitude indicates that all 11 assets co-move strongly on this dimension.

Factor 2 (λ₂ = 1.5929): The top three loadings are AMZN (−0.399), XOM (+0.395), and CVX (+0.368). This factor separates technology (AMZN, negative loading) from energy (XOM, CVX, positive loadings), consistent with a sector-rotation or growth-versus-value dimension. The opposing signs indicate that when energy stocks rise on this factor, technology stocks fall, and vice versa.

Time variation in factor count (2010–2024)

The per-year significant factor count reveals clear temporal structure:

  • 2010–2015: Stable single-factor regime. Each year exhibits n_significant_factors = 1, indicating that a single market-wide mode dominates correlation structure.
  • 2016: Transition to two-factor regime (n_significant_factors = 2).
  • 2017: Two-factor regime persists.
  • 2018–2019: Reversion to single-factor regime.
  • 2020–2024: Sustained two-factor regime. Every year from 2020 onward exhibits n_significant_factors = 2.

The transition points (2016, 2020) coincide with known market regime shifts: the 2016 U.S. election and associated policy uncertainty, and the 2020 COVID-19 pandemic and subsequent monetary/fiscal intervention. The sustained two-factor structure post-2020 suggests persistent differentiation between growth and value/energy sectors, consistent with the factor-2 loadings.

Spectral gap and noise floor

The spectral gap between the second significant eigenvalue (λ₂ = 1.5929) and the third eigenvalue (λ₃ = 0.9728) is 0.6201. The third eigenvalue lies 0.1381 below the MP upper bound, firmly in the noise regime. This gap is large relative to the spacing among noise eigenvalues (e.g., λ₃ − λ₄ = 0.2454), indicating a clean separation between signal and noise. The absence of eigenvalues in the interval (1.1109, 1.5929) confirms that the two-factor model is not an artifact of threshold choice.

Interpretation

What the result supports

The eigenvalue spectrum provides strong evidence for time-varying low-dimensional factor structure in large-cap equity returns. The full-sample result (2 significant factors, 61.28% variance explained) demonstrates that the correlation matrix is not well-approximated by a random-matrix null: the top two eigenvalues are 4.6× and 1.4× the Marchenko-Pastur upper bound, respectively, far beyond sampling variation.

The rolling factor count reveals that this structure is not static. The single-factor regime of 2010–2015 reflects a period of broad, undifferentiated co-movement (the "risk-on/risk-off" regime documented in post-crisis equity markets). The emergence of a second factor in 2016 and its persistence from 2020 onward indicates regime-dependent spectral gap collapse: the gap between λ₂ and the noise floor narrows or widens as market conditions change, but the gap between λ₂ and λ₃ remains stable when two factors are present.

The factor loadings provide economic content. Factor 1 is a market-wide mode, consistent with systematic risk. Factor 2 is a sector-rotation mode (technology vs. energy), consistent with the growth-value divergence that intensified during the pandemic and subsequent inflation cycle. The fact that this second factor crosses the significance threshold only in specific years (2016–2017, 2020–2024) suggests that sector differentiation is regime-dependent, not a permanent feature of the correlation structure.

What the result does not support

The result does not support the hypothesis of a "spectral gap collapse" in the sense of eigenvalues merging or the gap vanishing. The gap between λ₂ and λ₃ remains large (0.6201) in the full sample, and the per-year analysis shows discrete transitions (1 factor → 2 factors) rather than continuous gap erosion. The term "collapse" is a misnomer; the phenomenon is better described as discrete regime switching in the number of significant factors.

The result does not establish causality or predictive power. The per-year factor counts are in-sample within each year; we have not tested whether a change in factor count at year-end predicts returns or volatility in the subsequent year. The correlation between regime transitions (2016, 2020) and known macro events is suggestive but not dispositive—many macro events do not coincide with factor-count changes (e.g., 2018 volatility spike, 2022 rate-hike cycle).

The result does not generalize beyond the specific universe (11 large-cap U.S. equities, 2010–2024). The q-ratio = 0.003 is extremely low (T >> n), which sharpens the Marchenko-Pastur bounds but also means the test has high power to detect even weak factors. A larger universe or shorter window would yield different bounds and potentially different factor counts. The choice of daily frequency and 15-year window is arbitrary; weekly or monthly data might reveal different dynamics.

Out-of-sample considerations

This analysis is entirely in-sample. The per-year factor counts are computed on the same data used to estimate the correlation matrix within each year; there is no hold-out period or forward-looking test. An out-of-sample extension would require: (1) estimating the correlation matrix on data up to year t, (2) computing the factor count, (3) testing whether that count predicts the correlation structure or return dispersion in year t+1. Without such a test, we cannot claim that the factor-count signal is exploitable or that regime transitions are detectable in real time.

Relation to the Literature

The result sits at the intersection of random-matrix theory (Marchenko-Pastur null) and empirical asset pricing (factor models). While none of the supplied papers directly address equity correlation matrices, several provide relevant context:

[P2] develops a spectral framework for tracking communities in evolving networks, treating community detection as subspace tracking on the Grassmann manifold. The analogy to our setting is direct: each year's correlation matrix defines a subspace (the span of the top k eigenvectors), and the per-year factor-count series tracks how that subspace dimension changes. [P2]'s Riemannian optimization framework could, in principle, be applied to smooth the year-to-year transitions and estimate a continuous trajectory of factor structure, though we have not done so here.

[P7] applies correlation-based complex network analysis and community detection to tourism seasonality, using modularity optimization to classify temporal patterns into regional groups. The method—correlation matrix → eigenvalue decomposition → community structure—parallels our approach. [P7]'s finding of "four groups of seasonality" with "distinct seasonal, geographical, and socio-economic profiles" mirrors our finding of regime-dependent factor structure, though the domain (tourism) and the number of regimes differ.

[P9] uses principal component analysis on socioeconomic indicators to cluster African countries, then examines how the clustering relates to social contact patterns. The PCA step is mechanically identical to our eigenvalue decomposition, and the interpretation (PCA as dimensionality reduction revealing latent structure) is the same. [P9]'s focus on cross-sectional heterogeneity (countries) versus our focus on time-series variation (years) highlights the dual use of spectral methods: spatial clustering and temporal regime detection.

The remaining papers ([P1], [P3], [P4], [P5], [P6], [P8]) address spatial weights matrices, urban park systems, social networks, megaprojects, topological field theory, and urban-rural development, respectively. These domains are distant from equity markets, but [P1]'s estimation of spatial weights matrices "consistent with a given pattern of spatial autocovariance" shares a conceptual link: both [P1] and our analysis infer latent structure (spatial weights, factor loadings) from an observed covariance/correlation matrix. The difference is that [P1] treats the weights as parameters to estimate, whereas we treat the eigenvalues as test statistics against a null.

Tension with prior work: The random-matrix literature (not represented in the supplied papers) predicts that for large n and finite T, the bulk of eigenvalues should lie within the Marchenko-Pastur bounds, with only a few "spikes" corresponding to genuine factors. Our result (2 spikes, 9 noise eigenvalues) is consistent with this prediction. However, the time variation in spike count is less commonly documented. Most random-matrix applications to finance (e.g., Laloux et al. 1999, Plerou et al. 2002, not supplied) focus on static, full-sample spectra. Our per-year analysis extends this by showing that the number of spikes is not a fixed property of the universe but a regime-dependent quantity.

Agreement with prior work: The two-factor structure (market + sector) aligns with the Fama-French literature (not supplied), which documents that equity returns are well-described by a market factor plus size, value, and momentum factors. Our factor 2 (technology vs. energy) is not identical to HML or SMB, but the existence of a second dimension beyond the market is consistent. The regime-dependence (single-factor in 2010–2015, two-factor post-2020) is less emphasized in the Fama-French framework, which treats factors as static, but is consistent with the "betting-against-beta" and "quality-minus-junk" literature (not supplied), which documents time-varying factor premia.

Limitations

  1. Small universe, low q-ratio: With n = 11 and T = 3,772, the q-ratio = 0.003 is far below the regime (q ≈ 0.1–1) where Marchenko-Pastur asymptotics are typically applied. The bounds [0.8949, 1.1109] are tight, which increases test power but also means the result may not generalize to larger universes (e.g., S&P 500) where q is higher and the noise floor is elevated. A robustness check with n = 50 or n = 100 would clarify whether the two-factor structure persists.

  2. Arbitrary universe composition: The 11 tickers span technology (AAPL, MSFT, GOOGL, META, NVDA, AMZN), financials (JPM, BAC), energy (XOM, CVX), and healthcare (JNJ, PFE). This mix is not a standard index or sector; it is a convenience sample. The factor-2 loadings (technology vs. energy) are partly an artifact of this composition. A universe of 11 technology stocks would likely yield a single factor; a universe of 11 stocks evenly split across 11 sectors might yield more factors. The result is conditional on the universe.

  3. In-sample only, no forward test: The per-year factor counts are computed on the same data used to estimate the correlation matrix. This is not a predictive exercise. We have not tested whether a transition from 1 to 2 factors at year-end predicts higher return dispersion, volatility, or Sharpe ratio in the subsequent year. Without an out-of-sample test, the regime-detection claim is descriptive, not actionable.

  4. Daily frequency and 15-year window: The choice of daily returns and a 15-year window is arbitrary. Weekly or monthly returns would reduce T and raise q, potentially changing the factor count. A 5-year rolling window (instead of per-calendar-year) would provide finer temporal resolution but at the cost of fewer observations per window. The result is sensitive to these choices.

  5. No control for known events: The per-year factor counts show transitions in 2016 and 2020, which coincide with the U.S. election and COVID-19 pandemic. However, we have not controlled for other macro variables (VIX, term spread, credit spread, policy uncertainty index) that might explain the transitions. The correlation between regime shifts and known events is suggestive but not causal. A regression of factor count on macro variables would strengthen the interpretation.

  6. Gaussian assumption: The Marchenko-Pastur null assumes returns are i.i.d. Gaussian. Equity returns exhibit fat tails, autocorrelation, and heteroskedasticity, all of which violate this assumption. The eigenvalue bounds [0.8949, 1.1109] are derived under the Gaussian null; deviations from Gaussianity could shift the bounds or introduce spurious spikes. A bootstrap or permutation test (not performed here) would provide a more robust null.

  7. No economic model: The factor loadings (market, technology-vs-energy) are interpreted post hoc. We have not specified an economic model (e.g., ICAPM, APT) that predicts these factors or their time variation. The interpretation is plausible but not derived from theory. A structural model linking factor structure to macro state variables (growth, inflation, uncertainty) would provide a firmer foundation.

What would strengthen the result: (1) Replication on a larger, more representative universe (e.g., S&P 100 or Russell 1000). (2) Out-of-sample test: estimate factor count on data up to year t, test whether it predicts return dispersion or portfolio performance in year t+1. (3) Bootstrap or permutation test to validate the Marchenko-Pastur bounds under realistic return distributions. (4) Regression of per-year factor count on macro variables (VIX, policy uncertainty, term spread) to establish whether regime transitions are predictable. (5) Extension to other asset classes (bonds, commodities, FX) to test whether spectral regime-switching is equity-specific or a general phenomenon. (6) Comparison with alternative regime-detection methods (hidden Markov models, change-point detection) to assess whether eigenvalue-based detection is more sensitive or robust.

Research evidence, not investment advice.