An Evaluation of the Replicable Factor Analytic Solutions Algorithm for Variable Selection

A Simulation Study

Bibliographic Data

ID20287366
AuthorsDaniel A Sass (0000-0001-6774-9102, University of Texas at San Antonio, USA, corresponding author), Michael A Sanchez (0000-0001-8621-8569, H-E-B and University of Texas at San Antonio, USA)
Year2026
Volume86
Issue2
Pages393-427
Publication date2026-04-01
Open AccessYes
TypeARTICLE
VenueEducational and Psychological Measurement (JOURNAL)
Journal identifiersISSN: 0013-1644 • E-ISSN: 1552-3888
PublisherSAGE Publications (PUBLISHER • US)
DOI10.1177/00131644251377381
PMID41195449
OpenAlexW4415791553
LanguageEN
References cited27

Observed variable and factor selection are critical components of factor analysis, particularly when the optimal subset of observed variables and the number of factors are unknown and results cannot be replicated across studies. The Replicable Factor Analytic Solutions (RFAS) algorithm was developed to assess the replicability of factor structures—both in terms of the number of factors and the variables retained—while identifying the “best” or most replicable solutions according to predefined criteria. This study evaluated RFAS performance across 54 experimental conditions that varied in model complexity (six-factor models), interfactor correlations (ρ = 0, .30, and .60), and sample sizes ( n = 300, 500, and 1000). Under default settings, RFAS generally performed well and demonstrated its utility in producing replicable factor structures. However, performance declined with highly correlated factors, smaller sample sizes, and more complex models. RFAS was also compared to four alternative variable selection methods: Ant Colony Optimization (ACO), Weighted Group Least Absolute Shrinkage and Selection Operator (LASSO), and stepwise procedures based on target Tucker–Lewis Index (TLI) and ΔTLI criteria. Stepwise and LASSO methods were largely ineffective at eliminating problematic variables under the studied conditions. In contrast, both RFAS and ACO successfully removed variables as intended, although the resulting factor structures often differed substantially between the two approaches. As with other variable selection methods, refining algorithmic criteria may be necessary to further enhance model performance

Factor (programming language) · Feature selection · Lasso (programming language) · Sample (material) · Sample size determination · Selection (genetic algorithm) · Variable (mathematics) · Variables · Advanced Multi-Objective Optimization Algorithms · Optimal Experimental Design Methods · Psychometric Methodologies and Testing

Tools

Open DOIOpen Access