A Novel Evolutionary Feature Selection Method for High-Dimensional Data Classification

Abstract

The explosion of high-dimensional data across genomics, finance, and large-scale socio-technical infrastructures has necessitated a paradigm shift in how feature selection is integrated into the machine learning pipeline. Conventional dimensionality reduction techniques often fail to account for the complex, non-linear interdependencies inherent in massive datasets, leading to computational bottlenecks and degraded classification accuracy. This research proposes a novel evolutionary feature selection method designed to navigate the high-dimensional search space through a systemic, biologically inspired optimization framework. Beyond the algorithmic mechanics, this paper provides an extensive analytical discussion on the system-level integration of evolutionary computation within enterprise data infrastructures. We explore the structural trade-offs between global search exploration and local exploitation, the architectural requirements for distributed evolutionary deployment, and the socio-technical implications of automated feature engineering. The discussion emphasizes the importance of robustness, particularly in the context of "noisy" real-world data environments, and the sustainability of high-compute optimization processes. Furthermore, we examine the governance and policy frameworks necessary to ensure fairness and transparency in automated classification systems that rely on evolved feature subsets. By positioning feature selection as a critical component of systemic governance rather than a mere preprocessing step, this research offers a comprehensive roadmap for the next generation of scalable and accountable artificial intelligence.