Leveraging Model Inherent Variable Importance for Stable Online Feature Selection
Aug 13, 20202 views
Feature selection can be a crucial factor in obtaining robust and,accurate predictions. Online feature selection models, however,,operate under considerable restrictions; they need to efficiently extract salient input features based on a bounded set of observations,,while enabling robust and accurate predictions. In this work, we introduce,FIRES,, a novel framework for online feature selection. The,proposed feature weighting mechanism leverages the importance,information inherent in the parameters of a predictive model. By,treating model parameters as random variables, we can penalize,features with high uncertainty and thus generate more stable feature sets. Our framework is generic in that it leaves the choice of,the underlying model to the user. Strikingly, experiments suggest,that the model complexity has only a minor effect on the discriminative power and stability of the selected feature sets. In fact, using,a simple linear model,,FIRES,obtains feature sets that compete with,state-of-the-art methods, while dramatically reducing computation,time. In addition, experiments show that the proposed framework,is clearly superior in terms of feature selection stability.