One-Pass Learning with Incremental and Decremental Features

In many real tasks the features are evolving, with some features vanished and some other features being augmented. For example, in environment monitoring some sensors expired whereas some new ones were deployed; in mobile game recommendation some games dropped whereas some new ones were added. Learning with such incremental and decremental features is crucial but rarely studied, particularly when the data comes like a stream and thus it is infeasible to keep the whole data for optimization. In this paper, we study this challenging problem and present the OPID approach. Our approach attempts to compress important information of vanished features into functions of survived features, and then expand to include the augmented features. It is an one-pass learning approach, which only needs to scan each instance once and does not need to store the whole data, and thus satisfies the evolving streaming data nature. After tackling this problem in one-shot scenario, we then extend it to multi-shot case. Empirical study on a broad range of data sets shows that our approach can address this problem effectively.