This study addresses the actual behavior of the credit-card fraud detection environment where financial transactions containing sensitive data must not be amassed in an enormous amount to conduct learning. We introduce a new adaptive learning approach that adjusts frequently and efficiently to new transaction chunks; each chunk is discarded after each incremental training step. Our approach combines transfer learning and incremental feature learning. The former improves the feature relevancy for subsequent chunks, and the latter, a new paradigm, increases accuracy during training by determining the optimal network architecture dynamically for each new chunk. The architectures of past incremental approaches are fixed; thus, the accuracy may not improve with new chunks. We show the effectiveness and superiority of our approach experimentally on an actual fraud dataset.