Are We Really Doing Few-Shot Learning? A Critical Examination of Pre-Training Assumptions

RESUMO

Few-shot learning is commonly evaluated under protocols pre-training on an auxiliary set with disjoint classes from the target episodes but the same visual domain. Across eight datasets and three architectures, class disjointness alone is insufficient to remove the influence of target-domain data: in-domain pre-training improves over no pre-training by 33.4 points versus 23.75 for out-of-domain, a 9.66-point optimistic bias from domain overlap. A descriptor-based source-selection strategy narrows the gap to oracle selection to 1.37 points.