Abstract: Despite the unprecedented improvement of face recognition, existing face recognition models still show considerably low performances in determining whether pairs with large age gaps (i.e., child and adult image pairs) belong to the same identity. Previous approaches mainly focused on increasing the similarity between child and adult images of an identity to overcome the discrepancy of facial appearances due to aging. However, we observe that reducing the similarity between child images of different identities is crucial for learning discriminative features of children and thus improving face recognition with large age gaps. Based on this intuition, we propose a novel loss function called the Inter-Prototype loss which minimizes the similarity between child images. Unlike the previous studies, the Inter-Prototype loss does not require additional child images or training additional learnable parameters. Our extensive experiments and in-depth analyses show that our approach outperforms existing baselines in recognizing the child-adult pairs with large age gaps.