[HuggingFace] VLM Fine tuning: Training Loss goes near 0 on train dataset. But the performance on training dataset is poor

Part of

Deep Learning for Pavement Infrastructure Assessment