Vef-BART: an effective method to mitigate hallucinations through vision enhancement and fusion in BART-based multimodal abstractive summarization

A novel visual encoder is designed that aims to obtain summary-oriented visual features to help generate higher-quality summaries and introduces a minimum margin loss to suppress the overconfidence problem of the model when generating text during reasoning.