Introduction: The COVID-19 pandemic and the lockdowns have affected many aspects of university students’ daily lives, including their dietary habits. This study aimed to evaluate the change of diet quality of university students before and during the COVID-19 period, and the factors associated with unfavorable changes in diet quality. Methods: An online cross-sectional study was performed in May 2021 among Rouen (France) university students. Socio-demographic characteristics, body mass index, depression, academic stress, risk of eating disorders and food security were collected. The French “Programme National Nutrition Santé-Guidelines Score 2” (PNNS-GS2) was used to access diet quality. Results: A total of 3508 students were included, 74.4% were female, the mean age was 20.7 (SD = 2.3), and 7.0% were in a situation of food insecurity. The PNNS-GS2 score decreased between the pre- and the COVID-19 pandemic period for 33.1% of university students. The associated factors with the decrease in the PNNS-GS2 score were food insecurity, financial insecurity, not living with parents, depression, academic stress, eating disorders, being in the two first years of study and having been infected by COVID-19. Conclusions: Diets with healthy components decreased for one-third of university students since the COVID-19 pandemic, and this was shown to be associated with food insecurity, poor mental health and eating disorder. This study provides important information to help public health authorities and universities give better support to student health feeding programs during pandemics and lockdowns.