Understanding Neural Tangent Kernel : Key Theories and Experimental Insights - Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes
Pré-Publication, Document De Travail (Working Paper) Année : 2024

Understanding Neural Tangent Kernel : Key Theories and Experimental Insights

Résumé

The Neural Tangent Kernel (NTK) has become a powerful framework for analyzing the behavior of deep neural networks in the infinite-width limit. This paper presents a concise overview of the key theoretical foundations of NTK, covering its origins, the proof of deterministic behavior at initialization, and its role in bounding the training loss for regression tasks. Additionally, we extend this analysis by establishing a bound for the training loss in classification problems. Each theoretical property of the NTK is validated through experiments on various datasets.
Fichier principal
Vignette du fichier
Neural_Tangent_Kernel-3.pdf (1.27 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04784111 , version 1 (19-11-2024)
hal-04784111 , version 2 (12-12-2024)

Identifiants

  • HAL Id : hal-04784111 , version 1

Citer

Samy Vilhes. Understanding Neural Tangent Kernel : Key Theories and Experimental Insights. 2024. ⟨hal-04784111v1⟩
6 Consultations
0 Téléchargements

Partager

More