Jeff Liu Lab
Home
Projects
Workshop
AI Wiki
AI Lab
Shop
中
Sign In
All
Computing Science
Artificial Intelligence
Deep Learning
Reinforcement Learning
AI Agents
Embodied Intelligence
Robot Engineering
Human-Like Intelligence
AI Engineering
← Back to Wiki
Deep Learning
Fundamentals & Summary
Deep Learning Landscape
Introduction to Deep Learning
Feedforward Neural Networks
Probability & Statistics in DL
Loss Functions
Convolutional Neural Networks
Recurrent Neural Networks
Generative Models
Graph Neural Networks
Large Language Models
Foundation Models
New Sequence Model Paradigms
Optimization & Regularization
DL Frontiers
Comments (0)
Sign in to comment
Table of Contents
How to Read This Report
Table of Contents
1. Cross-Dimensional Alignment of Five Core Concepts
1.1 Full Concept Comparison Matrix
1.2 Hierarchical Relationships: The Containment Structure of the Five Concepts
1.3 Practical Example: All Five Concepts in a Single Project
2. Loss Functions from Interdisciplinary Perspectives
2.1 The Statistical Perspective: Prior Assumptions About Noise Distributions
2.2 The Information-Theoretic Perspective: "Communication Cost" Between Distributions
2.3 The Physics Perspective: Energy Equilibria and Potential Energy Surfaces
2.4 The Economics and Decision Theory Perspective: Regret and Utility
3. Comprehensive Summary of Loss Functions by Task
3.1 Regression Tasks: Pursuing Accuracy in Continuous Values
3.2 Classification Tasks: Pursuing Overlap of Probability Distributions
3.3 Metric Learning and Geometric Spaces: Pursuing Clustering in Embeddings
4. Modern Frontier Loss Functions
4.1 Preference Alignment for Large Language Models (LLMs)
4.2 Self-Supervised Learning (SSL)
4.3 Loss Functions for Generative Models
5. Engineering Implementation: Framework Differences and Common Pitfalls
5.1 Logits vs. Probabilities: The Most Common Fatal Error
5.2 Reduction Strategies
5.3 Numerical Stability Best Practices
6. Selection Guide and Summary
6.1 Decision Tree: How to Choose a Loss Function
6.2 Core Philosophy Summary
6.3 The Essence in One Sentence
7. References and Further Reading
Foundational Theory
Classic Papers
Frontier Alignment Research
Comments
Comments (0)
Sign in to comment