Why Is the Loss Function of Large Models Cross-Entropy

Prologue When first entering the world of large models, due to gaps in foundational math knowledge such as linear algebra, probability theory, and information theory, it’s easy to get lost among numerous terms: logprob (log probability), likelihood, NLL (Negative Log Likelihood), cross entropy, perplexity. They frequently appear in various corners of papers and documentation, yet they all feel like acquaintances you only know by name — you see them often, but don’t truly understand them. Then one day, after slowly catching up on some basic math knowledge and immersing myself in the company con

• #llm
• #loss