Wired

Distillation Can Make AI Models Smaller and Cheaper

The original version of this story appeared in Quanta Magazine. The Chinese AI company DeepSeek released a chatbot earlier this year called R1, which drew a huge amount of attention. Most of it ...
CNBC

How DeepSeek used distillation to train its artificial intelligence model, and what it means for companies such as OpenAI

Chinese artificial intelligence lab DeepSeek roiled markets in January, setting off a massive tech and semiconductor selloff after unveiling AI models that it said were cheaper and more efficient than ...
Forbes

Knowledge Distillation For AI Democratization

Businesses are increasingly aiming to scale AI, but they often encounter constraints such as infrastructure costs and computational demands. Although large language models (LLMs) offer great potential ...
Forbes

Here’s How Big LLMs Teach Smaller AI Models Via Leveraging Knowledge Distillation

Forbes contributors publish independent expert analyses and insights. Dr. Lance B. Eliot is a world-renowned AI scientist and consultant. In today’s column, I examine the rising tendency of employing ...
AlphaGalileo

Key findings illustrating dark knowledge to facilitate powerful distillation

As large models advance, there’s growing demand to use knowledge distillation to produce smaller, more portable models (student) that match ...
AlphaGalileo

Awakening Dark Knowledge: Addressing Capacity Mismatch in Distillation

Sub-headline: Nanjing University researchers explore dark knowledge mechanisms to tackle the teacher-student capacity gap.