3小时看懂ChatGPT内核：前OpenAI天才揭秘AI如何“思考“

刚刚花了3个小时观看了Andrej Karpathy的《Deep Dive into LLMs like ChatGPT》，这位前OpenAI创始成员、前特斯拉AI总监用最通俗的语言解开了大语言模型的神秘面纱。574万次观看量证明了这个视频的价值——这可能是2025年最重要的AI科普内容。

如果你想真正理解ChatGPT是如何工作的，而不仅仅是使用它，这篇总结将为你节省3小时，但给你同样深刻的洞察。

🔍 核心发现：AI不是魔法，是数学的胜利

大语言模型的本质

Karpathy用一个简单而深刻的比喻解释了LLM：它们本质上是超级压缩器。

#mermaid-svg-z3wEbJzdBa3RGL9T{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-z3wEbJzdBa3RGL9T .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-z3wEbJzdBa3RGL9T .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-z3wEbJzdBa3RGL9T .error-icon{fill:#552222;}#mermaid-svg-z3wEbJzdBa3RGL9T .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-z3wEbJzdBa3RGL9T .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-z3wEbJzdBa3RGL9T .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-z3wEbJzdBa3RGL9T .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-z3wEbJzdBa3RGL9T .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-z3wEbJzdBa3RGL9T .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-z3wEbJzdBa3RGL9T .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-z3wEbJzdBa3RGL9T .marker{fill:#333333;stroke:#333333;}#mermaid-svg-z3wEbJzdBa3RGL9T .marker.cross{stroke:#333333;}#mermaid-svg-z3wEbJzdBa3RGL9T svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-z3wEbJzdBa3RGL9T p{margin:0;}#mermaid-svg-z3wEbJzdBa3RGL9T .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-z3wEbJzdBa3RGL9T .cluster-label text{fill:#333;}#mermaid-svg-z3wEbJzdBa3RGL9T .cluster-label span{color:#333;}#mermaid-svg-z3wEbJzdBa3RGL9T .cluster-label span p{background-color:transparent;}#mermaid-svg-z3wEbJzdBa3RGL9T .label text,#mermaid-svg-z3wEbJzdBa3RGL9T span{fill:#333;color:#333;}#mermaid-svg-z3wEbJzdBa3RGL9T .node rect,#mermaid-svg-z3wEbJzdBa3RGL9T .node circle,#mermaid-svg-z3wEbJzdBa3RGL9T .node ellipse,#mermaid-svg-z3wEbJzdBa3RGL9T .node polygon,#mermaid-svg-z3wEbJzdBa3RGL9T .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-z3wEbJzdBa3RGL9T .rough-node .label text,#mermaid-svg-z3wEbJzdBa3RGL9T .node .label text,#mermaid-svg-z3wEbJzdBa3RGL9T .image-shape .label,#mermaid-svg-z3wEbJzdBa3RGL9T .icon-shape .label{text-anchor:middle;}#mermaid-svg-z3wEbJzdBa3RGL9T .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-z3wEbJzdBa3RGL9T .rough-node .label,#mermaid-svg-z3wEbJzdBa3RGL9T .node .label,#mermaid-svg-z3wEbJzdBa3RGL9T .image-shape .label,#mermaid-svg-z3wEbJzdBa3RGL9T .icon-shape .label{text-align:center;}#mermaid-svg-z3wEbJzdBa3RGL9T .node.clickable{cursor:pointer;}#mermaid-svg-z3wEbJzdBa3RGL9T .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-z3wEbJzdBa3RGL9T .arrowheadPath{fill:#333333;}#mermaid-svg-z3wEbJzdBa3RGL9T .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-z3wEbJzdBa3RGL9T .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-z3wEbJzdBa3RGL9T .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-z3wEbJzdBa3RGL9T .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-z3wEbJzdBa3RGL9T .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-z3wEbJzdBa3RGL9T .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-z3wEbJzdBa3RGL9T .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-z3wEbJzdBa3RGL9T .cluster text{fill:#333;}#mermaid-svg-z3wEbJzdBa3RGL9T .cluster span{color:#333;}#mermaid-svg-z3wEbJzdBa3RGL9T div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-z3wEbJzdBa3RGL9T .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-z3wEbJzdBa3RGL9T rect.text{fill:none;stroke-width:0;}#mermaid-svg-z3wEbJzdBa3RGL9T .icon-shape,#mermaid-svg-z3wEbJzdBa3RGL9T .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-z3wEbJzdBa3RGL9T .icon-shape p,#mermaid-svg-z3wEbJzdBa3RGL9T .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-z3wEbJzdBa3RGL9T .icon-shape rect,#mermaid-svg-z3wEbJzdBa3RGL9T .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-z3wEbJzdBa3RGL9T .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-z3wEbJzdBa3RGL9T .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-z3wEbJzdBa3RGL9T :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

互联网海量文本

数据清洗与预处理

分词器Tokenizer

神经网络训练

参数压缩1750亿个

预测下一个Token

生成智能回答

整个过程可以概括为：

分词：被忽视的关键环节

最令人意外的发现是分词器的重要性。这个看似简单的组件实际上是AI能力的最大限制之一。

#mermaid-svg-QBKYION6NGAGLapM{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-QBKYION6NGAGLapM .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-QBKYION6NGAGLapM .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-QBKYION6NGAGLapM .error-icon{fill:#552222;}#mermaid-svg-QBKYION6NGAGLapM .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-QBKYION6NGAGLapM .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-QBKYION6NGAGLapM .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-QBKYION6NGAGLapM .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-QBKYION6NGAGLapM .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-QBKYION6NGAGLapM .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-QBKYION6NGAGLapM .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-QBKYION6NGAGLapM .marker{fill:#333333;stroke:#333333;}#mermaid-svg-QBKYION6NGAGLapM .marker.cross{stroke:#333333;}#mermaid-svg-QBKYION6NGAGLapM svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-QBKYION6NGAGLapM p{margin:0;}#mermaid-svg-QBKYION6NGAGLapM .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-QBKYION6NGAGLapM .cluster-label text{fill:#333;}#mermaid-svg-QBKYION6NGAGLapM .cluster-label span{color:#333;}#mermaid-svg-QBKYION6NGAGLapM .cluster-label span p{background-color:transparent;}#mermaid-svg-QBKYION6NGAGLapM .label text,#mermaid-svg-QBKYION6NGAGLapM span{fill:#333;color:#333;}#mermaid-svg-QBKYION6NGAGLapM .node rect,#mermaid-svg-QBKYION6NGAGLapM .node circle,#mermaid-svg-QBKYION6NGAGLapM .node ellipse,#mermaid-svg-QBKYION6NGAGLapM .node polygon,#mermaid-svg-QBKYION6NGAGLapM .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-QBKYION6NGAGLapM .rough-node .label text,#mermaid-svg-QBKYION6NGAGLapM .node .label text,#mermaid-svg-QBKYION6NGAGLapM .image-shape .label,#mermaid-svg-QBKYION6NGAGLapM .icon-shape .label{text-anchor:middle;}#mermaid-svg-QBKYION6NGAGLapM .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-QBKYION6NGAGLapM .rough-node .label,#mermaid-svg-QBKYION6NGAGLapM .node .label,#mermaid-svg-QBKYION6NGAGLapM .image-shape .label,#mermaid-svg-QBKYION6NGAGLapM .icon-shape .label{text-align:center;}#mermaid-svg-QBKYION6NGAGLapM .node.clickable{cursor:pointer;}#mermaid-svg-QBKYION6NGAGLapM .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-QBKYION6NGAGLapM .arrowheadPath{fill:#333333;}#mermaid-svg-QBKYION6NGAGLapM .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-QBKYION6NGAGLapM .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-QBKYION6NGAGLapM .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-QBKYION6NGAGLapM .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-QBKYION6NGAGLapM .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-QBKYION6NGAGLapM .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-QBKYION6NGAGLapM .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-QBKYION6NGAGLapM .cluster text{fill:#333;}#mermaid-svg-QBKYION6NGAGLapM .cluster span{color:#333;}#mermaid-svg-QBKYION6NGAGLapM div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-QBKYION6NGAGLapM .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-QBKYION6NGAGLapM rect.text{fill:none;stroke-width:0;}#mermaid-svg-QBKYION6NGAGLapM .icon-shape,#mermaid-svg-QBKYION6NGAGLapM .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-QBKYION6NGAGLapM .icon-shape p,#mermaid-svg-QBKYION6NGAGLapM .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-QBKYION6NGAGLapM .icon-shape rect,#mermaid-svg-QBKYION6NGAGLapM .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-QBKYION6NGAGLapM .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-QBKYION6NGAGLapM .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-QBKYION6NGAGLapM :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

strawberry

分词器

['straw', 'berry']

神经网络

无法看到单个字符 'r'

这就解释了为什么ChatGPT能写出莎士比亚级别的诗歌，却可能数错"strawberry"中有几个’r’——它从未真正"看到"过单个字符。

🎯 训练的两个关键阶段

阶段一：预训练（获得世界知识）

#mermaid-svg-H89C6SlICFWalEgO{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-H89C6SlICFWalEgO .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-H89C6SlICFWalEgO .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-H89C6SlICFWalEgO .error-icon{fill:#552222;}#mermaid-svg-H89C6SlICFWalEgO .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-H89C6SlICFWalEgO .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-H89C6SlICFWalEgO .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-H89C6SlICFWalEgO .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-H89C6SlICFWalEgO .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-H89C6SlICFWalEgO .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-H89C6SlICFWalEgO .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-H89C6SlICFWalEgO .marker{fill:#333333;stroke:#333333;}#mermaid-svg-H89C6SlICFWalEgO .marker.cross{stroke:#333333;}#mermaid-svg-H89C6SlICFWalEgO svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-H89C6SlICFWalEgO p{margin:0;}#mermaid-svg-H89C6SlICFWalEgO .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-H89C6SlICFWalEgO .cluster-label text{fill:#333;}#mermaid-svg-H89C6SlICFWalEgO .cluster-label span{color:#333;}#mermaid-svg-H89C6SlICFWalEgO .cluster-label span p{background-color:transparent;}#mermaid-svg-H89C6SlICFWalEgO .label text,#mermaid-svg-H89C6SlICFWalEgO span{fill:#333;color:#333;}#mermaid-svg-H89C6SlICFWalEgO .node rect,#mermaid-svg-H89C6SlICFWalEgO .node circle,#mermaid-svg-H89C6SlICFWalEgO .node ellipse,#mermaid-svg-H89C6SlICFWalEgO .node polygon,#mermaid-svg-H89C6SlICFWalEgO .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-H89C6SlICFWalEgO .rough-node .label text,#mermaid-svg-H89C6SlICFWalEgO .node .label text,#mermaid-svg-H89C6SlICFWalEgO .image-shape .label,#mermaid-svg-H89C6SlICFWalEgO .icon-shape .label{text-anchor:middle;}#mermaid-svg-H89C6SlICFWalEgO .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-H89C6SlICFWalEgO .rough-node .label,#mermaid-svg-H89C6SlICFWalEgO .node .label,#mermaid-svg-H89C6SlICFWalEgO .image-shape .label,#mermaid-svg-H89C6SlICFWalEgO .icon-shape .label{text-align:center;}#mermaid-svg-H89C6SlICFWalEgO .node.clickable{cursor:pointer;}#mermaid-svg-H89C6SlICFWalEgO .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-H89C6SlICFWalEgO .arrowheadPath{fill:#333333;}#mermaid-svg-H89C6SlICFWalEgO .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-H89C6SlICFWalEgO .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-H89C6SlICFWalEgO .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-H89C6SlICFWalEgO .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-H89C6SlICFWalEgO .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-H89C6SlICFWalEgO .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-H89C6SlICFWalEgO .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-H89C6SlICFWalEgO .cluster text{fill:#333;}#mermaid-svg-H89C6SlICFWalEgO .cluster span{color:#333;}#mermaid-svg-H89C6SlICFWalEgO div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-H89C6SlICFWalEgO .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-H89C6SlICFWalEgO rect.text{fill:none;stroke-width:0;}#mermaid-svg-H89C6SlICFWalEgO .icon-shape,#mermaid-svg-H89C6SlICFWalEgO .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-H89C6SlICFWalEgO .icon-shape p,#mermaid-svg-H89C6SlICFWalEgO .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-H89C6SlICFWalEgO .icon-shape rect,#mermaid-svg-H89C6SlICFWalEgO .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-H89C6SlICFWalEgO .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-H89C6SlICFWalEgO .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-H89C6SlICFWalEgO :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

预训练阶段

海量互联网文本

学习语言模式

获得世界知识

理解物理、历史、数学

文档补全器

在这个阶段，模型学会了：

• 语言的统计规律
• 世界的基本知识
• 逻辑推理能力
• 但它只是一个"文档补全器"

阶段二：后训练（成为助手）

#mermaid-svg-NeJQEjPN7b3jZw64{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-NeJQEjPN7b3jZw64 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-NeJQEjPN7b3jZw64 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-NeJQEjPN7b3jZw64 .error-icon{fill:#552222;}#mermaid-svg-NeJQEjPN7b3jZw64 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-NeJQEjPN7b3jZw64 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-NeJQEjPN7b3jZw64 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-NeJQEjPN7b3jZw64 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-NeJQEjPN7b3jZw64 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-NeJQEjPN7b3jZw64 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-NeJQEjPN7b3jZw64 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-NeJQEjPN7b3jZw64 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-NeJQEjPN7b3jZw64 .marker.cross{stroke:#333333;}#mermaid-svg-NeJQEjPN7b3jZw64 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-NeJQEjPN7b3jZw64 p{margin:0;}#mermaid-svg-NeJQEjPN7b3jZw64 .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-NeJQEjPN7b3jZw64 .cluster-label text{fill:#333;}#mermaid-svg-NeJQEjPN7b3jZw64 .cluster-label span{color:#333;}#mermaid-svg-NeJQEjPN7b3jZw64 .cluster-label span p{background-color:transparent;}#mermaid-svg-NeJQEjPN7b3jZw64 .label text,#mermaid-svg-NeJQEjPN7b3jZw64 span{fill:#333;color:#333;}#mermaid-svg-NeJQEjPN7b3jZw64 .node rect,#mermaid-svg-NeJQEjPN7b3jZw64 .node circle,#mermaid-svg-NeJQEjPN7b3jZw64 .node ellipse,#mermaid-svg-NeJQEjPN7b3jZw64 .node polygon,#mermaid-svg-NeJQEjPN7b3jZw64 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-NeJQEjPN7b3jZw64 .rough-node .label text,#mermaid-svg-NeJQEjPN7b3jZw64 .node .label text,#mermaid-svg-NeJQEjPN7b3jZw64 .image-shape .label,#mermaid-svg-NeJQEjPN7b3jZw64 .icon-shape .label{text-anchor:middle;}#mermaid-svg-NeJQEjPN7b3jZw64 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-NeJQEjPN7b3jZw64 .rough-node .label,#mermaid-svg-NeJQEjPN7b3jZw64 .node .label,#mermaid-svg-NeJQEjPN7b3jZw64 .image-shape .label,#mermaid-svg-NeJQEjPN7b3jZw64 .icon-shape .label{text-align:center;}#mermaid-svg-NeJQEjPN7b3jZw64 .node.clickable{cursor:pointer;}#mermaid-svg-NeJQEjPN7b3jZw64 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-NeJQEjPN7b3jZw64 .arrowheadPath{fill:#333333;}#mermaid-svg-NeJQEjPN7b3jZw64 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-NeJQEjPN7b3jZw64 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-NeJQEjPN7b3jZw64 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NeJQEjPN7b3jZw64 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-NeJQEjPN7b3jZw64 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NeJQEjPN7b3jZw64 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-NeJQEjPN7b3jZw64 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-NeJQEjPN7b3jZw64 .cluster text{fill:#333;}#mermaid-svg-NeJQEjPN7b3jZw64 .cluster span{color:#333;}#mermaid-svg-NeJQEjPN7b3jZw64 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-NeJQEjPN7b3jZw64 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-NeJQEjPN7b3jZw64 rect.text{fill:none;stroke-width:0;}#mermaid-svg-NeJQEjPN7b3jZw64 .icon-shape,#mermaid-svg-NeJQEjPN7b3jZw64 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-NeJQEjPN7b3jZw64 .icon-shape p,#mermaid-svg-NeJQEjPN7b3jZw64 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-NeJQEjPN7b3jZw64 .icon-shape rect,#mermaid-svg-NeJQEjPN7b3jZw64 .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-NeJQEjPN7b3jZw64 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-NeJQEjPN7b3jZw64 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-NeJQEjPN7b3jZw64 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

后训练阶段

文档补全器

监督微调SFT

人类反馈RLHF

价值观对齐

有用的AI助手

通过人类反馈强化学习，模型从一个"补全器"变成了一个"助手"：

• 学会遵循指令
• 拒绝有害请求
• 承认不确定性
• 提供有帮助的回答

🧩 “锯齿状智能”：AI的独特特征

Karpathy提出了一个深刻的概念：锯齿状智能。

#mermaid-svg-8IvIsr2JJQ5xcbfw{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-8IvIsr2JJQ5xcbfw .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-8IvIsr2JJQ5xcbfw .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-8IvIsr2JJQ5xcbfw .error-icon{fill:#552222;}#mermaid-svg-8IvIsr2JJQ5xcbfw .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-8IvIsr2JJQ5xcbfw .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-8IvIsr2JJQ5xcbfw .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-8IvIsr2JJQ5xcbfw .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-8IvIsr2JJQ5xcbfw .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-8IvIsr2JJQ5xcbfw .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-8IvIsr2JJQ5xcbfw .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-8IvIsr2JJQ5xcbfw .marker{fill:#333333;stroke:#333333;}#mermaid-svg-8IvIsr2JJQ5xcbfw .marker.cross{stroke:#333333;}#mermaid-svg-8IvIsr2JJQ5xcbfw svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-8IvIsr2JJQ5xcbfw p{margin:0;}#mermaid-svg-8IvIsr2JJQ5xcbfw .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-8IvIsr2JJQ5xcbfw .cluster-label text{fill:#333;}#mermaid-svg-8IvIsr2JJQ5xcbfw .cluster-label span{color:#333;}#mermaid-svg-8IvIsr2JJQ5xcbfw .cluster-label span p{background-color:transparent;}#mermaid-svg-8IvIsr2JJQ5xcbfw .label text,#mermaid-svg-8IvIsr2JJQ5xcbfw span{fill:#333;color:#333;}#mermaid-svg-8IvIsr2JJQ5xcbfw .node rect,#mermaid-svg-8IvIsr2JJQ5xcbfw .node circle,#mermaid-svg-8IvIsr2JJQ5xcbfw .node ellipse,#mermaid-svg-8IvIsr2JJQ5xcbfw .node polygon,#mermaid-svg-8IvIsr2JJQ5xcbfw .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-8IvIsr2JJQ5xcbfw .rough-node .label text,#mermaid-svg-8IvIsr2JJQ5xcbfw .node .label text,#mermaid-svg-8IvIsr2JJQ5xcbfw .image-shape .label,#mermaid-svg-8IvIsr2JJQ5xcbfw .icon-shape .label{text-anchor:middle;}#mermaid-svg-8IvIsr2JJQ5xcbfw .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-8IvIsr2JJQ5xcbfw .rough-node .label,#mermaid-svg-8IvIsr2JJQ5xcbfw .node .label,#mermaid-svg-8IvIsr2JJQ5xcbfw .image-shape .label,#mermaid-svg-8IvIsr2JJQ5xcbfw .icon-shape .label{text-align:center;}#mermaid-svg-8IvIsr2JJQ5xcbfw .node.clickable{cursor:pointer;}#mermaid-svg-8IvIsr2JJQ5xcbfw .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-8IvIsr2JJQ5xcbfw .arrowheadPath{fill:#333333;}#mermaid-svg-8IvIsr2JJQ5xcbfw .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-8IvIsr2JJQ5xcbfw .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-8IvIsr2JJQ5xcbfw .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-8IvIsr2JJQ5xcbfw .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-8IvIsr2JJQ5xcbfw .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-8IvIsr2JJQ5xcbfw .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-8IvIsr2JJQ5xcbfw .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-8IvIsr2JJQ5xcbfw .cluster text{fill:#333;}#mermaid-svg-8IvIsr2JJQ5xcbfw .cluster span{color:#333;}#mermaid-svg-8IvIsr2JJQ5xcbfw div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-8IvIsr2JJQ5xcbfw .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-8IvIsr2JJQ5xcbfw rect.text{fill:none;stroke-width:0;}#mermaid-svg-8IvIsr2JJQ5xcbfw .icon-shape,#mermaid-svg-8IvIsr2JJQ5xcbfw .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-8IvIsr2JJQ5xcbfw .icon-shape p,#mermaid-svg-8IvIsr2JJQ5xcbfw .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-8IvIsr2JJQ5xcbfw .icon-shape rect,#mermaid-svg-8IvIsr2JJQ5xcbfw .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-8IvIsr2JJQ5xcbfw .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-8IvIsr2JJQ5xcbfw .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-8IvIsr2JJQ5xcbfw :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

AI能力分布

优秀

优秀

优秀

优秀

困难

困难

写诗歌

高级推理

数学证明

代码生成

简单拼写

字符计数

能力水平

能力水平

AI在某些高难度任务上表现出色，但在看似简单的任务上却会失败。这不是缺陷，而是其架构的必然结果。

💭 AI如何"思考"：Token就是思维

一个关键洞察：模型需要Token来思考。

思考过程

AI模型

用户

思考过程

AI模型

用户

#mermaid-svg-9iUlFziniFmb1bmD{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-9iUlFziniFmb1bmD .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-9iUlFziniFmb1bmD .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-9iUlFziniFmb1bmD .error-icon{fill:#552222;}#mermaid-svg-9iUlFziniFmb1bmD .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-9iUlFziniFmb1bmD .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-9iUlFziniFmb1bmD .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-9iUlFziniFmb1bmD .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-9iUlFziniFmb1bmD .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-9iUlFziniFmb1bmD .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-9iUlFziniFmb1bmD .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-9iUlFziniFmb1bmD .marker{fill:#333333;stroke:#333333;}#mermaid-svg-9iUlFziniFmb1bmD .marker.cross{stroke:#333333;}#mermaid-svg-9iUlFziniFmb1bmD svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-9iUlFziniFmb1bmD p{margin:0;}#mermaid-svg-9iUlFziniFmb1bmD .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-9iUlFziniFmb1bmD text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-9iUlFziniFmb1bmD .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-9iUlFziniFmb1bmD .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-9iUlFziniFmb1bmD .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-9iUlFziniFmb1bmD .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-9iUlFziniFmb1bmD #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-9iUlFziniFmb1bmD .sequenceNumber{fill:white;}#mermaid-svg-9iUlFziniFmb1bmD #sequencenumber{fill:#333;}#mermaid-svg-9iUlFziniFmb1bmD #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-9iUlFziniFmb1bmD .messageText{fill:#333;stroke:none;}#mermaid-svg-9iUlFziniFmb1bmD .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-9iUlFziniFmb1bmD .labelText,#mermaid-svg-9iUlFziniFmb1bmD .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-9iUlFziniFmb1bmD .loopText,#mermaid-svg-9iUlFziniFmb1bmD .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-9iUlFziniFmb1bmD .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-9iUlFziniFmb1bmD .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-9iUlFziniFmb1bmD .noteText,#mermaid-svg-9iUlFziniFmb1bmD .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-9iUlFziniFmb1bmD .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-9iUlFziniFmb1bmD .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-9iUlFziniFmb1bmD .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-9iUlFziniFmb1bmD .actorPopupMenu{position:absolute;}#mermaid-svg-9iUlFziniFmb1bmD .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-9iUlFziniFmb1bmD .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-9iUlFziniFmb1bmD .actor-man circle,#mermaid-svg-9iUlFziniFmb1bmD line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-9iUlFziniFmb1bmD :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

更多Token = 更深思考

复杂问题

生成中间Token

逐步推理

完整思路

最终答案

就像人类需要时间深度思考一样，给AI更多的"思考空间"（更多中间Token），它就能解决更复杂的问题。这就是DeepSeek-R1等推理模型的核心原理。

⚠️ 幻觉：不是Bug，是Feature

人们经常抱怨AI的"幻觉"问题，但Karpathy指出：这不是缺陷，而是设计的必然结果。

#mermaid-svg-57VJ6QPr5hG2qH1s{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-57VJ6QPr5hG2qH1s .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-57VJ6QPr5hG2qH1s .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-57VJ6QPr5hG2qH1s .error-icon{fill:#552222;}#mermaid-svg-57VJ6QPr5hG2qH1s .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-57VJ6QPr5hG2qH1s .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-57VJ6QPr5hG2qH1s .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-57VJ6QPr5hG2qH1s .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-57VJ6QPr5hG2qH1s .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-57VJ6QPr5hG2qH1s .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-57VJ6QPr5hG2qH1s .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-57VJ6QPr5hG2qH1s .marker{fill:#333333;stroke:#333333;}#mermaid-svg-57VJ6QPr5hG2qH1s .marker.cross{stroke:#333333;}#mermaid-svg-57VJ6QPr5hG2qH1s svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-57VJ6QPr5hG2qH1s p{margin:0;}#mermaid-svg-57VJ6QPr5hG2qH1s .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-57VJ6QPr5hG2qH1s .cluster-label text{fill:#333;}#mermaid-svg-57VJ6QPr5hG2qH1s .cluster-label span{color:#333;}#mermaid-svg-57VJ6QPr5hG2qH1s .cluster-label span p{background-color:transparent;}#mermaid-svg-57VJ6QPr5hG2qH1s .label text,#mermaid-svg-57VJ6QPr5hG2qH1s span{fill:#333;color:#333;}#mermaid-svg-57VJ6QPr5hG2qH1s .node rect,#mermaid-svg-57VJ6QPr5hG2qH1s .node circle,#mermaid-svg-57VJ6QPr5hG2qH1s .node ellipse,#mermaid-svg-57VJ6QPr5hG2qH1s .node polygon,#mermaid-svg-57VJ6QPr5hG2qH1s .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-57VJ6QPr5hG2qH1s .rough-node .label text,#mermaid-svg-57VJ6QPr5hG2qH1s .node .label text,#mermaid-svg-57VJ6QPr5hG2qH1s .image-shape .label,#mermaid-svg-57VJ6QPr5hG2qH1s .icon-shape .label{text-anchor:middle;}#mermaid-svg-57VJ6QPr5hG2qH1s .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-57VJ6QPr5hG2qH1s .rough-node .label,#mermaid-svg-57VJ6QPr5hG2qH1s .node .label,#mermaid-svg-57VJ6QPr5hG2qH1s .image-shape .label,#mermaid-svg-57VJ6QPr5hG2qH1s .icon-shape .label{text-align:center;}#mermaid-svg-57VJ6QPr5hG2qH1s .node.clickable{cursor:pointer;}#mermaid-svg-57VJ6QPr5hG2qH1s .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-57VJ6QPr5hG2qH1s .arrowheadPath{fill:#333333;}#mermaid-svg-57VJ6QPr5hG2qH1s .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-57VJ6QPr5hG2qH1s .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-57VJ6QPr5hG2qH1s .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-57VJ6QPr5hG2qH1s .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-57VJ6QPr5hG2qH1s .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-57VJ6QPr5hG2qH1s .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-57VJ6QPr5hG2qH1s .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-57VJ6QPr5hG2qH1s .cluster text{fill:#333;}#mermaid-svg-57VJ6QPr5hG2qH1s .cluster span{color:#333;}#mermaid-svg-57VJ6QPr5hG2qH1s div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-57VJ6QPr5hG2qH1s .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-57VJ6QPr5hG2qH1s rect.text{fill:none;stroke-width:0;}#mermaid-svg-57VJ6QPr5hG2qH1s .icon-shape,#mermaid-svg-57VJ6QPr5hG2qH1s .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-57VJ6QPr5hG2qH1s .icon-shape p,#mermaid-svg-57VJ6QPr5hG2qH1s .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-57VJ6QPr5hG2qH1s .icon-shape rect,#mermaid-svg-57VJ6QPr5hG2qH1s .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-57VJ6QPr5hG2qH1s .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-57VJ6QPr5hG2qH1s .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-57VJ6QPr5hG2qH1s :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

是

否

用户问题

训练数据中存在?

基于数据回答

生成最可能的回答

看起来合理但可能错误

AI的目标是预测下一个最可能的Token，而不是提供绝对真实的信息。解决方案不是消除幻觉，而是：

• 集成外部工具（搜索、计算器）
• 建立验证机制
• 提高透明度

🔮 未来展望：智能的工业化

观看这个视频后，我意识到我们正在见证智能的工业化：

#mermaid-svg-QCoVE7JP4HXIpgdx{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-QCoVE7JP4HXIpgdx .error-icon{fill:#552222;}#mermaid-svg-QCoVE7JP4HXIpgdx .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-QCoVE7JP4HXIpgdx .marker{fill:#333333;stroke:#333333;}#mermaid-svg-QCoVE7JP4HXIpgdx .marker.cross{stroke:#333333;}#mermaid-svg-QCoVE7JP4HXIpgdx svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-QCoVE7JP4HXIpgdx p{margin:0;}#mermaid-svg-QCoVE7JP4HXIpgdx .edge{stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .section–1 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section–1 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section–1 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section–1 path{fill:hsl(240, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section–1 text{fill:#ffffff;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon–1{font-size:40px;color:#ffffff;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge–1{stroke:hsl(240, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth–1{stroke-width:17;}#mermaid-svg-QCoVE7JP4HXIpgdx .section–1 line{stroke:hsl(60, 100%, 86.2745098039%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:#ffffff;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-0 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-0 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-0 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section-0 path{fill:hsl(60, 100%, 73.5294117647%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-0 text{fill:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon-0{font-size:40px;color:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge-0{stroke:hsl(60, 100%, 73.5294117647%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth-0{stroke-width:14;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-0 line{stroke:hsl(240, 100%, 83.5294117647%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-1 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-1 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-1 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section-1 path{fill:hsl(80, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-1 text{fill:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon-1{font-size:40px;color:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge-1{stroke:hsl(80, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth-1{stroke-width:11;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-1 line{stroke:hsl(260, 100%, 86.2745098039%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-2 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-2 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-2 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section-2 path{fill:hsl(270, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-2 text{fill:#ffffff;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon-2{font-size:40px;color:#ffffff;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge-2{stroke:hsl(270, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth-2{stroke-width:8;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-2 line{stroke:hsl(90, 100%, 86.2745098039%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:#ffffff;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-3 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-3 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-3 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section-3 path{fill:hsl(300, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-3 text{fill:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon-3{font-size:40px;color:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge-3{stroke:hsl(300, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth-3{stroke-width:5;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-3 line{stroke:hsl(120, 100%, 86.2745098039%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-4 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-4 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-4 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section-4 path{fill:hsl(330, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-4 text{fill:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon-4{font-size:40px;color:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge-4{stroke:hsl(330, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth-4{stroke-width:2;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-4 line{stroke:hsl(150, 100%, 86.2745098039%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-5 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-5 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-5 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section-5 path{fill:hsl(0, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-5 text{fill:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon-5{font-size:40px;color:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge-5{stroke:hsl(0, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth-5{stroke-width:-1;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-5 line{stroke:hsl(180, 100%, 86.2745098039%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-6 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-6 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-6 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section-6 path{fill:hsl(30, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-6 text{fill:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon-6{font-size:40px;color:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge-6{stroke:hsl(30, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth-6{stroke-width:-4;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-6 line{stroke:hsl(210, 100%, 86.2745098039%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-7 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-7 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-7 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section-7 path{fill:hsl(90, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-7 text{fill:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon-7{font-size:40px;color:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge-7{stroke:hsl(90, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth-7{stroke-width:-7;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-7 line{stroke:hsl(270, 100%, 86.2745098039%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-8 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-8 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-8 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section-8 path{fill:hsl(150, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-8 text{fill:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon-8{font-size:40px;color:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge-8{stroke:hsl(150, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth-8{stroke-width:-10;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-8 line{stroke:hsl(330, 100%, 86.2745098039%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-9 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-9 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-9 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section-9 path{fill:hsl(180, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-9 text{fill:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon-9{font-size:40px;color:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge-9{stroke:hsl(180, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth-9{stroke-width:-13;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-9 line{stroke:hsl(0, 100%, 86.2745098039%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-10 rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-10 path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-10 circle,#mermaid-svg-QCoVE7JP4HXIpgdx .section-10 path{fill:hsl(210, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-10 text{fill:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .node-icon-10{font-size:40px;color:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-edge-10{stroke:hsl(210, 100%, 76.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .edge-depth-10{stroke-width:-16;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-10 line{stroke:hsl(30, 100%, 86.2745098039%);stroke-width:3;}#mermaid-svg-QCoVE7JP4HXIpgdx .lineWrapper line{stroke:black;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled circle,#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:lightgray;}#mermaid-svg-QCoVE7JP4HXIpgdx .disabled text{fill:#efefef;}#mermaid-svg-QCoVE7JP4HXIpgdx .section-root rect,#mermaid-svg-QCoVE7JP4HXIpgdx .section-root path,#mermaid-svg-QCoVE7JP4HXIpgdx .section-root circle{fill:hsl(240, 100%, 46.2745098039%);}#mermaid-svg-QCoVE7JP4HXIpgdx .section-root text{fill:#ffffff;}#mermaid-svg-QCoVE7JP4HXIpgdx .icon-container{height:100%;display:flex;justify-content:center;align-items:center;}#mermaid-svg-QCoVE7JP4HXIpgdx .edge{fill:none;}#mermaid-svg-QCoVE7JP4HXIpgdx .eventWrapper{filter:brightness(120%);}#mermaid-svg-QCoVE7JP4HXIpgdx :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

2020s初

GPT-3突破

大模型崛起

2020s中

ChatGPT现象

AI助手普及

2020s末

推理模型

工具集成

2030s

AGI实现

智能商品化

AI发展时间线

就像电力从稀缺资源变成基础设施一样，智能正在从人类独有能力变成可按需调用的服务。

🌟 开源的力量

视频中展示的所有工具——从llm.c到HuggingFace——都是开源的。这证明了开放协作推动最重要技术突破的理念。

竞争推动创新，但开源确保了创新的民主化。

💡 关键启示

Karpathy的这次深度剖析不仅仅是技术讲解，更是对人类智能本质的深刻思考。我们不仅在构建工具，更在创造新的智能形式。

这3个小时可能是你今年最重要的投资。理解AI不再是选择，而是在智能时代生存的必需品。

我们正站在人类文明史上最重要的转折点。让我们确保我们创造的是一个值得生活的未来。

如果你还没看过这个视频，强烈建议立即观看。在AI重塑世界的过程中，理解比使用更重要。

🔗 视频链接: Deep Dive into LLMs like ChatGPT