webrtc源码走读（三）核心引擎层——音频引擎

三、核心引擎层——音频引擎

1. 音频引擎概述

音频引擎是WebRTC的核心模块之一，位于核心模块层（位于API层和底层实现层之间），负责处理所有音频相关的事务。它集成了先进的音频处理技术，包括编解码、回声消除、降噪、静音检测等，确保音频在实时通信中的高质量传输。

它包含以下关键组件：

• 音频编解码器：iSAC、iLBC、Opus等
• 音频处理模块：AEC（回声消除）、AGC（自动增益）、NS（降噪）、VAD（静音检测）
• 网络适应模块：NetEQ（自适应抖动控制算法和语音包丢失隐藏算法）

2. 关键源码文件及作用

音频引擎的核心代码位于webrtc/modules/audio_processing目录下，以下是关键文件及其作用：

2.1 audio_processing.h 和 audio_processing_impl.h

// webrtc/modules/audio_processing/audio_processing.h
// 音频处理模块的接口类，提供音频处理功能的公共接口
class AudioProcessing {
public:
// 初始化音频处理模块
virtual int Initialize() = 0;

// 设置回声消除
virtual int set_echo_cancellation(bool enable) = 0;

// 设置降噪
virtual int set_noise_suppression(bool enable) = 0;

// 处理音频数据
virtual int ProcessAudio(AudioFrame* audio_frame) = 0;

// 其他音频处理功能
virtual int set_volume(AutoGainControl* agc) = 0;
virtual int set_voice_detection(bool enable) = 0;
};

作用：定义音频处理的公共接口，是音频处理模块的抽象层，实现对音频数据的处理。

2.2 echo_cancellation.h 和 echo_cancellation_impl.h

// webrtc/modules/audio_processing/echo_cancellation.h
// 回声消除模块的接口类
class EchoCancellation {
public:
// 初始化回声消除
virtual int Initialize() = 0;

// 启用/禁用回声消除
virtual int enable(bool enable) = 0;

// 处理音频数据
virtual int Process(const AudioFrame* input, AudioFrame* output) = 0;

// 设置回声消除参数
virtual int set_routing_mode(EchoCancellation::RoutingMode mode) = 0;
};

作用：实现回声消除功能，通过频域分块处理，利用FftManager类进行高效的FFT变换，实时消除因扬声器声音被麦克风再次拾取而产生的回声。

2.3 noise_suppression.h 和 noise_suppression_impl.h

// webrtc/modules/audio_processing/noise_suppression.h
// 降噪模块的接口类
class NoiseSuppression {
public:
// 初始化降噪
virtual int Initialize() = 0;

// 设置降噪级别
virtual int set_level(Level level) = 0;

// 处理音频数据
virtual int Process(const AudioFrame* input, AudioFrame* output) = 0;

// 设置降噪模式
virtual int set_mode(Mode mode) = 0;
};

作用：实现降噪功能，去除背景噪声，提升语音的纯净度，包括消除嘶嘶声、风扇噪音等。

2.4 neteq.h 和 neteq_impl.h

// webrtc/modules/audio_processing/neteq.h
// 网络适应模块的接口类
class NetEq {
public:
// 初始化网络适应
virtual int Initialize() = 0;

// 添加音频数据包
virtual int InsertPacket(const uint8_t* payload, size_t payload_length,
uint32_t timestamp) = 0;

// 获取解码后的音频数据
virtual int GetAudio(AudioFrame* audio_frame) = 0;

// 处理丢包
virtual int ProcessPacketLoss() = 0;
};

作用：实现网络适应功能，包括抖动缓冲（Jitter Buffer）平滑网络抖动，丢包隐藏（PLC）以插值或预测算法生成缺失的音频数据，时间戳调整同步音频流与播放时钟。

2.5 voice_engine.h 和 voice_engine_impl.h

// webrtc/modules/voice_engine/voice_engine.h
// 语音引擎的主接口类
class VoiceEngine {
public:
// 创建语音引擎实例
static VoiceEngine* Create();

// 销毁语音引擎实例
virtual void Destroy() = 0;

// 获取音频处理模块
virtual AudioProcessing* audio_processing() = 0;

// 获取音频编解码器
virtual AudioEncoder* audio_encoder() = 0;

// 获取音频解码器
virtual AudioDecoder* audio_decoder() = 0;

// 音频数据处理
virtual int ProcessAudio(const AudioFrame* input, AudioFrame* output) = 0;
};

作用：语音引擎是音频处理的总控模块，管理音频处理的整个流程，协调音频处理、编解码和网络适应等子模块。

3. 音频引擎在整个系统中的作用

音频引擎在WebRTC架构中扮演着关键角色：

WebRTC的音频网络对抗同时体现在发送端和接收端：

• 发送端：通过Opus编解码器和QoS模块实现网络适应
• 接收端：通过NetEQ模块处理抖动、丢包和延迟

4. 与其他模块的交互

4.1 与媒体引擎（MediaEngine）的交互

核心引擎层音频引擎媒体引擎核心引擎层音频引擎媒体引擎#mermaid-svg-ejJR6lFlyVmaDhsr{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-ejJR6lFlyVmaDhsr .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-ejJR6lFlyVmaDhsr .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-ejJR6lFlyVmaDhsr .error-icon{fill:#552222;}#mermaid-svg-ejJR6lFlyVmaDhsr .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-ejJR6lFlyVmaDhsr .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-ejJR6lFlyVmaDhsr .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-ejJR6lFlyVmaDhsr .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-ejJR6lFlyVmaDhsr .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-ejJR6lFlyVmaDhsr .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-ejJR6lFlyVmaDhsr .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-ejJR6lFlyVmaDhsr .marker{fill:#333333;stroke:#333333;}#mermaid-svg-ejJR6lFlyVmaDhsr .marker.cross{stroke:#333333;}#mermaid-svg-ejJR6lFlyVmaDhsr svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-ejJR6lFlyVmaDhsr p{margin:0;}#mermaid-svg-ejJR6lFlyVmaDhsr .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-ejJR6lFlyVmaDhsr text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-ejJR6lFlyVmaDhsr .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-ejJR6lFlyVmaDhsr .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-ejJR6lFlyVmaDhsr .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-ejJR6lFlyVmaDhsr .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-ejJR6lFlyVmaDhsr #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-ejJR6lFlyVmaDhsr .sequenceNumber{fill:white;}#mermaid-svg-ejJR6lFlyVmaDhsr #sequencenumber{fill:#333;}#mermaid-svg-ejJR6lFlyVmaDhsr #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-ejJR6lFlyVmaDhsr .messageText{fill:#333;stroke:none;}#mermaid-svg-ejJR6lFlyVmaDhsr .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-ejJR6lFlyVmaDhsr .labelText,#mermaid-svg-ejJR6lFlyVmaDhsr .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-ejJR6lFlyVmaDhsr .loopText,#mermaid-svg-ejJR6lFlyVmaDhsr .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-ejJR6lFlyVmaDhsr .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-ejJR6lFlyVmaDhsr .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-ejJR6lFlyVmaDhsr .noteText,#mermaid-svg-ejJR6lFlyVmaDhsr .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-ejJR6lFlyVmaDhsr .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-ejJR6lFlyVmaDhsr .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-ejJR6lFlyVmaDhsr .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-ejJR6lFlyVmaDhsr .actorPopupMenu{position:absolute;}#mermaid-svg-ejJR6lFlyVmaDhsr .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-ejJR6lFlyVmaDhsr .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-ejJR6lFlyVmaDhsr .actor-man circle,#mermaid-svg-ejJR6lFlyVmaDhsr line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-ejJR6lFlyVmaDhsr :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}获取音频数据传递原始音频数据处理音频（AEC, NS, VAD等）返回处理后的音频数据返回处理后的音频数据

音频引擎与媒体引擎紧密交互，媒体引擎负责音频数据的采集和渲染，音频引擎负责对采集的音频数据进行处理。

4.2 与网络传输模块的交互

核心引擎层音频引擎网络传输模块核心引擎层音频引擎网络传输模块#mermaid-svg-8NdLmtukNixkkxUv{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-8NdLmtukNixkkxUv .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-8NdLmtukNixkkxUv .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-8NdLmtukNixkkxUv .error-icon{fill:#552222;}#mermaid-svg-8NdLmtukNixkkxUv .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-8NdLmtukNixkkxUv .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-8NdLmtukNixkkxUv .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-8NdLmtukNixkkxUv .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-8NdLmtukNixkkxUv .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-8NdLmtukNixkkxUv .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-8NdLmtukNixkkxUv .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-8NdLmtukNixkkxUv .marker{fill:#333333;stroke:#333333;}#mermaid-svg-8NdLmtukNixkkxUv .marker.cross{stroke:#333333;}#mermaid-svg-8NdLmtukNixkkxUv svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-8NdLmtukNixkkxUv p{margin:0;}#mermaid-svg-8NdLmtukNixkkxUv .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-8NdLmtukNixkkxUv text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-8NdLmtukNixkkxUv .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-8NdLmtukNixkkxUv .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-8NdLmtukNixkkxUv .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-8NdLmtukNixkkxUv .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-8NdLmtukNixkkxUv #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-8NdLmtukNixkkxUv .sequenceNumber{fill:white;}#mermaid-svg-8NdLmtukNixkkxUv #sequencenumber{fill:#333;}#mermaid-svg-8NdLmtukNixkkxUv #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-8NdLmtukNixkkxUv .messageText{fill:#333;stroke:none;}#mermaid-svg-8NdLmtukNixkkxUv .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-8NdLmtukNixkkxUv .labelText,#mermaid-svg-8NdLmtukNixkkxUv .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-8NdLmtukNixkkxUv .loopText,#mermaid-svg-8NdLmtukNixkkxUv .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-8NdLmtukNixkkxUv .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-8NdLmtukNixkkxUv .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-8NdLmtukNixkkxUv .noteText,#mermaid-svg-8NdLmtukNixkkxUv .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-8NdLmtukNixkkxUv .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-8NdLmtukNixkkxUv .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-8NdLmtukNixkkxUv .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-8NdLmtukNixkkxUv .actorPopupMenu{position:absolute;}#mermaid-svg-8NdLmtukNixkkxUv .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-8NdLmtukNixkkxUv .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-8NdLmtukNixkkxUv .actor-man circle,#mermaid-svg-8NdLmtukNixkkxUv line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-8NdLmtukNixkkxUv :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}编码后的音频数据发送编码后的音频数据接收编码后的音频数据传递接收的音频数据解码并处理音频数据返回处理后的音频数据

音频引擎与网络传输模块交互，将处理后的音频数据交给网络传输模块进行发送，同时从网络传输模块接收接收到的音频数据。

4.3 与ICE/STUN模块的交互

核心引擎层音频引擎ICE/STUN模块核心引擎层音频引擎ICE/STUN模块#mermaid-svg-uF0PyDjj9XYWges7{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-uF0PyDjj9XYWges7 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-uF0PyDjj9XYWges7 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-uF0PyDjj9XYWges7 .error-icon{fill:#552222;}#mermaid-svg-uF0PyDjj9XYWges7 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-uF0PyDjj9XYWges7 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-uF0PyDjj9XYWges7 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-uF0PyDjj9XYWges7 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-uF0PyDjj9XYWges7 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-uF0PyDjj9XYWges7 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-uF0PyDjj9XYWges7 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-uF0PyDjj9XYWges7 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-uF0PyDjj9XYWges7 .marker.cross{stroke:#333333;}#mermaid-svg-uF0PyDjj9XYWges7 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-uF0PyDjj9XYWges7 p{margin:0;}#mermaid-svg-uF0PyDjj9XYWges7 .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-uF0PyDjj9XYWges7 text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-uF0PyDjj9XYWges7 .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-uF0PyDjj9XYWges7 .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-uF0PyDjj9XYWges7 .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-uF0PyDjj9XYWges7 .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-uF0PyDjj9XYWges7 #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-uF0PyDjj9XYWges7 .sequenceNumber{fill:white;}#mermaid-svg-uF0PyDjj9XYWges7 #sequencenumber{fill:#333;}#mermaid-svg-uF0PyDjj9XYWges7 #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-uF0PyDjj9XYWges7 .messageText{fill:#333;stroke:none;}#mermaid-svg-uF0PyDjj9XYWges7 .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-uF0PyDjj9XYWges7 .labelText,#mermaid-svg-uF0PyDjj9XYWges7 .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-uF0PyDjj9XYWges7 .loopText,#mermaid-svg-uF0PyDjj9XYWges7 .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-uF0PyDjj9XYWges7 .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-uF0PyDjj9XYWges7 .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-uF0PyDjj9XYWges7 .noteText,#mermaid-svg-uF0PyDjj9XYWges7 .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-uF0PyDjj9XYWges7 .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-uF0PyDjj9XYWges7 .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-uF0PyDjj9XYWges7 .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-uF0PyDjj9XYWges7 .actorPopupMenu{position:absolute;}#mermaid-svg-uF0PyDjj9XYWges7 .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-uF0PyDjj9XYWges7 .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-uF0PyDjj9XYWges7 .actor-man circle,#mermaid-svg-uF0PyDjj9XYWges7 line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-uF0PyDjj9XYWges7 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}获取网络信息返回网络信息传递网络信息优化音频传输

音频引擎通过ICE/STUN获取网络信息，优化音频传输参数，确保在不同网络条件下都能获得良好的音频体验。

5. 音频引擎工作流程

5.1 音频处理流程

#mermaid-svg-xEMvjcPbyvVuvCm5{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-xEMvjcPbyvVuvCm5 .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-xEMvjcPbyvVuvCm5 .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-xEMvjcPbyvVuvCm5 .error-icon{fill:#552222;}#mermaid-svg-xEMvjcPbyvVuvCm5 .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-xEMvjcPbyvVuvCm5 .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-xEMvjcPbyvVuvCm5 .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-xEMvjcPbyvVuvCm5 .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-xEMvjcPbyvVuvCm5 .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-xEMvjcPbyvVuvCm5 .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-xEMvjcPbyvVuvCm5 .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-xEMvjcPbyvVuvCm5 .marker{fill:#333333;stroke:#333333;}#mermaid-svg-xEMvjcPbyvVuvCm5 .marker.cross{stroke:#333333;}#mermaid-svg-xEMvjcPbyvVuvCm5 svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-xEMvjcPbyvVuvCm5 p{margin:0;}#mermaid-svg-xEMvjcPbyvVuvCm5 .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-xEMvjcPbyvVuvCm5 .cluster-label text{fill:#333;}#mermaid-svg-xEMvjcPbyvVuvCm5 .cluster-label span{color:#333;}#mermaid-svg-xEMvjcPbyvVuvCm5 .cluster-label span p{background-color:transparent;}#mermaid-svg-xEMvjcPbyvVuvCm5 .label text,#mermaid-svg-xEMvjcPbyvVuvCm5 span{fill:#333;color:#333;}#mermaid-svg-xEMvjcPbyvVuvCm5 .node rect,#mermaid-svg-xEMvjcPbyvVuvCm5 .node circle,#mermaid-svg-xEMvjcPbyvVuvCm5 .node ellipse,#mermaid-svg-xEMvjcPbyvVuvCm5 .node polygon,#mermaid-svg-xEMvjcPbyvVuvCm5 .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-xEMvjcPbyvVuvCm5 .rough-node .label text,#mermaid-svg-xEMvjcPbyvVuvCm5 .node .label text,#mermaid-svg-xEMvjcPbyvVuvCm5 .image-shape .label,#mermaid-svg-xEMvjcPbyvVuvCm5 .icon-shape .label{text-anchor:middle;}#mermaid-svg-xEMvjcPbyvVuvCm5 .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-xEMvjcPbyvVuvCm5 .rough-node .label,#mermaid-svg-xEMvjcPbyvVuvCm5 .node .label,#mermaid-svg-xEMvjcPbyvVuvCm5 .image-shape .label,#mermaid-svg-xEMvjcPbyvVuvCm5 .icon-shape .label{text-align:center;}#mermaid-svg-xEMvjcPbyvVuvCm5 .node.clickable{cursor:pointer;}#mermaid-svg-xEMvjcPbyvVuvCm5 .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-xEMvjcPbyvVuvCm5 .arrowheadPath{fill:#333333;}#mermaid-svg-xEMvjcPbyvVuvCm5 .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-xEMvjcPbyvVuvCm5 .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-xEMvjcPbyvVuvCm5 .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-xEMvjcPbyvVuvCm5 .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-xEMvjcPbyvVuvCm5 .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-xEMvjcPbyvVuvCm5 .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-xEMvjcPbyvVuvCm5 .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-xEMvjcPbyvVuvCm5 .cluster text{fill:#333;}#mermaid-svg-xEMvjcPbyvVuvCm5 .cluster span{color:#333;}#mermaid-svg-xEMvjcPbyvVuvCm5 div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-xEMvjcPbyvVuvCm5 .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-xEMvjcPbyvVuvCm5 rect.text{fill:none;stroke-width:0;}#mermaid-svg-xEMvjcPbyvVuvCm5 .icon-shape,#mermaid-svg-xEMvjcPbyvVuvCm5 .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-xEMvjcPbyvVuvCm5 .icon-shape p,#mermaid-svg-xEMvjcPbyvVuvCm5 .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-xEMvjcPbyvVuvCm5 .icon-shape rect,#mermaid-svg-xEMvjcPbyvVuvCm5 .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-xEMvjcPbyvVuvCm5 .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-xEMvjcPbyvVuvCm5 .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-xEMvjcPbyvVuvCm5 :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

原始音频数据

音频处理模块

回声消除

降噪

静音检测

处理后的音频数据

音频编解码

编码后的音频数据

网络传输

5.2 音频编解码流程

#mermaid-svg-Cxglpl99eB9NJ3GQ{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-Cxglpl99eB9NJ3GQ .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-Cxglpl99eB9NJ3GQ .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-Cxglpl99eB9NJ3GQ .error-icon{fill:#552222;}#mermaid-svg-Cxglpl99eB9NJ3GQ .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-Cxglpl99eB9NJ3GQ .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-Cxglpl99eB9NJ3GQ .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-Cxglpl99eB9NJ3GQ .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-Cxglpl99eB9NJ3GQ .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-Cxglpl99eB9NJ3GQ .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-Cxglpl99eB9NJ3GQ .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-Cxglpl99eB9NJ3GQ .marker{fill:#333333;stroke:#333333;}#mermaid-svg-Cxglpl99eB9NJ3GQ .marker.cross{stroke:#333333;}#mermaid-svg-Cxglpl99eB9NJ3GQ svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-Cxglpl99eB9NJ3GQ p{margin:0;}#mermaid-svg-Cxglpl99eB9NJ3GQ .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-Cxglpl99eB9NJ3GQ .cluster-label text{fill:#333;}#mermaid-svg-Cxglpl99eB9NJ3GQ .cluster-label span{color:#333;}#mermaid-svg-Cxglpl99eB9NJ3GQ .cluster-label span p{background-color:transparent;}#mermaid-svg-Cxglpl99eB9NJ3GQ .label text,#mermaid-svg-Cxglpl99eB9NJ3GQ span{fill:#333;color:#333;}#mermaid-svg-Cxglpl99eB9NJ3GQ .node rect,#mermaid-svg-Cxglpl99eB9NJ3GQ .node circle,#mermaid-svg-Cxglpl99eB9NJ3GQ .node ellipse,#mermaid-svg-Cxglpl99eB9NJ3GQ .node polygon,#mermaid-svg-Cxglpl99eB9NJ3GQ .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-Cxglpl99eB9NJ3GQ .rough-node .label text,#mermaid-svg-Cxglpl99eB9NJ3GQ .node .label text,#mermaid-svg-Cxglpl99eB9NJ3GQ .image-shape .label,#mermaid-svg-Cxglpl99eB9NJ3GQ .icon-shape .label{text-anchor:middle;}#mermaid-svg-Cxglpl99eB9NJ3GQ .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-Cxglpl99eB9NJ3GQ .rough-node .label,#mermaid-svg-Cxglpl99eB9NJ3GQ .node .label,#mermaid-svg-Cxglpl99eB9NJ3GQ .image-shape .label,#mermaid-svg-Cxglpl99eB9NJ3GQ .icon-shape .label{text-align:center;}#mermaid-svg-Cxglpl99eB9NJ3GQ .node.clickable{cursor:pointer;}#mermaid-svg-Cxglpl99eB9NJ3GQ .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-Cxglpl99eB9NJ3GQ .arrowheadPath{fill:#333333;}#mermaid-svg-Cxglpl99eB9NJ3GQ .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-Cxglpl99eB9NJ3GQ .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-Cxglpl99eB9NJ3GQ .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Cxglpl99eB9NJ3GQ .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-Cxglpl99eB9NJ3GQ .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Cxglpl99eB9NJ3GQ .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-Cxglpl99eB9NJ3GQ .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-Cxglpl99eB9NJ3GQ .cluster text{fill:#333;}#mermaid-svg-Cxglpl99eB9NJ3GQ .cluster span{color:#333;}#mermaid-svg-Cxglpl99eB9NJ3GQ div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-Cxglpl99eB9NJ3GQ .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-Cxglpl99eB9NJ3GQ rect.text{fill:none;stroke-width:0;}#mermaid-svg-Cxglpl99eB9NJ3GQ .icon-shape,#mermaid-svg-Cxglpl99eB9NJ3GQ .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-Cxglpl99eB9NJ3GQ .icon-shape p,#mermaid-svg-Cxglpl99eB9NJ3GQ .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-Cxglpl99eB9NJ3GQ .icon-shape rect,#mermaid-svg-Cxglpl99eB9NJ3GQ .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-Cxglpl99eB9NJ3GQ .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-Cxglpl99eB9NJ3GQ .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-Cxglpl99eB9NJ3GQ :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

原始音频数据

音频编码器

编码后的音频数据

网络传输

音频解码器

解码后的音频数据

音频处理

播放的音频

5.3 音频网络适应流程

#mermaid-svg-t4Lh9OvGKlUQFlNt{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-t4Lh9OvGKlUQFlNt .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-t4Lh9OvGKlUQFlNt .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-t4Lh9OvGKlUQFlNt .error-icon{fill:#552222;}#mermaid-svg-t4Lh9OvGKlUQFlNt .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-t4Lh9OvGKlUQFlNt .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-t4Lh9OvGKlUQFlNt .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-t4Lh9OvGKlUQFlNt .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-t4Lh9OvGKlUQFlNt .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-t4Lh9OvGKlUQFlNt .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-t4Lh9OvGKlUQFlNt .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-t4Lh9OvGKlUQFlNt .marker{fill:#333333;stroke:#333333;}#mermaid-svg-t4Lh9OvGKlUQFlNt .marker.cross{stroke:#333333;}#mermaid-svg-t4Lh9OvGKlUQFlNt svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-t4Lh9OvGKlUQFlNt p{margin:0;}#mermaid-svg-t4Lh9OvGKlUQFlNt .label{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;color:#333;}#mermaid-svg-t4Lh9OvGKlUQFlNt .cluster-label text{fill:#333;}#mermaid-svg-t4Lh9OvGKlUQFlNt .cluster-label span{color:#333;}#mermaid-svg-t4Lh9OvGKlUQFlNt .cluster-label span p{background-color:transparent;}#mermaid-svg-t4Lh9OvGKlUQFlNt .label text,#mermaid-svg-t4Lh9OvGKlUQFlNt span{fill:#333;color:#333;}#mermaid-svg-t4Lh9OvGKlUQFlNt .node rect,#mermaid-svg-t4Lh9OvGKlUQFlNt .node circle,#mermaid-svg-t4Lh9OvGKlUQFlNt .node ellipse,#mermaid-svg-t4Lh9OvGKlUQFlNt .node polygon,#mermaid-svg-t4Lh9OvGKlUQFlNt .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-t4Lh9OvGKlUQFlNt .rough-node .label text,#mermaid-svg-t4Lh9OvGKlUQFlNt .node .label text,#mermaid-svg-t4Lh9OvGKlUQFlNt .image-shape .label,#mermaid-svg-t4Lh9OvGKlUQFlNt .icon-shape .label{text-anchor:middle;}#mermaid-svg-t4Lh9OvGKlUQFlNt .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-t4Lh9OvGKlUQFlNt .rough-node .label,#mermaid-svg-t4Lh9OvGKlUQFlNt .node .label,#mermaid-svg-t4Lh9OvGKlUQFlNt .image-shape .label,#mermaid-svg-t4Lh9OvGKlUQFlNt .icon-shape .label{text-align:center;}#mermaid-svg-t4Lh9OvGKlUQFlNt .node.clickable{cursor:pointer;}#mermaid-svg-t4Lh9OvGKlUQFlNt .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-t4Lh9OvGKlUQFlNt .arrowheadPath{fill:#333333;}#mermaid-svg-t4Lh9OvGKlUQFlNt .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-t4Lh9OvGKlUQFlNt .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-t4Lh9OvGKlUQFlNt .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-t4Lh9OvGKlUQFlNt .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-t4Lh9OvGKlUQFlNt .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-t4Lh9OvGKlUQFlNt .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-t4Lh9OvGKlUQFlNt .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-t4Lh9OvGKlUQFlNt .cluster text{fill:#333;}#mermaid-svg-t4Lh9OvGKlUQFlNt .cluster span{color:#333;}#mermaid-svg-t4Lh9OvGKlUQFlNt div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-t4Lh9OvGKlUQFlNt .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-t4Lh9OvGKlUQFlNt rect.text{fill:none;stroke-width:0;}#mermaid-svg-t4Lh9OvGKlUQFlNt .icon-shape,#mermaid-svg-t4Lh9OvGKlUQFlNt .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-t4Lh9OvGKlUQFlNt .icon-shape p,#mermaid-svg-t4Lh9OvGKlUQFlNt .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-t4Lh9OvGKlUQFlNt .icon-shape rect,#mermaid-svg-t4Lh9OvGKlUQFlNt .image-shape rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-t4Lh9OvGKlUQFlNt .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-t4Lh9OvGKlUQFlNt .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-t4Lh9OvGKlUQFlNt :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}

接收到的编码音频数据

网络适应模块

抖动缓冲

丢包隐藏

处理后的音频数据

音频解码器

播放的音频

6. 源码示例

6.1 音频处理流程（AudioProcessing）

// webrtc/modules/audio_processing/audio_processing_impl.cc

// 音频处理实现类
class AudioProcessingImpl : public AudioProcessing {
public:
AudioProcessingImpl() :
echo_cancellation_(EchoCancellation::Create()),
noise_suppression_(NoiseSuppression::Create()),
voice_detection_(VoiceDetection::Create()) {
// 初始化各音频处理模块
echo_cancellation_->Initialize();
noise_suppression_->Initialize();
voice_detection_->Initialize();
}

int ProcessAudio(AudioFrame* audio_frame) override {
// 1. 回声消除
echo_cancellation_->Process(audio_frame, audio_frame);

// 2. 降噪
noise_suppression_->Process(audio_frame, audio_frame);

// 3. 静音检测
voice_detection_->Process(audio_frame);

// 4. 返回处理后的音频帧
return 0;
}

private:
std::unique_ptr<EchoCancellation> echo_cancellation_;
std::unique_ptr<NoiseSuppression> noise_suppression_;
std::unique_ptr<VoiceDetection> voice_detection_;
};

• AudioProcessingImpl 是音频处理模块的实现类
• 通过组合模式管理多个音频处理子模块
• ProcessAudio() 方法处理音频数据的完整流程

6.2 音频编解码流程（VoiceEngine）

// webrtc/modules/voice_engine/voice_engine_impl.cc

// 语音引擎实现类
class VoiceEngineImpl : public VoiceEngine {
public:
VoiceEngineImpl() :
audio_processing_(AudioProcessing::Create()),
audio_encoder_(AudioEncoder::Create()),
audio_decoder_(AudioDecoder::Create()) {
// 初始化音频处理、编码和解码模块
audio_processing_->Initialize();
audio_encoder_->Initialize();
audio_decoder_->Initialize();
}

int ProcessAudio(const AudioFrame* input, AudioFrame* output) {
// 1. 处理输入音频
audio_processing_->ProcessAudio(const_cast<AudioFrame*>(input));

// 2. 编码音频
if (audio_encoder_->Encode(input, output) != 0) {
return –1;
}

// 3. 返回编码后的音频
return 0;
}

int DecodeAudio(const AudioFrame* input, AudioFrame* output) {
// 1. 解码音频
if (audio_decoder_->Decode(input, output) != 0) {
return –1;
}

// 2. 处理输出音频
audio_processing_->ProcessAudio(output);

// 3. 返回处理后的音频
return 0;
}

private:
std::unique_ptr<AudioProcessing> audio_processing_;
std::unique_ptr<AudioEncoder> audio_encoder_;
std::unique_ptr<AudioDecoder> audio_decoder_;
};

• VoiceEngineImpl 是语音引擎的实现类
• ProcessAudio() 处理音频发送流程
• DecodeAudio() 处理音频接收流程
• 通过audio_processing_统一处理音频

6.3 网络适应流程（NetEq）

// webrtc/modules/audio_processing/neteq_impl.cc

// 网络适应实现类
class NetEqImpl : public NetEq {
public:
NetEqImpl() :
jitter_buffer_(JitterBuffer::Create()),
packet_buffer_(PacketBuffer::Create()),
fec_decoder_(FecDecoder::Create()) {
// 初始化网络适应模块
jitter_buffer_->Initialize();
packet_buffer_->Initialize();
fec_decoder_->Initialize();
}

int InsertPacket(const uint8_t* payload, size_t payload_length,
uint32_t timestamp) override {
// 1. 将数据包插入缓冲区
packet_buffer_->InsertPacket(payload, payload_length, timestamp);

// 2. 处理抖动
jitter_buffer_->ProcessPacket(packet_buffer_->GetPacket());

// 3. 处理丢包
if (packet_buffer_->IsPacketMissing()) {
fec_decoder_->RecoverPacket(packet_buffer_->GetMissingPacket());
}

return 0;
}

int GetAudio(AudioFrame* audio_frame) override {
// 1. 从抖动缓冲区获取音频
if (jitter_buffer_->GetAudio(audio_frame) != 0) {
return –1;
}

// 2. 返回处理后的音频帧
return 0;
}

private:
std::unique_ptr<JitterBuffer> jitter_buffer_;
std::unique_ptr<PacketBuffer> packet_buffer_;
std::unique_ptr<FecDecoder> fec_decoder_;
};

• NetEqImpl 是网络适应模块的实现类
• InsertPacket() 处理接收到的数据包
• GetAudio() 提供处理后的音频数据
• 通过jitter_buffer_、packet_buffer_、fec_decoder_实现网络适应功能

7. 时序图：音频引擎工作流程

Network音频引擎核心引擎层API层应用层Network音频引擎核心引擎层API层应用层#mermaid-svg-Xx9O4oJMxQqy33RH{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-Xx9O4oJMxQqy33RH .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-Xx9O4oJMxQqy33RH .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-Xx9O4oJMxQqy33RH .error-icon{fill:#552222;}#mermaid-svg-Xx9O4oJMxQqy33RH .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-Xx9O4oJMxQqy33RH .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-Xx9O4oJMxQqy33RH .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-Xx9O4oJMxQqy33RH .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-Xx9O4oJMxQqy33RH .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-Xx9O4oJMxQqy33RH .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-Xx9O4oJMxQqy33RH .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-Xx9O4oJMxQqy33RH .marker{fill:#333333;stroke:#333333;}#mermaid-svg-Xx9O4oJMxQqy33RH .marker.cross{stroke:#333333;}#mermaid-svg-Xx9O4oJMxQqy33RH svg{font-family:\”trebuchet ms\”,verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-Xx9O4oJMxQqy33RH p{margin:0;}#mermaid-svg-Xx9O4oJMxQqy33RH .actor{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-Xx9O4oJMxQqy33RH text.actor>tspan{fill:black;stroke:none;}#mermaid-svg-Xx9O4oJMxQqy33RH .actor-line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-Xx9O4oJMxQqy33RH .innerArc{stroke-width:1.5;stroke-dasharray:none;}#mermaid-svg-Xx9O4oJMxQqy33RH .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333;}#mermaid-svg-Xx9O4oJMxQqy33RH .messageLine1{stroke-width:1.5;stroke-dasharray:2,2;stroke:#333;}#mermaid-svg-Xx9O4oJMxQqy33RH #arrowhead path{fill:#333;stroke:#333;}#mermaid-svg-Xx9O4oJMxQqy33RH .sequenceNumber{fill:white;}#mermaid-svg-Xx9O4oJMxQqy33RH #sequencenumber{fill:#333;}#mermaid-svg-Xx9O4oJMxQqy33RH #crosshead path{fill:#333;stroke:#333;}#mermaid-svg-Xx9O4oJMxQqy33RH .messageText{fill:#333;stroke:none;}#mermaid-svg-Xx9O4oJMxQqy33RH .labelBox{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-Xx9O4oJMxQqy33RH .labelText,#mermaid-svg-Xx9O4oJMxQqy33RH .labelText>tspan{fill:black;stroke:none;}#mermaid-svg-Xx9O4oJMxQqy33RH .loopText,#mermaid-svg-Xx9O4oJMxQqy33RH .loopText>tspan{fill:black;stroke:none;}#mermaid-svg-Xx9O4oJMxQqy33RH .loopLine{stroke-width:2px;stroke-dasharray:2,2;stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);}#mermaid-svg-Xx9O4oJMxQqy33RH .note{stroke:#aaaa33;fill:#fff5ad;}#mermaid-svg-Xx9O4oJMxQqy33RH .noteText,#mermaid-svg-Xx9O4oJMxQqy33RH .noteText>tspan{fill:black;stroke:none;}#mermaid-svg-Xx9O4oJMxQqy33RH .activation0{fill:#f4f4f4;stroke:#666;}#mermaid-svg-Xx9O4oJMxQqy33RH .activation1{fill:#f4f4f4;stroke:#666;}#mermaid-svg-Xx9O4oJMxQqy33RH .activation2{fill:#f4f4f4;stroke:#666;}#mermaid-svg-Xx9O4oJMxQqy33RH .actorPopupMenu{position:absolute;}#mermaid-svg-Xx9O4oJMxQqy33RH .actorPopupMenuPanel{position:absolute;fill:#ECECFF;box-shadow:0px 8px 16px 0px rgba(0,0,0,0.2);filter:drop-shadow(3px 5px 2px rgb(0 0 0 / 0.4));}#mermaid-svg-Xx9O4oJMxQqy33RH .actor-man line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;}#mermaid-svg-Xx9O4oJMxQqy33RH .actor-man circle,#mermaid-svg-Xx9O4oJMxQqy33RH line{stroke:hsl(259.6261682243, 59.7765363128%, 87.9019607843%);fill:#ECECFF;stroke-width:2px;}#mermaid-svg-Xx9O4oJMxQqy33RH :root{–mermaid-font-family:\”trebuchet ms\”,verdana,arial,sans-serif;}创建RTCPeerConnection初始化VoiceEngine创建音频引擎实例初始化音频处理模块初始化完成返回音频引擎实例返回音频引擎实例添加本地音频流获取音频数据处理原始音频数据回声消除降噪处理静音检测音频编码返回编码后的音频数据发送编码后的音频数据接收编码后的音频数据传递接收的音频数据音频解码处理解码后的音频返回处理后的音频数据返回处理后的音频数据处理后的音频数据播放音频