语音克隆是怎么做的How Voice Cloning Works
How Voice Cloning Works
几秒录音就能合成某人声音：原理是「声纹向量」与「读文本」分开训练。A few seconds of recording is enough to synthesise someone's voice: the principle is training 'voiceprint vector' and 'reading text' separately.
说话人编码器先把几秒参考录音压成一个固定长度的声纹向量。
语音合成模型读文本时以这个向量为条件，输出带该音色的语音。
2018 年 Google 的做法确立了这条路：编码器与合成器分开训练，互不拖累。
YourTTS（2021）把它做成开源零样本。几秒录音即可克隆，不需要微调，还能跨语言。
这是滥用风险最高的生成能力之一。中国《生成式 AI 服务管理暂行办法》第十二条要求生成内容按深度合成规定加标识。
A speaker encoder first compresses a few seconds of reference audio into a fixed-length voice embedding.
The synthesis model conditions on that vector while reading text, producing audio in that timbre.
Google's 2018 approach established the path: encoder and synthesizer are trained independently, so neither drags the other down.
YourTTS (2021) made it open-source and zero-shot: no fine-tuning, a few seconds of audio to clone, and it works across languages.
This is among the highest-abuse-risk generation capabilities. Article 12 of China's Interim Measures for Generative AI Services requires generated content to carry deep-synthesis labelling.
接到「像某某人的声音」的需求时要知道三件事。技术上很容易。合规上有标识义务，伦理上需要本人同意。When someone asks for 'a voice like so-and-so', you need to know it is technically easy, carries a labelling obligation under regulation, and ethically requires that person's consent.
回到名词地图看第4站的其他 35 个词 →Back to the map: the other 35 terms in station 4 →
- Jia 等《Transfer Learning from Speaker Verification to Multispeaker TTS》（NeurIPS 2018，arXiv）Transfer Learning from Speaker Verification to Multispeaker TTS (NeurIPS 2018) ↗
- YourTTS 论文（零样本多说话人 TTS，arXiv）YourTTS: Towards Zero-Shot Multi-Speaker TTS (arXiv 2112.02418) ↗
- 《生成式人工智能服务管理暂行办法》全文（国家网信办）Interim Measures for Generative AI Services (CAC, full text) ↗