第8章 Voice replication(声音复刻)
$p
Gemini 4 Argon 现已发布,立即试用。 $p Voice replication 让你可以通过简短的音频样本,利用 Gemini API 的 Voices 端点(POST /v1beta/voices)复刻说话人的声音特征。Gemini 3.8 Flash TTS(gemini-3.8-flash-tts)和 Gemini 3.8 Flash-Lite TTS(gemini-3.8-flash-lite-tts)均支持此功能。 $p 在 Google AI Studio 中体验交互式 Voice Replication 是快速复刻、验证授权并试听复刻声音的最佳方式。你可以直接在浏览器中录制或上传参考音频和授权音频,预览声音,并将生成的 voice_... ID 复制到应用代码中。 $p 在 Google AI Studio 中试用 $p 有状态与无状态存储模式 $p 调用 voices.create(POST /v1beta/voices)时,Voice replication 支持两种存储模式,默认启用有状态存储: $p 有状态存储(store=True,推荐默认设置):Google 在你的项目中存储经过验证的声音配置文件,并返回一个轻量级的持久化 voice_id(replicated_voice.id,如 voice_abc123...)。你可以跨请求传递此 voice_id,并通过 voices.list()、voices.get() 和 voices.delete() 进行管理。 无状态客户端托管密钥(store=False,可选):对于要求生物特征声音配置文件零服务端持久化的工作负载,请设置 store=False。API 将返回一个加密的、自包含的 voice_key(replicated_voice.key,以 voicekey_... 开头),你的应用程序需本地存储该密钥,并在合成请求中直接传入。 $p 存储模式 标识符 项目限制 保留期(TTL) 有状态声音(store=True) voice_... 每个项目 200 个声音(在提示生成和复刻声音之间共享) 1 年 无状态声音密钥(store=False) voicekey_... 客户端托管 7 天 $p 音频与授权要求 $p 每个 CreateVoice 复刻请求都需要同一位成年说话人(建议 24kHz 单声道 16-bit WAV)的两段真实人声录音: $p 参考音频(source_audio):10–30 秒清晰、自然的说话人语音片段,用于复刻该说话人的声音。 授权音频(consent_audio):同一说话人清晰朗读强制性授权声明的录音,使用受支持的语言之一(例如英语): > “我是此声音的所有者,我同意 Google 使用此声音创建合成语音模型。” $p 创建复刻声音(默认有状态) $p 使用 Google GenAI SDK(google-genai 2.25.0+ / @google/genai 2.24.0+)或 REST API 并设置 store=True,可在你的项目中创建并保存复刻声音配置文件: $p Python import base64 from google import genaiclient = genai.Client()
with open("reference_speaker.wav", "rb") as f: source_b64 = base64.b64encode(f.read()).decode("utf-8")
with open("speaker_consent.wav", "rb") as f: consent_b64 = base64.b64encode(f.read()).decode("utf-8")
# 创建持久化复刻声音 (store=True) replicated_voice = client.voices.create( store=True, voice={ "model": "gemini-3.8-flash-tts", "type": "replicated", "display_name": "Custom Replicated Speaker", "replicated": { "source_audio": { "mime_type": "audio/wav", "data": source_b64, }, "consent_audio": { "mime_type": "audio/wav", "data": consent_b64, }, }, }, )
print(f"Created voice ID: {replicated_voice.id}")
JavaScript import * as fs from "node:fs"; import { GoogleGenAI } from "@google/genai";
const ai = new GoogleGenAI();
const sourceB64 = fs.readFileSync("reference_speaker.wav").toString("base64"); const consentB64 = fs.readFileSync("speaker_consent.wav").toString("base64");
// 创建持久化复刻声音 (store: true) const replicatedVoice = await ai.voices.create({ store: true, voice: { model: "gemini-3.8-flash-tts", type: "replicated", display_name: "Custom Replicated Speaker", replicated: { source_audio: { mime_type: "audio/wav", data: sourceB64, }, consent_audio: { mime_type: "audio/wav", data: consentB64, }, }, }, });
console.log(`Created voice ID: ${replicatedVoice.id}`);
REST SOURCE_B64=$(base64 -w 0 reference_speaker.wav) CONSENT_B64=$(base64 -w 0 speaker_consent.wav)
curl "https://generativelanguage.googleapis.com/v1beta/voices" \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -X POST \ -d "{ \"store\": true, \"voice\": { \"model\": \"gemini-3.8-flash-tts\", \"type\": \"replicated\", \"display_name\": \"Custom Replicated Speaker\", \"replicated\": { \"source_audio\": { \"mime_type\": \"audio/wav\", \"data\": \"$SOURCE_B64\" }, \"consent_audio\": { \"mime_type\": \"audio/wav\", \"data\": \"$CONSENT_B64\" } } } }" $p
使用复刻声音合成语音 $p 在合成请求中传入返回的 id(voice_...): $p Python import base64 from google import genaiclient = genai.Client()
interaction = client.interactions.create( model="gemini-3.8-flash-tts", input=[{ "type": "user_input", "content": [{ "type": "text", "text": ( "Hello! This audio was synthesized using a replicated" " speaker voice." ), "annotations": [{ "type": "speech_metadata", "style": "warm and conversational", }], }], }], response_format={"type": "audio"}, generation_config={ "speech_config": [ {"voice": replicated_voice.id}, ] }, )
with open("replicated_speech.wav", "wb") as f: f.write(base64.b64decode(interaction.output_audio.data))
JavaScript import * as fs from "node:fs"; import { GoogleGenAI } from "@google/genai";
const ai = new GoogleGenAI();
const interaction = await ai.interactions.create({ model: "gemini-3.8-flash-tts", input: [{ type: "user_input", content: [{ type: "text", text: "Hello! This audio was synthesized using a replicated speaker voice.", annotations: [{ type: "speech_metadata", style: "warm and conversational", }], }], }], response_format: { type: "audio" }, generation_config: { speech_config: [ { voice: replicatedVoice.id }, ], }, });
fs.writeFileSync("replicated_speech.wav", Buffer.from(interaction.output_audio.data, "base64"));
REST curl "https://generativelanguage.googleapis.com/v1beta/interactions" \ -H "x-goog-api-key: $GEMINI_API_KEY" \ -H "Content-Type: application/json" \ -X POST \ -d '{ "model": "gemini-3.8-flash-tts", "input": [{ "type": "user_input", "content": [{ "type": "text", "text": "Hello! This audio was synthesized using a replicated speaker voice.", "annotations": [{ "type": "speech_metadata", "style": "warm and conversational" }] }] }], "response_format": {"type": "audio"}, "generation_config": { "speech_config": [ {"voice": "voice_YOUR_REPLICATED_VOICE_ID"} ] } }' | jq -r '[.steps[] | select(.type=="model_output") | .content[] | select(.type=="audio")] | last | .data' | base64 --decode > out.wav $p
管理已存储的复刻声音 $p 以 store=True 创建时,你可以通