ConversationalVoice

Several available models perform speaker separation, but they remain prone to characteristic speaker-assignment errors: speech from one speaker may be split across multiple tracks, while speech from multiple speakers may be assigned to the same track. Either failure compromises training data for full-duplex speech models, which depend on consistent speaker identity, clean signals, and accurate time alignment.

目前已有多种模型能够进行说话人语音分离，但仍容易出现典型的说话人归属错误：同一说话人的语音可能被分散到多条音轨中，而不同说话人的语音也可能被分配到同一条音轨。全双工语音模型依赖一致的说话人身份、干净的信号和准确的时间对齐，因此上述任一错误都会损害训练数据的质量。

Following an evaluation of several candidate models, we developed an end-to-end separation workflow. VAD first partitions hour-scale recordings at natural boundaries into segments of roughly 15 minutes. In our evaluation, applying Diarizen to these shorter inputs improved speaker attribution. After removing music and low-SNR regions, a greedy dynamic-window search selects the longest eligible segments that contain exactly two speakers, sustain reciprocal interaction without extended monologues, and provide sufficient speech from each participant for reliable DialogueSidon separation. Speaker-consistency checks then identify identity splits and merges. Segments that fail the quality gate are either reprocessed or excluded.

在评估多种候选模型后，我们开发了一套端到端的语音分离流程。首先，语音活动检测（VAD）沿自然边界将小时级录音切分为约 15 分钟的片段。在我们的评估中，将 Diarizen 应用于这些较短的输入，改善了说话人归属的准确性。去除音乐和低信噪比区域后，贪心式动态窗口搜索会选取满足条件的最长片段：仅包含两位说话人，双方持续互动且没有长时间的独白，并且每位参与者都有足够的语音，供 DialogueSidon 可靠地完成分离。随后，说话人一致性检查会识别同一说话人被拆分或不同说话人被合并的情况。未通过质量检查的片段会被重新处理或剔除。

Reconstruction synthesizes clean, high-fidelity dialogue from separated speaker tracks while preserving the expressive character of the original performance. A multimodal model analyzes each utterance and generates audio tags that describe its prosody, emotional tone, and other vocal cues. For TTS, the reference audio is formed by concatenating two complementary sources: a fixed, clean speaker sample that anchors speaker identity and the corresponding utterance from the separated track, which provides utterance-specific guidance for delivery and paralinguistic expression. The reconstructed utterances are then reassembled to preserve the source conversation’s turn order, pauses, and overlap, with timing adjusted to accommodate their synthesized durations while maintaining natural conversational pacing.

重建阶段以分离后的说话人音轨为基础，合成干净、高保真的对话，同时保留原始演绎的表达特点。多模态模型进行逐句分析，生成描述其韵律、情感基调和其他声音线索的音频标签。用于语音合成（TTS）的参考音频由两类互补素材拼接而成：一段固定、干净的说话人样本，用于锚定说话人身份；以及分离音轨后对应的说话人片段，为该话语的表达方式和副语言表现提供具体指导。随后，将重建的话语重新组装，保留原始对话的话轮顺序、停顿和重叠，并根据合成后的时长调整时序，以维持自然的对话节奏。

Scripts generated from prompts alone can be fluent but often lack the intent, cadence, and context of a real exchange. Template-based dialogue TTS compounds the problem by imposing regular pauses and mechanically scheduled overlap.

仅凭提示词生成的脚本虽然可以很流畅，却往往缺少真实交流中的意图、节奏和语境。基于模板的对话语音合成会引入规律化的停顿和机械安排的重叠。

ConversationalVoice uses a real exchange as its structural and contextual reference. The observed content, speaker profiles, pauses, and overlap serve as reference points, while up to five minutes of preceding dialogue provides broader context. The expansion stage can increase the volume and diversity of training data or target specific interaction patterns, including turn-taking, pause structure, overlap, and paralinguistic cues.

ConversationalVoice 以真实交流作为结构和语境的参照。实际对话内容、说话人特征、停顿和重叠都是生成时的参考依据，而此前最长五分钟的对话则提供更广泛的上下文。扩展阶段既可增加训练数据的规模和多样性，也可针对特定互动模式生成数据，包括话轮交替、停顿结构、重叠和副语言线索。

ConversationalVoice produces speaker-attributed transcripts with Audio Tags and Director Instructions alongside utterance-level and word-level timestamps, preserving the expressive and temporal details of each exchange. These annotations guide expressive, context-faithful TTS generation while providing valuable supervision labels for training.

ConversationalVoice 生成标明说话人归属的转录文本，包含音频标签（Audio Tags）、导演指令（Director Instructions），以及话语级和词级时间戳，从而保留每段交流的表达细节和时间信息。这些标注既能指导语音合成生成富有表现力、忠于语境的语音，也能为训练提供有价值的监督标签。

The pipeline is language-agnostic by design; its effective language coverage is determined by the capabilities of the component models. The default configuration currently supports eight languages: English, Chinese, German, Spanish, French, Italian, Portuguese, and Russian.

这套处理流程在设计上不依赖特定语言，实际覆盖哪些语言取决于各模型的能力。目前，默认配置支持八种语言：英语、中文、德语、西班牙语、法语、意大利语、葡萄牙语和俄语。

Extending coverage does not require changes to the overall pipeline architecture. Additional languages can be enabled, subject to model availability, by substituting compatible models for the three language-dependent components: ASR, forced alignment, and TTS.

扩展语言覆盖范围无需改变整体流程架构。在有相应模型可用的前提下，只需为自动语音识别（ASR）、强制对齐和语音合成（TTS）这三个依赖语言的组件替换兼容模型，即可支持更多语言。