DSH Voice AI Girlfriend

"Hi there, I'm Xiao Ya. From now on, DeepSeek Harness isn't just your coding buddy — it can talk."

📌 Streaming digital human (DUIX talking-head video) is included in this mainline: replies are rendered in real time into talking-head videos (≤10s segments, TTS+video in sync, digital-human toggle, 200 videos kept).

🎥 Showcase (Douyin):

• Talking-head video demo ①
• Talking-head video demo ②

An AI girlfriend living on your PC: tap the mic and she listens, banters with you, and reads her answers out loud word by word; when you leave, she follows you to QQ to keep chatting; the girl in the side window isn't just decoration — she spaces out when idle and speaks when talking.

She's a bit feisty, but you'll likely enjoy these too:

• ⚡ Quick on her feet: after you finish a sentence, she responds within 0.5 s — FunASR Chinese recognition takes only 150 ms, TTS starts almost instantly, so she reacts faster than you
• 🎧 Selective ears: a -35 dB noise gate + silero VAD double check — fans, keyboards, and TV audio are all filtered out; she only responds to your voice
• 📱 Clingy: her replies are pushed automatically to your QQ (text + voice + images) — even when you're not at the computer, she can find you
• 👧 Animated: the digital-human window on the right spaces out when idle and speaks when talking — a breathing AI, not a cold dialog box
• 🎬 Actress: replies aren't just audio — streaming digital human (DUIX) turns your reply into a real-time talking-head video: long replies are cut into ≤10 s segments, the next segment's audio is synthesized while the current segment's video is rendering, segments play back to back continuously, with TTS and lip-sync synchronized
• 🔇 Understands interruptions: when you butt in she stops and listens; to let her finish, flip the switch back to queue mode
• 🔊 Her voice is yours: TTS voice cloning (OmniVoice, 600+ languages) — the timbre is determined by the reference audio you provide, or designed via parameters — she can sound exactly the way you like
• 🎛️ Fully tunable by you: digital-human toggle (on = synchronized video + audio / off = near-instant pure audio playback), QQ push toggle, girlfriend-window toggle, interruption/queue mode — all one-click switches in the toolbar

┌────────────────────────────────────────────┐
│  浏览器（DSH Web GUI :3080）                 │
│  ┌──────────┐  ┌─────────────────────────┐  │
│  │ 对话面板   │  │ 女友窗（bg/task 视频）   │  │
│  │ 麦克风+⚡  │  │ 数字人视频（同步播放）    │  │
│  └──────────┘  └─────────────────────────┘  │
│   麦克风采集 ──▶ STT ──▶ 代理回复 ──▶ TTS ──▶ 播放 │
│     ▲回复文本          回复文本▼            │
└─────┼──────────────────────┬──────────────┘
      │ 插件 QQ 桥 (WS)       │ HTTP (CORS)
┌─────▼──────────────────────▼──────────────┐
│  voice_bridge (:8765)                      │
│  /api/stt  FunASR 中文 ASR                 │
│  /api/tts  OmniVoice 克隆（WSL2·FlashInfer 加速）│
│  /api/dh/* DUIX 数字人（分段流水线/播放/开关）│
│  /api/qq/* QQ 桥（收发 + 语音推送）          │
│  /api/vad  silero 打断 / media 素材         │
└────┬──────────────┬───────────────────────┘
     │ OneBot HTTP+WS │ HTTP（提交音频→轮询视频）
┌────▼──────────┐ ┌──▼──────────────────────┐
│  NapCatQQ      │ │  DUIX 数字人 (:8383)      │
│  小号在线       │ │  音频 → 口型同步视频       │
│  → 文本+语音   │ │  → 女友窗同步播放          │
└───────────────┘ └──────────────────────────┘

One-Click Full-Stack Startup (Recommended · Added 2026-09-14)

After restarting your PC you only need to do two things: open Docker Desktop yourself first, then double-click bridge/start-full-stack.cmd (you can also create a desktop shortcut to it). The script brings up the whole pipeline in the correct order and finally prints a self-check table; the browser opens automatically with a tokenized link.

Step	What it does	Idempotency
1/6	Check the Docker engine and the DUIX container; if the container is down, run docker start and wait for :9000 to respond	Skipped if already running
2/6	Invoke start-omnivoice-wsl.ps1: start WSL2 OmniVoice (:9877) + sync bridge-config IP + start the bridge (:8765)	Skipped if already running
3/6	Bridge safeguard: start it if down; auto-restart to fix when DEEPSEEK_API_KEY is missing (balance badge no longer 503s)	Idempotent
4/6	Start DSH Web (default new version 0.1.3-alpha.1 / profile web-v013, :3080) and open the tokenized link	Never interrupts an existing session if 3080 is already serving
5/6	Digital-human status: toggle / container / segment progress / submits, segment_submits counters	Read-only
6/6	Summary self-check table: OK / FAIL per pipeline stage	Read-only

Parameters: -NoDsh (start the voice stack only), -Rc8 (use the old rc.8 / profile web), -NoBrowser, -WithQQ (additionally launch NapCat), -SkipDockerCheck.

powershell -NoProfile -ExecutionPolicy Bypass -File bridge\start-full-stack.ps1 -NoBrowser

Version Compatibility & Maintenance Cadence (2026-09-14)

Component	Adapted Version
DSH (primary)	dsh 0.1.3-alpha.1 / profile web-v013 (:3080, desktop script "Start New DSH-v013")
DSH (compat)	rc.8 (E:\DSH\deepseek-harness master 141eb6fef) / profile web
Plugin	@beiyege-01/dsh-voice-ai-girlfriend v0.3.0 (synced as a standalone package repo; built with node build.mjs)
Bridge	bridge/voice_bridge.py (FastAPI/uvicorn :8765, Python 3.14 venv)
TTS engine	OmniVoice (WSL2 + FlashInfer, :9877)
Digital human	DUIX guiji2025/duix.avatar-5090:trt10.9 (host :9000, cold start 1–3 min)
Docker	Docker Desktop 29.x (DUIX container toggled via "Digital Human Switch" docker start/stop)

Bridge-side additions this round (v0.3.0 companion): POST /api/dh/enable runtime toggle (off = clear queue, stop pre-warming, zero DUIX traffic, and optionally chain docker stop to free VRAM; on = docker start + pre-warm after ready), rotating log logs/bridge.log, and /api/dh/status now includes the duix container status and stats counters (submits / segment_submits / status_polls, used to verify "did the task actually get submitted").

⏳ Maintenance cadence: the author is preparing for a wedding this month; DSH adaptation and compatibility follow-up is deferred until after National Day (early Oct). Issues/PRs welcome, but responses may be slow; versions above are fully functional.

Directory Layout

dsh-voice-ai-girlfriend/
├── bridge/            # 语音桥接（独立可跑，Python/FastAPI）
│   ├── voice_bridge.py            # STT/TTS/数字人(DUIX)/QQ/VAD 全部端点
│   ├── bridge-config.example.json # 配置模板（复制为 bridge-config.json）
│   ├── requirements.txt
│   ├── start-bridge.cmd           # 只起桥接
│   └── start-all.cmd              # 桥接 + DSH Web 一键启动
├── models/            # 模型（gitignore，不入库）：funasr/ + silero-vad/
├── assets/            # 素材：内置 5 套默认待机动画（bg2/bg4/bg9/bg56/bg5），可直接用；也可自备
│   ├── bg-images/     # 空闲动画：内置 bg2/bg4/bg9/bg56/bg5 五套，自建子文件夹即可添加自定义待机
│   └── task-videos/   # 备用说话动画（可选；开了数字人后自动用 DUIX 视频）
├── voices/            # 音色库（自建）：每个子文件夹 = 一个 TTS 音色（见「自定义音色」）
├── dsh-plugin/        # DSH 客户端插件源码（mic/开关/女友窗/数字人/流式朗读）
│   └── README.md      # 安装到 DSH 的详细步骤
└── docs/              # 开发日志等

---

Install from Scratch (Beginner)

Everything is done on Windows. The commands below are assumed to be run in PowerShell; except for steps marked "run inside the project folder," you can run them from anywhere.

1. Prerequisites (one-time install)

1. Check your computer

Check	Requirement	Verify command
OS	Windows 10/11 64-bit	winver
GPU	NVIDIA discrete GPU (16GB+ VRAM recommended)	nvidia-smi (just needs to display GPU info)
Disk	At least 30GB free space	—
RAM	16GB+ recommended	—

Does nvidia-smi work without an NVIDIA GPU? No — if you don't have an NVIDIA GPU or the driver isn't properly installed, you'll get "not recognized as an internal or external command" or an error. You cannot install this project without an NVIDIA GPU (model inference relies on CUDA GPU).

VRAM usage (measured at runtime):

Mode	VRAM Usage
Recommended: OmniVoice TTS (WSL2 + FlashInfer)	~5.3GB (idle) ~ 11GB (inference peak, CUDA graph bucket cache)
Qwen3-TTS 1.7B (fp16, fallback)	~3.7GB
FunASR Paraformer-large (fp16)	~1GB
DUIX digital human (during video gen)	~4-6GB

Full pipeline (TTS + digital human working simultaneously) peaks at ~15GB measured, which basically maxes out 16GB VRAM; OmniVoice and DUIX are recommended to run staggered (finish TTS before generating video). 16GB VRAM is the recommended config; 8GB will be very tight and is not recommended.

2. Install Git

Used to clone the repo. Download and install: https://git-scm.com/download/win, just click next through the wizard.

Verify: git --version prints a version number.

3. Install Python (3.10 or higher)

Download and install: https://www.python.org/downloads/windows/

⚠️ Make sure to check "Add Python to PATH" during installation, otherwise the python command won't be found later.

Verify: open a new terminal, python --version prints a version number.

4. Update NVIDIA Drivers

Download and install the latest driver from https://www.nvidia.cn/drivers/. Outdated drivers will cause CUDA-related errors.

This project does not require a separate CUDA Toolkit installation — the PyTorch installed via pip bundles the CUDA runtime; you just need a recent enough driver.

5. Install Node.js and pnpm (for running DSH)

• Node.js: download and install from https://nodejs.org/ (choose the LTS version), just keep clicking Next.
• Verify: node --version
• pnpm (after Node.js is installed, run in your terminal):

npm install -g pnpm

• Verify: pnpm --version

6. Prepare the deepseek-harness (DSH) source code

DSH is an open-source project, and this project is a plugin for it. You need a DSH source tree first (the plugin is installed inside its source tree):

git clone https://github.com/deepseek-ai/deepseek-harness.git
cd deepseek-harness
pnpm install

✅ Compatible with DSH rc.8 and dsh 0.1.3: the plugin protocol (dsh.bundle manifest + conversation.input.dock/left slots) is implemented for rc.8, so upgrading to rc.8 requires no changes; tested and verified on dsh v0.1.3-alpha.1 on 2026-09-05 (the 0.1.3 slots / __ModuleLoader__ / session prompt contracts are unchanged, zero code changes). Recommended install (works for both rc.8 and 0.1.3): dsh plugin --profile web add github:beiyege-01/dsh-voice-ai-girlfriend-plugin. Balance statistics use the conversation.composer.dock slot. pnpm install takes anywhere from tens of seconds to a few minutes. After it finishes, just leave this folder as-is — you'll need it for the "Install the DSH voice plugin" step later. Remember its path (e.g. C:\dev\deepseek-harness); the one-click launcher will use it later.

7. Disk space estimate

Item	Size
This project's code + assets	~8MB
Python virtual environment + dependencies (incl. PyTorch)	~5-8GB
FunASR Paraformer model (models/funasr/)	~850MB
OmniVoice TTS environment (inside WSL2, incl. model + torch cu130 + FlashInfer)	~10GB (WSL2 disk)
deepseek-harness + node_modules	~2-4GB

2. Install this project

1. Clone the repository

git clone https://github.com/beiyege-01/dsh-voice-ai-girlfriend.git
cd dsh-voice-ai-girlfriend

All subsequent steps are performed inside this folder (the project root).

2. Create a Python virtual environment

python -m venv venv-speech

Activate it:

venv-speech\Scripts\activate

Once activated, (venv-speech) will appear at the beginning of the terminal prompt line.

3. Install dependencies

pip install -r bridge\requirements.txt

This step installs PyTorch, transformers, HuggingFace speech-to-speech, etc. — large in size and time-consuming (a few minutes up to tens of minutes), so please be patient. Network too slow to install? See "FAQ" item 1 at the end of this document (switch to the Tsinghua mirror).

3. Prepare the models (two models)

1. STT model: FunASR Paraformer Chinese model (placed in the local models/ directory)

Speech recognition uses Alibaba FunASR Chinese ASR (Paraformer-large, designed specifically for Chinese, with far higher accuracy on homophones and accents than whisper). The model is stored in models/funasr/ at the repo root (~850MB, gitignored, not included in the repo):

pip install modelscope
# 1) 先下载到缓存（首次）
python -c "from modelscope import snapshot_download; snapshot_download('iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch')"
# 2) 把模型拷到项目的 models\funasr\ 下（文件名随意，配置里指向它）
xcopy /E /I %USERPROFILE%\.cache\modelscope\models\iic--speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch\snapshots\master models\funasr\speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-pytorch

The VAD model used for barge-in (models/silero-vad/silero_vad_v4.jit, 2MB) is also not committed to the repo; get it from files/silero_vad.jit in the silero-vad v4.0 tag (or use the file bundled with this project's release).

Want to switch back to whisper (e.g. openai/whisper-large-v3 or -turbo)? Just change stt.backend in bridge-config.json to "whisper" and replace model_name with a whisper model id (the bridge supports both backends).

2. TTS engine: OmniVoice (recommended, runs in WSL2 with FlashInfer acceleration)

The project's default TTS has been switched from Qwen3-TTS to OmniVoice (Xiaomi k2-fsa/OmniVoice, 600+ languages, dual clone/design modes), and runs in WSL2 together with FlashInfer (CUDA Graph acceleration, exclusive to Blackwell). Measured benefits:

Metric	Old (Qwen3-TTS)	New (OmniVoice + FlashInfer)
First-time audio latency in clone mode	18.3s	1.6s
Warm-state audio latency	~2.2s	~1.3s
Idle VRAM	7GB (resident)	5.3GB
Voice quantity	Reference-audio cloning	Cloning + parameter-based design (gender/age/pitch/dialect)

Deployment prerequisites: Windows 11 + WSL2 (Ubuntu 22.04). Inside WSL2, use uv to set up the environment: torch 2.13.0+cu130 + flashinfer 0.6.18 + jit-cache cu130 + nvcc cu13 (Blackwell sm_120 requires the cu13 series; cu129 will report "requires GPUs with sm75"). Note: if an older nvcc (e.g. 12.8) exists in the WSL2 system, it will interfere with flashinfer's CUDA version detection — you must set CUDA_HOME to point at the cu13 nvcc directory before starting the service.

Start the OmniVoice service (inside WSL2):

# 一键（含 CUDA_HOME 配置，模型复用 Windows 挂载 /mnt/e 的路径）
wsl -d Ubuntu-22.04 -- bash ~/omnivoice-wsl/start_server.sh

The service listens on 0.0.0.0:9877; the Windows side accesses it via the WSL2's IP (query with wsl hostname -I, which may change after a reboot). Point omnivoice.base in bridge-config to http://<WSL2-IP>:9877. Windows localhost forwarding does not work under the local WSL2 network mode — you must use the IP.

Fallback: Qwen3-TTS (no WSL2 required, degraded option): if you don't use WSL2, you can switch the tts-related settings in bridge-config.json back to Qwen3-TTS-12Hz-1.7B-Base (only Base supports reference-audio cloning; VoiceDesign does not). Download the model from ModelScope:

pip install modelscope
modelscope download --model Qwen/Qwen3-TTS-12Hz-1.7B-Base --local_dir ./Qwen3-TTS-12Hz-1.7B-Base

4. Prepare reference audio

The AI girlfriend's voice is cloned from your reference audio. Please prepare one:

• Length: around 10 seconds (anywhere from 5 to 20 s is fine)
• Content: clean voice only, no background music, no noise — just read some content naturally in your own words (anything you like)
• File: name it ref_audio.wav and place it in the project root (at the same level as the bridge/ folder).

tts.ref_text must match the exact sentence read in the recording (word-for-word, punctuation included) — cloning quality depends on text-audio alignment.

Multi-voice switching (optional)

Multiple voice presets are built in; use the toolbar "🎙️ Voice" button to switch anytime (voice changes instantly, no restart needed). Add your own voice:

1. Create a subfolder under voices/, name = voice name (e.g. voices/我的声音/)
2. Put two files inside:
   • ref_audio.wav — reference audio (~10s clean voice)
   • ref_text.txt — the text actually read in that audio (UTF-8)
3. Restart the bridge; the new voice will appear in the "🎙️ 音色" switching list

Bundled reference voices are not distributed with the repo — add your own this way (audio and text are personal assets, not committed).

5. Fill in the configuration (copy the template + 1 required change)

In the project root, run:

copy bridge\bridge-config.example.json bridge\bridge-config.json

Open bridge\bridge-config.json in Notepad:

Must change (1 place)

Location	Change to
omnivoice.base	OmniVoice service address (WSL2 IP, e.g. http://192.168.1.244:9877)
persona.default_voice	Default voice name (see the folder names under voices/)
tts.ref_text (for the Qwen3 fallback)	The text actually read in your reference audio (see Part 4)

In OmniVoice mode, the model paths under tts.* are no longer used (the engine runs inside the WSL2 service); you only need to fill in tts.model_name when switching back to the Qwen3-TTS fallback:

"C:/你的QwenTTS模型目录/Qwen3-TTS-12Hz-1.7B-Base"     ← 正斜杠
"C:\\你的QwenTTS模型目录\\Qwen3-TTS-12Hz-1.7B-Base"    ← 双反斜杠

Recommended change (1 place)

Location	Change to
tts.ref_text	The text actually read in your reference audio (see Part 4)

No changes needed

Config	Description
media.bg_images_dir / task_videos_dir	Relative paths, resolved automatically from the project root
tts.ref_audio	Defaults to ref_audio.wav in the project root
stt.*	FunASR Chinese recognition config (backend=funasr, model in models/funasr/), works out of the box

6. Verify the bridge first

启动桥接：

bridge\start-bridge.cmd

A minimized terminal window will pop up. Wait 1-2 seconds, then open in your browser:

http://127.0.0.1:8765/api/health

Seeing {"status":"ok", ...} means the bridge is up (models haven't loaded yet; they load on first call).

More complete test (open a separate terminal, in the project root, with venv activated):

venv-speech\Scripts\python.exe bridge\smoke_tts.py --text "你好，我是小雅。"

When done, tts_out.wav will be generated in the project root. Play it — if you hear the cloned voice, the TTS pipeline is working.

7. Install the DSH Voice Plugin

The bridge is just the "voice service"; the chat UI and microphone button live in DSH, so you need to install the plugin. Follow the steps in dsh-plugin/README.md (copy the entire dsh-plugin\ directory into your deepseek-harness source tree, register in three places, build, restart).

8. Streaming digital human: lip-sync replies

A reply is more than just audio — with the DUIX digital human engine, Xiaoya generates a lip-synced video of your reply in real time: TTS audio is fed to DUIX (http://127.0.0.1:8383), which renders the matching lip movements, and the girlfriend window plays video + audio simultaneously once generation completes (audio is baked into the video, so sync is inherent).

Streaming segmentation: long replies are auto-split into ≤10s segments; while the current segment's video renders, the next segment's audio is already synthesizing in the background — seamless continuation, no idle wait. Segment preemption: when a newer reply arrives, remaining segments of the current task yield automatically (already-generated segments are kept), so the latest reply always starts first. The 200 most recent lip-sync videos are kept (GET /api/dh/history).

Segmentation strategy (default: sentence-first + 48-char cap fallback): split by sentence punctuation (。！？!?；;，,); short sentences accumulate into segments (≤48 chars ≈ 8-10s); for overlong sentences, find the nearest break point within the 48-char window (comma/period/space/colon), or hard-cut at 48. → Sentence breaks: semantically complete, TTS prosody coherent, lips match; 48-char cap: ≤10s per segment, DUIX renders in ~15s, fast continuation.

⚙️ Tune it yourself: segment length is controlled by bridge-config.json → digital_human.segment_chars (default 48). Smaller (e.g. 40) → shorter segments, faster rendering but more choppy; larger (e.g. 60) → longer, more coherent segments, but each takes longer to generate (DUIX is single-task, so total wait increases). Adjust to your GPU/listening preference.

麦克风打断：你开始说话（点麦克风/插话）时数字人视频立即停止播放——只停播放，生成任务与已存视频不受影响。

Deploy DUIX (pick one of two):

1. One-click Docker (recommended): this project ships a streamlined, adapted compose (bridge/docker-compose-5060ti.yml) — adapted for the RTX 50 series (5060 Ti / 5090, same architecture), with the original project's Fish Speech TTS and FunASR ASR removed (voice is handled by this project's own voice_bridge), keeping only the digital-human service:

   docker compose -f bridge\docker-compose-5060ti.yml up -d
   

   Once up, the digital-human service runs at http://127.0.0.1:8383; the shared volume d:/duix_avatar_data/face2face maps to the container's /code/data, and finished videos land directly in the host's D:\duix_avatar_data\face2face\temp\.

2. Manual: pull the guiji2025/duix.avatar-5090 image, then run python /code/app_local.py (inside the container).

Configuration (bridge-config.json → digital_human section): duix_base service address, avatar_video avatar video (place in the shared volume temp, e.g. my_avatar.mp4), segment_chars chars per segment (default 48 ≈ 10s), max_keep number of videos to retain (default 200).

Speed-up tips (measured: 7.3s audio 20.1s → 14.1s, ~-30%):

• Switch avatar video to 15fps (output specs follow the avatar video; 1080p quality preserved): ffmpeg -i my_avatar.mp4 -vf fps=15 -c:v libx264 -crf 20 -preset fast -an my_avatar_15fps.mp4 then rename to my_avatar.mp4 (back up the original).
• Disable super-resolution: place a config.ini in the shared volume ([digital] chaofen = 0, batch_size = 2; compose already mounts /code/config/config.ini), survives container rebuilds, ~10% faster.

开关：工具行「数字人」按钮——开：等视频生成完，TTS+画面同步播放；关：去掉视频生成，回到接近即时的纯语音朗读。

不开数字人也完全可用：回复走即时逐句 TTS，女友窗播空闲动画即可。

9. Launch

Bridge only (want to test voice in isolation first):

bridge\start-bridge.cmd

One-click full stack (bridge + NapCatQQ + DSH Web + browser):

set DSH_HARNESS=C:\dev\deepseek-harness
set NAPCAT_DIR=D:\QQ\NapCat\napcat   rem 可选：NapCat 安装目录（默认此值）
bridge\start-all.cmd

• DSH_HARNESS points to the DSH source tree you prepared in step 1 (the one where you installed the plugin in Part 6).
• start-all.cmd automatically: starts the bridge → starts NapCatQQ (will kill all running QQ processes before injecting the sub-account) → starts DSH Web → opens the browser.
• Don't want auto-start of NapCat (e.g. you need to use your main QQ account normally on this computer): bridge\start-all.cmd nq (skips NapCat; two-way QQ chat won't work).
• Want to start NapCat on its own: bridge\start-napcat.cmd (waits for OneBot :3000 to be ready; on failure, prompts you to check the WebUI).

10. Two-way QQ chat (optional)

Chat with the AI girlfriend directly on QQ: your QQ messages are injected into the DSH conversation, and replies are pushed back to your QQ as text + Xiaoya's voice.

1. Install NapCatQQ (log into a QQ sub-account)

• Download NapCat.Shell.Windows.Node.zip (the official-recommended Shell edition; the Framework/LiteLoaderQQNT routes are no longer officially recommended)
• Extract to any directory (e.g. D:\QQ\NapCat), then launch with napcat\launcher-win10-user.bat (this will inject/spin up the desktop QQ client; close any running QQ first)
• Log into QQ with a sub-account (to avoid risk control; after injection the QQ window is usually hidden — check messages via mobile QQ)
• After launching, open NapCat's WebUI on your PC: http://127.0.0.1:6099 (token found in the token field of napcat\config\webui.json)

2. Configure networking in NapCat WebUI

WebUI → Network configuration:

1. HTTP server: Add → name anything, Host 127.0.0.1, port 3000, note the Token (used for sending messages)
2. WebSocket client: Add → fill the full URL ws://127.0.0.1:8765/api/qq/onebot (NapCat actively connects to the bridge to receive QQ messages; do NOT configure it as a WebSocket server, that would compete for port 8765)

3. Configure the bridge

Add a qq section to bridge-config.json:

"qq": {
  "enabled": true,
  "napcat_base": "http://127.0.0.1:3000",
  "napcat_token": "你在 WebUI 设置的 token",
  "target_qq": 你的主号QQ号
}

target_qq is the account that receives replies (your main account); the NapCat sub-account handles sending/receiving.

4. Verify

• Send a test from the bridge: POST /api/qq/send {"text":"你好","voice":true} → main account receives text + voice
• Send an image: POST /api/qq/image {"path":"C:/path/to/a.png"} → main account receives the image (after the agent generates an image it can push it to you directly)
• Full two-way: use mobile QQ main account to send a message to the sub-account → the message appears in the DSH conversation → the reply is automatically sent back to the main account as text + voice

⚠️ Common pitfalls

Pitfall	Explanation
Voice not received	① Don't send voice to the sub-account itself (QQ doesn't support sending voice to yourself); ② voice must be in silk format (pip install pilk, the bridge has built-in conversion); mp3 base64 won't go out
QQ events not arriving	Must configure a WebSocket client (NapCat connects to ws://127.0.0.1:8765/api/qq/onebot); configuring it as a WebSocket server will compete for port 8765 and fail
OneBot service lost after NapCat restart	NapCat's OneBot HTTP (3000) sometimes doesn't auto-load after restart — manage everything in the WebUI (enable/save), don't kill the process and restart
WebUI asks for token every time	Set the token field in napcat\config\webui.json to a fixed value
Reply latency	Latency is mainly from LLM generation + full TTS synthesis; longer replies are slower (segmented push is a future option)
Port conflicts	8765 bridge, 3000 OneBot, 6099 WebUI — if occupied, change the config and sync-bridge's napcat_base

Usage

1. Tap Mic: start continuous listening (each sentence is auto-recognized and sent; tap again to stop)
2. Tap ⚡: barge-in (lit, default) / queue (unlit) — speak to interrupt the reply vs. let the reply finish before auto-continuing
3. Tap 👧: show/hide the girlfriend animation window; the window can be dragged wider, double-click to switch sides
4. Tap 🎭: digital human toggle — ON: replies generate lip-sync video, TTS + visuals play in sync (segment continuation); OFF: near-instant pure audio reading
5. Tap 🔊: toggle audio reading on/off
6. QQ chat: just send a message from your mobile QQ main account to the sub-account (requires "Part 10: Two-way QQ chat" to be set up)

---

FAQ

1. pip 装依赖太慢/超时 → 用清华镜像：

   pip install -r bridge\requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
   

2. HuggingFace 下载模型慢/失败 → 用国内镜像：

   set HF_ENDPOINT=https://hf-mirror.com
   

   设置后重新执行下载命令（huggingface-cli download ...）。

3. 第一次 TTS 很慢（10~60s） → 正常。模型首次懒加载 + 预热，之后每句约 0.5s 出音。

4. STT 偶尔识别为空 → FunASR 对超短语音/纯噪声会返回空结果被丢弃（日志见 funasr returned empty result）；若切换到 whisper 后端，则其 1-token 退化也会被判空，均属正常防护。

5. 桥接启动但模型加载失败 → 检查 bridge-config.json 里 tts.model_name 路径是否存在（路径拼写、盘符、斜杠方向），以及显卡驱动是否够新（见前置第 4 条）。

6. 桥接端口被占 → 改 bridge-config.json 的 port，并同步改插件的 s2s.voice.bridge（浏览器 localStorage）。

7. 女友窗不显示 → 确认 assets/ 目录存在、桥接已启动（窗口每 30s 拉一次素材列表）。

8. 数字人没出视频 → 检查：① 工具行数字人开关是否打开；② bridge-config.json 的 digital_human.enabled 是否为 true、duix_base 是否指向 DUIX 服务；③ 形象视频（avatar_video 配置的文件名，如 my_avatar.mp4）是否放在共享卷 temp 目录（DUIX 容器映射的 D:\duix_avatar_data\face2face\temp\）；④ 生成状态用 GET http://127.0.0.1:8765/api/dh/status 查看（done 且带 video_url 即为就绪）。

9. 数字人视频生成了但没播 → 刷新 DSH 页面（插件逻辑更新后需刷新加载）；确认女友窗可见。生成中窗口底部会显示「数字人生成中 i/n…」，一段播完自动续接下一段。

10. 声音不像参考音频 → 检查 ref_audio.wav 是否清晰无杂音、参考文本（voices/<音色名>/ref_text.txt）是否与录音逐字一致（标点也要对）；参考音频建议 3-10 秒单一完整句，不要跨句截取（跨句会导致输出混入参考内容）。OmniVoice 服务未启动时检查 ~/omnivoice-wsl/server.log 是否有报错。

---

Custom assets (voice / idle animation / digital human avatar)

The project ships with 5 default idle animations (assets/bg-images/bg2|bg4|bg9|bg56|bg5), ready out of the box. Below is how to add your own assets — the three categories don't affect each other and can be switched on the toolbar at any time:

🎙️ Custom voice (voices/ subfolder, multiple can coexist)

1. 在 voices/ 下新建子文件夹，文件夹名即音色名（如 voices/我的声音/）
2. 放入 ref_audio.wav（10 秒左右干净人声）+ ref_text.txt（该音频实际朗读的文本，UTF-8）
3. 重启桥接 → 工具行「🎙️ 音色」按钮循环切换

🖼️ Custom idle animation (assets/bg-images/ subfolder, multiple can coexist)

1. 在 assets/bg-images/ 下新建子文件夹（如 assets/bg-images/我的背景/），放入视频/图片
2. 无需重启：工具行「🖼️ 待机」按钮即时切换（每次查询自动扫描新文件夹）
3. 文件夹名即待机组名，建议用英文/数字避免 URL 编码问题

👤 Custom digital human avatar (shared volume temp directory, D:\duix_avatar_data\face2face\temp\)

1. Put the avatar video (mp4) into the temp directory (shared volume used by both the DUIX container and host)
2. Switch with the toolbar "👤 Avatar" button; or set digital_human.avatar_video in bridge-config.json to specify the default avatar
3. Avatar assets are not distributed with the repo (they're personal); output specs (resolution/frame rate) follow the avatar video file

All three selections are remembered (localStorage) and auto-restored after refresh/restart.

---

License

Apache-2.0 (see LICENSE for details). Reuses HuggingFace speech-to-speech (Apache-2.0) and the deepseek-harness plugin framework (MIT); assets/ materials are AI-generated and for demo use only.