feat(asr): add pluggable ONNX streaming backend and model registry
Add a second backend (sherpa-onnx) alongside llama.cpp, so native streaming models can be selected by ID from a JSON registry. All such models share one adapter: only the model files and a few options differ. Models now live under the app data directory and are fetched by scripts/fetch-models.sh. Adding a model is a data change, not code. Measured on the same audio: the streaming backend is ~35x cheaper in CPU than Qwen3-ASR (0.10-0.12 vs 4.27 core-seconds per audio second), shows the first caption at 0.48-1.44s, and matches its accuracy for zh/en. 新增 sherpa-onnx 流式后端,与 llama.cpp 并存;模型按注册表 id 从应用 数据目录加载。同族模型共用一个适配器,新增模型只改 data/models.json。 实测流式后端 CPU 约为 Qwen3-ASR 的 1/35,首个字幕 0.48~1.44s,中英 准确率持平。注意预编译库是旧 C++ ABI,必须走其 C API。
This commit is contained in:
1 parent
6d64987d64
commit
bd921b1ac5
15 files changed
+869
-18
No files matched your search
@@ -47,6 +47,22 @@ cmake --build build -j
|
||||
ctest --test-dir build --output-on-failure
|
||||
```
|
||||
|
||||
## 推理后端
|
||||
|
||||
支持两条后端,用 `--asr-backend` 切换(详见 [docs/asr-backends.md](docs/asr-backends.md)):
|
||||
|
||||
- **`llama`(默认)**:llama.cpp/ggml + Qwen3-ASR 的 GGUF 模型 —— 准确率高、52 种语言,但每步成本随段落增长。
|
||||
- **`sherpa`**:sherpa-onnx + 流式 ONNX 模型(X-ASR / zipformer / Nemotron) —— 原生流式,每步成本恒定,CPU 约为前者的 1/35,中英准确率持平。
|
||||
|
||||
模型都放在 `~/.local/share/live-caption/models/`,注册表是 `data/models.json`
|
||||
(**加模型只加一条数据,不改代码**),用 `scripts/fetch-models.sh` 下载。
|
||||
|
||||
```bash
|
||||
# 流式后端
|
||||
./build/live-caption-cli --asr-backend sherpa --asr-model zipformer-zh-2025
|
||||
./build/live-caption-cli --asr-backend sherpa --asr-model nemotron35-560ms --language ja
|
||||
```
|
||||
|
||||
## 代码结构
|
||||
|
||||
```
|
||||
|
||||
Reference in new issue
Block a user