feat(stream): add append-only streaming mode with flat cost
Add an append-only streaming path that feeds each audio chunk as a new conversation turn, so nothing is ever recomputed: the KV grows by append, no prefix is re-decoded and no audio is re-encoded. Per-step cost becomes flat instead of growing with the segment (RTF 1.06 -> 0.26 on 62.6s of continuous speech), but accuracy requires ~8s chunks because the audio encoder needs its 8s attention window, so text only appears every 8s. The incremental mode stays the default. 新增追加式流式:每块音频作为一个新的对话轮追加,前缀不重算、旧音频不重 编码,单步成本不再随段落增长(连续 62.6s 语音 RTF 从 1.06 降到 0.26)。 但准确率要求 chunk≈8s,否则会被截断,所以出字粒度是 8s;增量式仍为默认。
This commit is contained in:
1 parent
7e5257429a
commit
85b206f1cd
10 files changed
+326
-14
No files matched your search
+4
-3
@@ -72,12 +72,13 @@ int runStreamFile(lc::Qwen3Asr & asr, const QString & path, const lc::StreamOpti
|
||||
const size_t chunkSamples = (size_t) (sop.chunkSec * asr.sampleRate());
|
||||
|
||||
const auto onStep = [](const lc::StreamingAsr::Step & s) {
|
||||
std::printf("\n[chunk %2d] 窗口 %5.1fs / 编码 %5.2fs / 本步 %5.2fs%s\n",
|
||||
s.chunkIndex, s.windowSeconds, s.encodeSeconds, s.inferSeconds,
|
||||
std::printf("\n[chunk %2d] 累计 %5.1fs | 音频 %5.2fs | 前缀 %5.2fs | 生成 %5.2fs (%d tok) "
|
||||
"| 合计 %5.2fs%s\n",
|
||||
s.chunkIndex, s.windowSeconds, s.encodeSeconds, s.prefillSeconds,
|
||||
s.genSeconds, s.newTokens, s.inferSeconds,
|
||||
s.ok ? "" : " <失败>");
|
||||
if (!s.ok) std::printf(" error: %s\n", s.error.c_str());
|
||||
std::printf(" 已提交: %s\n", s.committed.c_str());
|
||||
std::printf(" 含尾部: %s\n", s.full.c_str());
|
||||
std::fflush(stdout);
|
||||
};
|
||||
|
||||
|
||||
@@ -44,7 +44,8 @@ inline void addStreamOptions(QCommandLineParser & p)
|
||||
p.addOption(makeOption("stream-segment", "单段最长时长,单位秒;0=不限", "sec", "30"));
|
||||
p.addOption(makeOption("unfixed-tokens", "前缀回滚丢弃的 token 数", "n", "5"));
|
||||
p.addOption(makeOption("unfixed-chunks", "冷启动不带前缀的 chunk 数", "n", "2"));
|
||||
p.addOption(makeOption("stream-mode", "流式后端: incremental 或 window", "mode", "incremental"));
|
||||
p.addOption(makeOption("stream-mode", "流式后端: incremental / append / window", "mode",
|
||||
"incremental"));
|
||||
}
|
||||
|
||||
inline AsrOptions asrOptions(const QCommandLineParser & p)
|
||||
@@ -67,7 +68,8 @@ inline StreamOptions streamOptions(const QCommandLineParser & p)
|
||||
o.unfixedTokenNum = p.value("unfixed-tokens").toInt();
|
||||
o.unfixedChunkNum = p.value("unfixed-chunks").toInt();
|
||||
o.maxTokensPerStep = p.value("max-tokens").toInt();
|
||||
o.incremental = p.value("stream-mode") != QLatin1String("window");
|
||||
o.incremental = p.value("stream-mode") == QLatin1String("incremental");
|
||||
o.appendOnly = p.value("stream-mode") == QLatin1String("append");
|
||||
return o;
|
||||
}
|
||||
|
||||
|
||||
@@ -2,13 +2,66 @@
|
||||
|
||||
状态:**完成**。准确率与离线转写一致,单步计算稳定在 2s 预算内。
|
||||
|
||||
## 两种流式模式(实测对比)
|
||||
|
||||
代码里实现了两条流式路径,用 `--stream-mode` 切换。它们对应一个**无法同时满足的三选二**:
|
||||
|
||||
| 目标 | `incremental`(默认) | `append`(追加式) |
|
||||
|---|---|---|
|
||||
| 成本不随说话时长增长 | ❌ | ✅ |
|
||||
| CPU 效率 | RTF 1.06(无段落上限时) | **RTF 0.26** |
|
||||
| 1–2s 就出字 | ✅ | ❌ 需 8s chunk |
|
||||
| 准确率 | ✅(2s 提交 + 8s 编码切片) | ✅(仅 chunk≈8s 时) |
|
||||
|
||||
同一段 62.6s 连续语音(不设段落上限):
|
||||
|
||||
| 模式 | 墙钟 | RTF | 首步 | 末步 | 最慢步 |
|
||||
|---|---|---|---|---|---|
|
||||
| `incremental` | 66.3s | 1.06 | 0.62s | 3.14s | 4.06s |
|
||||
| `append` chunk=2 | 32.6s | 0.52 | 0.70s | 0.67s | 1.32s |
|
||||
| `append` chunk=8 | **16.0s** | **0.26** | — | — | 2.52s |
|
||||
|
||||
`append` 的每步耗时几乎不随总长变化;`incremental` 则线性增长(尾步是最慢的)。
|
||||
|
||||
### 为什么 `append` 必须用大 chunk
|
||||
|
||||
准确率对 **chunk 长度**极其敏感,因为编码器(AuT)用的是 **8 秒注意力窗口**(`n_window_infer = 800` 帧 @100Hz)。每块只用自己的长度去编码时:
|
||||
|
||||
| chunk | 中文实测结果 |
|
||||
|---|---|
|
||||
| 2s | ❌ `不排除购房者可以在互。`(每块都被截断,`互联网上下单` 全丢) |
|
||||
| 4s | ⚠️ `不排除购房者可以在互联网上下单,不排除。`(基本对,边界多一个词) |
|
||||
| 8s | ✅ 与离线一致 |
|
||||
|
||||
所以「便宜/平坦」和「1–2s 出字」对 Qwen3-ASR 是互斥的:要准确就得给编码器 8s 上下文,给了就必然 8s 才出一次字。
|
||||
|
||||
### 追加式为什么便宜
|
||||
|
||||
`append` 把每块音频当作一个新的对话轮追加到 KV 末尾:
|
||||
|
||||
```
|
||||
[user: audio_1][assistant: text_1][user: audio_2][assistant: text_2]...
|
||||
```
|
||||
|
||||
纯追加 —— **前缀完全不重算、旧音频不重编码、KV 永不回退**,所以单步成本与音频总长无关。
|
||||
|
||||
而 `incremental` 每步都要把「音频尾部 + 到目前为止的全部文本前缀」按新的音频上下文重算一遍(因果注意力决定了:把新音频插到文本之前,文本之后的 KV 全部作废)。实测拆解显示这一步的代价是 O(N²):
|
||||
|
||||
| 累计音频 | 音频(编码+写KV) | 文本前缀 prefill | 生成 | 合计 |
|
||||
|---|---|---|---|---|
|
||||
| 2.0s | 0.35s | 0.03s | 0.19s | 0.58s |
|
||||
| 46.0s | 1.23s | 1.20s | 0.38s | 2.81s |
|
||||
| 62.6s | 1.46s | **1.53s** | 0.16s | 3.15s |
|
||||
|
||||
「文本前缀 prefill」从 0.03s 涨到 1.53s,占末尾步耗时的一半。`append` 模式把它降到了 **0.06s**。
|
||||
|
||||
## 结论
|
||||
|
||||
| 指标 | 结果 |
|
||||
| 数据 | 结果 |
|
||||
|---|---|
|
||||
| 准确率 | 流式输出与离线转写逐字一致(中/英各自验证) |
|
||||
| 单步计算 | 最慢 1.83s(预算 2.0s,提交粒度 2s) |
|
||||
| 整体 RTF | 约 0.66 |
|
||||
| 单步计算(incremental,segment≤20s) | 最慢 1.84s(预算 2.0s,提交粒度 2s) |
|
||||
| 整体 RTF | 约 0.59 |
|
||||
| 实现 | 纯 C++,libllama + libmtmd,无 Python |
|
||||
|
||||
## 算法
|
||||
@@ -74,15 +127,19 @@
|
||||
## 用法
|
||||
|
||||
```bash
|
||||
# 流式转写(增量 + 8s 切片 + 2s 提交)
|
||||
# 流式转写(默认 incremental:8s 切片 + 前缀回滚 + 2s 提交)
|
||||
./build/live-caption-cli --model MODEL --mmproj MMPROJ \
|
||||
--stream-file audio_16k_mono.wav [--stream-chunk 2] [--stream-segment 30]
|
||||
|
||||
# 对比用:旧的"重编码整个窗口"实现
|
||||
# 追加式:成本平坦、省 CPU,但需要大 chunk 才准确
|
||||
./build/live-caption-cli --model MODEL --mmproj MMPROJ \
|
||||
--stream-file audio_16k_mono.wav --stream-mode append --stream-chunk 8
|
||||
|
||||
# 对比用:最早的"重编码整个窗口"实现
|
||||
./build/live-caption-cli ... --stream-mode window --stream-window 4
|
||||
```
|
||||
|
||||
相关参数:`--stream-chunk`(提交粒度)、`--stream-window`(仅 window 模式)、`--stream-segment`(单段上限)、`--unfixed-tokens`、`--unfixed-chunks`。
|
||||
相关参数:`--stream-mode`(incremental / append / window)、`--stream-chunk`(提交粒度)、`--stream-segment`(单段上限)、`--stream-window`(仅 window 模式)、`--unfixed-tokens`、`--unfixed-chunks`。
|
||||
|
||||
## 遗留
|
||||
|
||||
|
||||
+11
-2
@@ -24,8 +24,10 @@ struct AsrStreamStep {
|
||||
std::string newText; // 本步新生成的文本(不含前缀,未 trim)
|
||||
int newTokens = 0;
|
||||
int audioTokens = 0;
|
||||
double encodeSeconds = 0.0;
|
||||
double inferSeconds = 0.0;
|
||||
double encodeSeconds = 0.0; // 音频编码 + 写入 KV
|
||||
double prefillSeconds = 0.0; // 文本前缀(tail + prefix)prefill
|
||||
double genSeconds = 0.0; // 贪心生成
|
||||
double inferSeconds = 0.0; // 以上合计
|
||||
};
|
||||
|
||||
// 把推理后端抽象出来,好处是流式策略(StreamingAsr)和实时管线(LivePipeline)
|
||||
@@ -44,6 +46,13 @@ public:
|
||||
// 流式:只编码新音频,已解码的音频留在 KV cache 里复用
|
||||
virtual AsrStreamStep streamAppend(const float * pcm, size_t nSamples,
|
||||
const std::string & prefix, int maxTokens) = 0;
|
||||
|
||||
// 追加式轮次(append-only):把这一小段音频当作一个新的对话轮,
|
||||
// 追加到已有 KV 的末尾,不重算任何前缀、也不重编码旧音频。
|
||||
// 代价是每块只用自己的长度做编码(没有 8s 上下文),准确率需要实测确认。
|
||||
virtual AsrStreamStep streamAppendTurn(const float * pcm, size_t nSamples,
|
||||
int maxTokens) = 0;
|
||||
|
||||
virtual void streamReset() = 0;
|
||||
|
||||
// 用词表把 text 编码成 token、丢掉最后 dropLast 个、再还原成文本。
|
||||
|
||||
@@ -38,6 +38,13 @@ struct Qwen3Asr::Impl {
|
||||
size_t stableSamples = 0; // 已固化进 KV 的音频样本数(8s 对齐)
|
||||
llama_pos nPastStable = 0; // stable 音频末尾的 KV 位置
|
||||
|
||||
// 追加式轮次(append-only)状态
|
||||
bool turnReady = false;
|
||||
std::vector<llama_token> firstHead; // 第 1 轮的轮首文本(含 <|audio_start|>)
|
||||
std::vector<llama_token> contHead; // 后续轮的轮首文本(<|im_end|>\n<|im_start|>user\n<|audio_start|>)
|
||||
llama_pos nPastTurn = 0;
|
||||
int turnCount = 0;
|
||||
|
||||
std::string buildPrompt(const std::string & marker) const
|
||||
{
|
||||
const char * tmpl = llama_model_chat_template(model, nullptr);
|
||||
@@ -112,6 +119,48 @@ struct Qwen3Asr::Impl {
|
||||
nPast = np;
|
||||
return true;
|
||||
}
|
||||
|
||||
// 把带 media marker 的 prompt 拆成「音频前文本」和「音频后文本」两组 token
|
||||
bool splitPrompt(const std::string & skeleton,
|
||||
std::vector<llama_token> & head,
|
||||
std::vector<llama_token> & tail,
|
||||
std::string * err)
|
||||
{
|
||||
std::vector<float> dummy(1600, 0.0f); // 0.1s 占位,只为触发音频分块
|
||||
mtmd_bitmap * bmp = mtmd_bitmap_init_from_audio(dummy.size(), dummy.data());
|
||||
mtmd_input_chunks * chunks = mtmd_input_chunks_init();
|
||||
const mtmd_input_text itext{ skeleton.c_str(), true, true };
|
||||
const mtmd_bitmap * bms[1] = { bmp };
|
||||
|
||||
if (mtmd_tokenize(mctx, chunks, &itext, bms, 1) != 0) {
|
||||
mtmd_bitmap_free(bmp);
|
||||
mtmd_input_chunks_free(chunks);
|
||||
if (err) *err = "mtmd_tokenize 失败";
|
||||
return false;
|
||||
}
|
||||
mtmd_bitmap_free(bmp);
|
||||
|
||||
bool seenAudio = false;
|
||||
for (size_t i = 0; i < mtmd_input_chunks_size(chunks); ++i) {
|
||||
const mtmd_input_chunk * c = mtmd_input_chunks_get(chunks, i);
|
||||
if (mtmd_input_chunk_get_type(c) == MTMD_INPUT_CHUNK_TYPE_AUDIO) {
|
||||
seenAudio = true;
|
||||
continue;
|
||||
}
|
||||
size_t n = 0;
|
||||
const llama_token * toks = mtmd_input_chunk_get_tokens_text(c, &n);
|
||||
if (!toks || n == 0) continue;
|
||||
std::vector<llama_token> & dst = seenAudio ? tail : head;
|
||||
dst.insert(dst.end(), toks, toks + n);
|
||||
}
|
||||
mtmd_input_chunks_free(chunks);
|
||||
|
||||
if (head.empty()) {
|
||||
if (err) *err = "prompt 里没有音频之前的文本";
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
};
|
||||
|
||||
Qwen3Asr::Qwen3Asr() : d(new Impl) {}
|
||||
@@ -313,6 +362,109 @@ void Qwen3Asr::streamReset()
|
||||
d->streamAudio.clear();
|
||||
d->stableSamples = 0;
|
||||
d->nPastStable = 0;
|
||||
|
||||
d->turnReady = false;
|
||||
d->firstHead.clear();
|
||||
d->contHead.clear();
|
||||
d->nPastTurn = 0;
|
||||
d->turnCount = 0;
|
||||
}
|
||||
|
||||
// 追加式轮次:把每小段音频当作一个新的对话轮追加到 KV 末尾。
|
||||
// 序列形如 [user: audio_1][assistant: text_1][user: audio_2][assistant: text_2]...
|
||||
// 好处是纯追加:不重算前缀、不重编码旧音频,单步成本与音频总长无关。
|
||||
bool Qwen3Asr::prepareTurn(std::string * error)
|
||||
{
|
||||
if (d->turnReady) return true;
|
||||
|
||||
const std::string marker = mtmd_default_marker();
|
||||
// 第 1 轮是完整的 user/assistant 结构;后续轮接在上一轮文本之后开新一轮。
|
||||
const std::string s1 = "<|im_start|>user\n" + marker +
|
||||
"<|im_end|>\n<|im_start|>assistant\n";
|
||||
const std::string sc = "<|im_end|>\n<|im_start|>user\n" + marker +
|
||||
"<|im_end|>\n<|im_start|>assistant\n";
|
||||
|
||||
std::vector<llama_token> tail1, tailC;
|
||||
if (!d->splitPrompt(s1, d->firstHead, tail1, error)) return false;
|
||||
if (!d->splitPrompt(sc, d->contHead, tailC, error)) return false;
|
||||
|
||||
// 两种轮尾应当一致
|
||||
d->tailTokens = tail1;
|
||||
|
||||
llama_memory_clear(llama_get_memory(d->ctx), true);
|
||||
d->nPastTurn = 0;
|
||||
d->turnCount = 0;
|
||||
d->turnReady = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
AsrStreamStep Qwen3Asr::streamAppendTurn(const float * pcm, size_t nSamples, int maxTokens)
|
||||
{
|
||||
AsrStreamStep res;
|
||||
if (!d->model || !d->ctx || !d->mctx) { res.error = "模型未加载"; return res; }
|
||||
if (nSamples == 0) { res.error = "音频为空"; return res; }
|
||||
|
||||
std::string err;
|
||||
if (!prepareTurn(&err)) { res.error = err; return res; }
|
||||
|
||||
const auto t0 = std::chrono::steady_clock::now();
|
||||
|
||||
// 1) 轮首文本(纯追加,位置接在 KV 末尾)
|
||||
const std::vector<llama_token> & head = (d->turnCount == 0) ? d->firstHead : d->contHead;
|
||||
llama_pos p = d->nPastTurn;
|
||||
if (!d->decodeTokens(head.data(), (int32_t) head.size(), p, false)) {
|
||||
res.error = "解码轮首失败";
|
||||
return res;
|
||||
}
|
||||
p += (llama_pos) head.size();
|
||||
|
||||
// 2) 这一小段音频:只编码一次,写进 KV 就永久保留
|
||||
const llama_pos beforeAudio = p;
|
||||
if (!d->appendAudio(pcm, nSamples, p, &err)) {
|
||||
res.error = "音频编码/解码失败: " + err;
|
||||
return res;
|
||||
}
|
||||
res.audioTokens = (int) (p - beforeAudio);
|
||||
const auto tEnc = std::chrono::steady_clock::now();
|
||||
|
||||
// 3) 轮尾文本 + 开始生成
|
||||
if (!d->decodeTokens(d->tailTokens.data(), (int32_t) d->tailTokens.size(), p, true)) {
|
||||
res.error = "解码轮尾失败";
|
||||
return res;
|
||||
}
|
||||
p += (llama_pos) d->tailTokens.size();
|
||||
|
||||
const auto tPre = std::chrono::steady_clock::now();
|
||||
|
||||
llama_sampler * smpl = llama_sampler_chain_init(llama_sampler_chain_default_params());
|
||||
llama_sampler_chain_add(smpl, llama_sampler_init_greedy());
|
||||
|
||||
std::string out;
|
||||
for (int i = 0; i < maxTokens; ++i) {
|
||||
llama_token id = llama_sampler_sample(smpl, d->ctx, -1);
|
||||
if (llama_vocab_is_eog(d->vocab, id)) break;
|
||||
|
||||
char buf[256];
|
||||
const int32_t n = llama_token_to_piece(d->vocab, id, buf, sizeof(buf), 0, true);
|
||||
if (n > 0) out.append(buf, (size_t) n);
|
||||
++res.newTokens;
|
||||
|
||||
if (!d->decodeTokens(&id, 1, p, true)) { res.error = "生成时解码失败"; break; }
|
||||
++p;
|
||||
}
|
||||
llama_sampler_free(smpl);
|
||||
|
||||
d->nPastTurn = p;
|
||||
++d->turnCount;
|
||||
|
||||
const auto t1 = std::chrono::steady_clock::now();
|
||||
res.encodeSeconds = std::chrono::duration<double>(tEnc - t0).count();
|
||||
res.prefillSeconds = std::chrono::duration<double>(tPre - tEnc).count();
|
||||
res.genSeconds = std::chrono::duration<double>(t1 - tPre).count();
|
||||
res.inferSeconds = std::chrono::duration<double>(t1 - t0).count();
|
||||
res.newText = out;
|
||||
res.ok = res.error.empty();
|
||||
return res;
|
||||
}
|
||||
|
||||
bool Qwen3Asr::prepareStream(std::string * error)
|
||||
@@ -452,6 +604,8 @@ AsrStreamStep Qwen3Asr::streamAppend(const float * pcm, size_t nSamples,
|
||||
llama_sampler * smpl = llama_sampler_chain_init(llama_sampler_chain_default_params());
|
||||
llama_sampler_chain_add(smpl, llama_sampler_init_greedy());
|
||||
|
||||
const auto tPre = std::chrono::steady_clock::now();
|
||||
|
||||
std::string out;
|
||||
for (int i = 0; i < maxTokens; ++i) {
|
||||
llama_token id = llama_sampler_sample(smpl, d->ctx, -1);
|
||||
@@ -469,6 +623,8 @@ AsrStreamStep Qwen3Asr::streamAppend(const float * pcm, size_t nSamples,
|
||||
|
||||
const auto t1 = std::chrono::steady_clock::now();
|
||||
res.encodeSeconds = std::chrono::duration<double>(tEnc - t0).count();
|
||||
res.prefillSeconds = std::chrono::duration<double>(tPre - tEnc).count();
|
||||
res.genSeconds = std::chrono::duration<double>(t1 - tPre).count();
|
||||
res.inferSeconds = std::chrono::duration<double>(t1 - t0).count();
|
||||
res.newText = out;
|
||||
res.ok = res.error.empty();
|
||||
|
||||
@@ -38,12 +38,14 @@ public:
|
||||
|
||||
AsrStreamStep streamAppend(const float * pcm, size_t nSamples,
|
||||
const std::string & prefix, int maxTokens) override;
|
||||
AsrStreamStep streamAppendTurn(const float * pcm, size_t nSamples, int maxTokens) override;
|
||||
void streamReset() override;
|
||||
|
||||
std::string dropLastTokens(const std::string & text, int dropLast) const override;
|
||||
|
||||
private:
|
||||
bool prepareStream(std::string * error);
|
||||
bool prepareTurn(std::string * error);
|
||||
|
||||
struct Impl;
|
||||
Impl * d = nullptr;
|
||||
|
||||
@@ -94,15 +94,33 @@ void StreamingAsr::runStep(const std::vector<float> & audio, bool finalStep,
|
||||
s.chunkIndex = chunkIndex_;
|
||||
|
||||
std::string newText;
|
||||
if (opt_.incremental) {
|
||||
if (opt_.appendOnly) {
|
||||
// 追加式:这一块作为一个新的对话轮,文本随即定稿,不做前缀回滚
|
||||
AsrStreamStep r = asr_.streamAppendTurn(audio.data(), audio.size(),
|
||||
opt_.maxTokensPerStep);
|
||||
s.ok = r.ok;
|
||||
s.error = r.error;
|
||||
s.encodeSeconds = r.encodeSeconds;
|
||||
s.prefillSeconds = r.prefillSeconds;
|
||||
s.genSeconds = r.genSeconds;
|
||||
s.inferSeconds = r.inferSeconds;
|
||||
s.audioTokens = r.audioTokens;
|
||||
s.newTokens = r.newTokens;
|
||||
newText = r.newText;
|
||||
totalSeconds_ += (double) audio.size() / asr_.sampleRate();
|
||||
s.windowSeconds = totalSeconds_;
|
||||
} else if (opt_.incremental) {
|
||||
// 只把这一块交给增量引擎;音频总长不影响单步成本
|
||||
AsrStreamStep r =
|
||||
asr_.streamAppend(audio.data(), audio.size(), prefix, opt_.maxTokensPerStep);
|
||||
s.ok = r.ok;
|
||||
s.error = r.error;
|
||||
s.encodeSeconds = r.encodeSeconds;
|
||||
s.prefillSeconds = r.prefillSeconds;
|
||||
s.genSeconds = r.genSeconds;
|
||||
s.inferSeconds = r.inferSeconds;
|
||||
s.audioTokens = r.audioTokens;
|
||||
s.newTokens = r.newTokens;
|
||||
newText = r.newText;
|
||||
totalSeconds_ += (double) audio.size() / asr_.sampleRate();
|
||||
s.windowSeconds = totalSeconds_;
|
||||
@@ -126,6 +144,23 @@ void StreamingAsr::runStep(const std::vector<float> & audio, bool finalStep,
|
||||
}
|
||||
|
||||
if (s.ok) {
|
||||
if (opt_.appendOnly) {
|
||||
// 追加式:每轮文本直接定稿,追加进历史,没有"不稳定尾部"
|
||||
const ParsedTranscript pTurn = parseAsrOutput(newText);
|
||||
if (!pTurn.language.empty()) language_ = pTurn.language;
|
||||
historyText_ += pTurn.text;
|
||||
s.rawFull = newText;
|
||||
s.full = historyText_;
|
||||
s.committed = historyText_;
|
||||
s.tail.clear();
|
||||
s.language = language_;
|
||||
lastRawFull_.clear();
|
||||
committedRaw_.clear();
|
||||
++chunkIndex_;
|
||||
if (onStep) onStep(s);
|
||||
return;
|
||||
}
|
||||
|
||||
// 拼接要用未经 trim 的文本:流式续写时开头那个空格是有意义的
|
||||
s.rawFull = prefix + newText;
|
||||
lastRawFull_ = s.rawFull;
|
||||
|
||||
@@ -21,6 +21,11 @@ struct StreamOptions {
|
||||
// false = 每次重编码整个窗口(Phase 2 早期的做法,仅用于对比)
|
||||
bool incremental = true;
|
||||
|
||||
// true = 追加式轮次(append-only):每块音频作为新一轮追加,前缀完全不重算、
|
||||
// 旧音频不重编码,单步成本与总长无关。代价是每块只用自己的长度编码
|
||||
// (没有 8s 上下文),准确率需实测确认。
|
||||
bool appendOnly = false;
|
||||
|
||||
// 单段最长时长(秒):超过就提交并重置上下文(0 = 不限)。
|
||||
// 真实场景应由静音检测决定分段;这里是一个兜底上界,
|
||||
// 因为解码器对 KV 的注意力随总长增长,段太长单步会超预算。
|
||||
@@ -42,8 +47,11 @@ public:
|
||||
std::string rawFull; // 未 trim 的完整文本(含 language 头)
|
||||
double windowSeconds = 0.0;
|
||||
double encodeSeconds = 0.0;
|
||||
double prefillSeconds = 0.0;
|
||||
double genSeconds = 0.0;
|
||||
double inferSeconds = 0.0;
|
||||
int audioTokens = 0;
|
||||
int newTokens = 0;
|
||||
bool ok = true;
|
||||
std::string error;
|
||||
};
|
||||
|
||||
@@ -18,6 +18,7 @@ public:
|
||||
|
||||
// 记录(供断言用)
|
||||
std::vector<std::string> prefixesSeen;
|
||||
std::vector<std::string> turnsSeen;
|
||||
std::vector<size_t> samplesSeen;
|
||||
int resets = 0;
|
||||
|
||||
@@ -52,6 +53,20 @@ public:
|
||||
return s;
|
||||
}
|
||||
|
||||
lc::AsrStreamStep streamAppendTurn(const float *, size_t nSamples, int) override
|
||||
{
|
||||
samplesSeen.push_back(nSamples);
|
||||
|
||||
lc::AsrStreamStep s;
|
||||
s.ok = true;
|
||||
if (call_ < scripted.size()) s.newText = scripted[call_];
|
||||
++call_;
|
||||
turnsSeen.push_back(s.newText);
|
||||
s.newTokens = 1;
|
||||
s.inferSeconds = 0.001;
|
||||
return s;
|
||||
}
|
||||
|
||||
void streamReset() override { ++resets; }
|
||||
|
||||
std::string dropLastTokens(const std::string & text, int dropLast) const override
|
||||
|
||||
@@ -150,6 +150,33 @@ private slots:
|
||||
QVERIFY2(all.contains("hello there world"), qPrintable(all));
|
||||
}
|
||||
|
||||
// 追加式:每块音频作为新一轮追加,文本立即定稿,不回喂任何前缀
|
||||
void appendModeCommitsEachChunkImmediately()
|
||||
{
|
||||
FakeAsr fake;
|
||||
fake.scripted = {"language English<asr_text>one two",
|
||||
" language English<asr_text> three four"};
|
||||
|
||||
lc::StreamOptions o = baseOptions();
|
||||
o.appendOnly = true;
|
||||
|
||||
lc::StreamingAsr stream(fake, o);
|
||||
std::vector<lc::StreamingAsr::Step> steps;
|
||||
const auto cb = [&steps](const lc::StreamingAsr::Step & s) { steps.push_back(s); };
|
||||
|
||||
const auto c = chunk(16000);
|
||||
stream.feed(c.data(), c.size(), cb);
|
||||
stream.feed(c.data(), c.size(), cb);
|
||||
|
||||
QCOMPARE(fake.turnsSeen.size(), size_t(2));
|
||||
QVERIFY2(fake.prefixesSeen.empty(), "追加式不应该回喂前缀");
|
||||
|
||||
QCOMPARE(steps.size(), size_t(2));
|
||||
QCOMPARE(q(steps[0].committed), QString("one two"));
|
||||
QVERIFY(steps[0].tail.empty()); // 追加式没有"不稳定尾部"
|
||||
QVERIFY(q(steps[1].committed).contains("three four"));
|
||||
}
|
||||
|
||||
// 同一次 feed 里跨多块也要正确切分
|
||||
void splitsMultipleChunksInOneFeed()
|
||||
{
|
||||
|
||||
Reference in new issue
Block a user