feat(stream): add append-only streaming mode with flat cost

Add an append-only streaming path that feeds each audio chunk as a new
conversation turn, so nothing is ever recomputed: the KV grows by
append, no prefix is re-decoded and no audio is re-encoded. Per-step
cost becomes flat instead of growing with the segment (RTF 1.06 -> 0.26
on 62.6s of continuous speech), but accuracy requires ~8s chunks because
the audio encoder needs its 8s attention window, so text only appears
every 8s. The incremental mode stays the default.

新增追加式流式:每块音频作为一个新的对话轮追加,前缀不重算、旧音频不重
编码,单步成本不再随段落增长(连续 62.6s 语音 RTF 从 1.06 降到 0.26)。
但准确率要求 chunk≈8s,否则会被截断,所以出字粒度是 8s;增量式仍为默认。
This commit is contained in:
blumia committed 2026-10-12 14:22:29 +08:00
1 parent 7e5257429a
commit 85b206f1cd
10 files changed
+330 -18

No files matched your search

+15
View File
@@ -18,6 +18,7 @@ public:
// 记录(供断言用)
std::vector<std::string> prefixesSeen;
std::vector<std::string> turnsSeen;
std::vector<size_t> samplesSeen;
int resets = 0;
@@ -52,6 +53,20 @@ public:
return s;
}
lc::AsrStreamStep streamAppendTurn(const float *, size_t nSamples, int) override
{
samplesSeen.push_back(nSamples);
lc::AsrStreamStep s;
s.ok = true;
if (call_ < scripted.size()) s.newText = scripted[call_];
++call_;
turnsSeen.push_back(s.newText);
s.newTokens = 1;
s.inferSeconds = 0.001;
return s;
}
void streamReset() override { ++resets; }
std::string dropLastTokens(const std::string & text, int dropLast) const override