跳到主要内容Skip to main content

人工复核队列Human Review Queue

Human Review Queue

  • 智能体Agents
  • 第三站 · 用深Station 3 · Going Deeper
  • 引用核验 2026-11-06refs checked 2026-11-06
解决什么问题What problem it solves

机器判定处理不了的那部分怎么排队给人看、人看完的结论回到哪里。How the cases a classifier cannot settle get queued for a person, and where the human verdict goes afterwards.

这是什么What it is

队列有两种:拦下来给人复核、执行前等人批准。Microsoft Agent Framework 的工作流用 request 与 response 机制实现第二种:执行到请求点就暂停并发出一个请求事件,外部回复以后才继续;需要批准的工具调用走的是同一条路。

暂停必须能恢复。同一份文档写明可以打 checkpoint,待处理请求会一起进 checkpoint 状态,恢复时请求会重新发出来。也就是说队列不能只放在内存里,进程重启以后还得知道谁在等人。

Roblox 的公开文档给的是第一种队列的样子:资产同时接受人工与自动审核,既有主动也有响应式;还在审核队列里的资产对用户不可见,直到审核通过。

队列本身要能被度量:等待时长、机器标记的命中率、人工翻案的比例,以及这些结论怎么反过来改阈值。各家都没有公布统一口径,只能自己定义并留下记录,否则说不清这条链路是在变好还是只是变忙。

There are two queues: content pulled aside for review, and actions waiting for approval before they run. Microsoft Agent Framework implements the second with a request-and-response mechanism — execution reaches a request port, pauses, emits a request event, and only continues when an outside reply arrives; tool calls that need approval follow the same path.

A pause has to be resumable. The same document states that pending requests are saved as part of a checkpoint and are re-emitted when the run is restored, which means the queue cannot live in memory alone: after a restart it still has to know who is waiting on whom.

Roblox's creator documentation shows the first kind. Assets get both human and automated moderation, proactively and reactively, and an asset still sitting in the moderation queue stays invisible to users until it is approved.

The queue itself needs measuring: how long items wait, what share of machine flags hold up, how often a reviewer overturns one, and how those verdicts feed back into the threshold. There is no published standard here, so you define it and keep the records — otherwise you cannot tell improvement from merely being busy.

为什么需要知道Why it matters

把「加人工复核」写成一行需求最容易,难的是积压、权限、复审记录和结论回流。这四件事想清楚再上,不然队列会变成流程里没人认领的那一格。Adding human review is the easiest line in a spec and the hardest thing to run: backlog, permissions, review records and conclusions that flow back. Settling those four first keeps the queue from becoming the step nobody owns.

出处Sources
  • Microsoft Agent Framework:Human-in-the-loop 工作流(RequestPort、暂停与恢复、工具批准、checkpoint)Agent Framework: human-in-the-loop workflows ↗
  • Roblox 创作者文档:资产的人工与自动审核队列Roblox Creator Docs: asset moderation ↗
  • CLI 的权限询问机制CLI permission prompts ↗