护栏与内容过滤Guardrails and Content Filtering
Guardrails and Content Filtering
在模型外面加一层检查和拦截，写清它拦什么、按什么阈值拦、拦不住什么。What an outer layer of checks actually blocks, at what threshold, and what it still misses.
护栏可以放在输入侧、输出侧，也可以夹在工具调用之间。检查手段有多种：关键词与阻断列表、分类器打分、再用一个模型判断。微软那份文档把 guardrail 拆成三问：检测什么风险、在哪里扫、命中以后模型或 agent 应该怎么回应。
内容过滤交给你的是类别加严重级别，不是一个布尔值。Azure 的文本与图像分析会按 Hate、Sexual、Violence、SelfHarm 四类返回分级，阈值和动作由你配置，同一个分数在两个产品里完全可能选不同处置。
专门针对注入的护栏也已经产品化。Prompt Shields 扫的是用户输入里有没有针对语言模型的攻击；另有一个 task adherence API 看 agent 的工具调用是否偏离或提前于用户意图。
护栏会漏，也会有代价。部分能力只提供英文、多个功能处于 preview 并已宣布退役时间（图像与文本联合分析将在 2027-04-30 退役）。官方文档要你自己测；误伤以后走哪条申诉路径，也要在上线前定下来。
Guardrails sit on the input side, on the output side, or between tool calls. The check itself can be a keyword list, a classifier score, or a second model making the judgement. Microsoft's framing asks three things of every guardrail: which risk to detect, where to scan, and how the model or agent should respond on a hit.
Content filtering hands you a category and a severity, not a yes-or-no. Azure's text and image analysis returns graded levels across hate, sexual, violence and self-harm; you set the threshold and the action, so the same score can be handled completely differently by two products.
Injection-specific guards are productised too. Prompt Shields scans user input for attack patterns aimed at a language model, and a task-adherence API looks for agent tool use that is misaligned, unintended or premature relative to what the user asked.
Guards leak and they cost something. Some features are English-only, several are preview and already carry retirement dates — image-with-text analysis is scheduled for 30 April 2027. Microsoft tells you to test it yourself, and you need an appeal path for false positives decided before launch.
「我们加了护栏」这句话本身不含信息。要问的是拦在哪一侧、阈值谁定的、命中率与误伤率怎么记录、被拦的人下一步做什么。这几个答案才决定它值不值得信。The sentence we added guardrails carries no information by itself. What decides whether to trust it is which side blocks, who set the threshold, whether hit and false-positive rates are recorded, and what happens to the person whose request got blocked.