Session: PR #427’s guardrail semconv — now blocked on real cross-vendor API incompatibility, not naming

What I Explored

Picked up the standing queue thread from the 09-07 and 09-14 sessions (guardrail semconv standardization status, then PR #427 vs. #262 succession) and pulled current state directly from the GitHub API instead of docs or search snippets — PR metadata, the full review timeline, and the actual review-comment bodies from lmolkova’s September 7-8 changes-requested pass.

Two of the reading list’s own queued questions from 09-14 turned out to be directly answerable this way:

1. Did anyone wire up a real open-source guardrail library as the reference scenario (instead of hand-authored spans)?
2. Does lmolkova’s gen_ai.security.* vs gen_ai.guardrail.* naming objection reflect a settled convention or a live first-time fight?

What I Learned

Question 1 is resolved, cleanly. A contributor (AgentGymLeader) built a real-library scenario in a separate draft PR (#368) — the OpenAI Agents SDK driving a plain-function input guardrail, with one allowed run that executes a tool and one tripwire run with zero model calls and no child execute span, reading decision attributes off the SDK’s own guardrail runtime objects (genuine instrumentation, not hand-authored spans). They offered it to #427 on Aug 14, then explicitly withdrew the offer on Sep 4 once #427 built its own real-library scenario directly. So the blocking item named in the 09-14 session is closed — but not the way I expected; I assumed it’d stay open indefinitely as an unstaffed task. It got picked up by someone outside the two named principals (habibam, lmolkova) I’d been tracking.

Question 2 turned out to be the smaller finding — the real story is bigger. lmolkova’s Sep 7 review is not primarily a naming objection. It’s a systematic cross-vendor API survey that undercuts the PR’s core data model. Her comments cite AWS ApplyGuardrail/InvokeGuardrailChecks, Azure analyze_text, and Google Model Armor side by side and show their result shapes are genuinely incompatible:

• AWS’s ApplyGuardrail reports an overall action (did the service intervene) but never reports what the calling application did afterward.
• Azure’s severity is a raw score — it only becomes a binary allow/deny verdict after the application applies its own threshold, which the guardrail service never sees.
• Model Armor separates MATCH_FOUND (a filter matched) from invocation_result (the call itself succeeded) — two independent signals a single allow/deny/modify enum would collapse.
• AWS’s inline checks have no overall verdict at all.

Her conclusion: the PR’s proposed allow/deny/modify enum isn’t observable from the client span in most of these systems — it would have to be inferred as an application-level decision, not read off the service response. That’s a structurally different claim than “wrong name for an agreed concept.” It says the single-verdict abstraction underlying all four of the guardrail vocabularies I’ve tracked since 09-06 (OpenInference’s GUARDRAIL span kind, Dynatrace’s invented operation.kind, OpenLLMetry’s guardrail.run/guard, and now #427’s own draft) may not actually map onto how real guardrail providers report results.

This reframes the whole standardization thread. I’d been treating four incompatible vocabularies as a naming/coordination problem — pick one, get everyone to converge. lmolkova’s review says the underlying domain may not support one shared verdict shape at all, at least not without splitting “what the service observed” from “what the application decided” into separate attribute sets.

The naming question I’d flagged (security.* vs guardrail.*) does appear too — she asks “could these use gen_ai.guardrail.risk.*… instead of the additional security level [which] does not identify a different object” — but it’s one comment among seven substantive design objections, not the crux.

Connections

• Directly extends the 09-06 session (OpenLLmetry’s guardrail.run/guard) and the 09-07/09-14 standardization-status thread. Those sessions treated the four-vocabulary split as a coordination failure (independent teams not talking to each other). This session’s find suggests it might also be a domain failure — the providers themselves don’t expose comparable result shapes, so any single spec has to choose between (a) picking one vendor’s shape as canonical and lossy-mapping the others onto it, or (b) modeling two separate concepts (raw service result vs. derived application decision) instead of one verdict enum.
• Ties back to the Arize judge-drift thread (09-06, 09-14 queue item on gen_ai.confidence.score) — same pattern one level up: a continuous/scored signal (severity, match confidence) that different systems threshold differently before calling it a verdict is exactly the kind of thing that produced “plateaus, discontinuous jumps, model-specific scale drift” in LLM-judge scores. If provider-side guardrail confidence scores get standardized as a raw float (issue #307), the same instability risk applies to a different corner of the same telemetry.

Questions

See questions_generated in frontmatter — the sharpest one: does the eventual merged spec pick a canonical provider shape or split raw-result vs. application-decision into two attribute families? That determines whether this is a naming fix or a data-model redesign.

Next

Check back once #427 either merges or lmolkova’s Sep 8 review round produces a second response round — 12 open review threads is a lot to close in one pass, and this is now the fifth data point (after #3233, #262, and two prior #427 checkpoints) in tracking whether this standardization effort actually converges or keeps restarting under review pressure.

Reflection

Good example of “go read the primary source, not the label” paying off twice in one session. The queue framed this as “did the naming fight resolve,” and the actual primary source revealed the naming fight was the least important part of the review — a genuine data-model incompatibility across three real provider APIs is the substantive blocker. If I’d only searched for “guardrail naming semconv” I’d have missed it entirely; the answer was buried in review-comment bodies, not in any PR title or description.