Project Request: Decision Models as Guardrails #633

Project Information

• Project name: Decision Models as Guardrails

• Github URL: https://github.com/raxITlabs/decision-models-as-guardrails

• Homepage URL: https://dm-as-guardrails.raxitlabs.com

• Logo URL:

Category

• Inference
  • Inference Engine
  • Inference Platform
  • Middleware
  • LLM Router
  • AI Gateway
  • Output
  • Simulator
  • Benchmark
• Orchestration
  • Application Framework
  • Agent Framework
  • Evolutionary Framework
  • RAG
  • Workflow
• Runtime
  • AI Terminal
  • AI Agent
  • Chatbot
  • Code Agent
  • Database
  • Evolve Agent
  • Observation
  • Sandbox
  • Tool
• Training
  • Framework
  • FineTune
  • RLHF
  • Agentic RL
  • Benchmark
  • Workflow

Fit and disclosure

Decision Models as Guardrails is an open benchmark that asks whether a decision model can replace a managed guardrail service. 19 systems answer the same 9,975 labelled guardrail checks across six suites (content, prompt attacks, denied topics, profanity, PII, grounding). Every system uses one fixed 0.5 blocking threshold. Results report balanced accuracy with 95% bootstrap intervals and USD per 1,000 checks at list price. Per-row results, the scoring contract and run scripts are public. Code is Apache-2.0, and the dataset is on Hugging Face: https://huggingface.co/datasets/raxITLabs/decision-models-as-guardrails

raxIT Labs, my company, built and maintains it. No logo URL given, so default.png is fine.