AgentGradePortable rubrics for agent quality — offline and online.
Define what good looks like, run offline packs, compare to online monitoring — decision-support grades, not a pass certificate.
No credit card required · Cancel anytime
Key Takeaways
- Rubrics that travel across eval harnesses.
- Offline packs vs online monitoring clarity.
- Honesty: grading decision-support — not a guarantee agents never fail.
Rubric first. Packs next.
Decision-support themes — not a pass certificate.
AgentGrade — definition
AgentGrade is an evaluation harness for AI agents that scores trajectories against task rubrics, safety checks, and regression sets, so teams can gate releases on measured quality instead of vibes.
What this is NOT
- Not a benchmark lab, model leaderboard, or pass certificate.
- Not a guarantee that scores match every harness or that a high score means production-ready.
- Not a policy engine, pentest suite, or compliance certificate.
Controller / Processor — When traces mention personal data, treat scores as decision-support and follow your own privacy process.
Harness changes. Rubrics break.
One-off checklists
Rubrics die when the harness moves.
Offline-only
Live drift is invisible.
Online-only
Hard scenarios never get stress-tested.
Overclaim tools
Promises of certified agents erode trust.
Describe. Grade. Act.
Decision-support grades — not a pass certificate.
Portable rubrics
Criteria that travel.
Offline + online lens
Two questions, two packs.
Team-ready
Built for release reviews.
Honest demo vs live
Demo is labelled; live needs a configured key where applicable.
Everything you need for a first-pass grade
Built for teams shipping agents who need rubric reviews.
Quality score
Criteria that travel.
Faithfulness check
Two questions, two packs.
Task-success rate
Built for release reviews.
Exportable rubric
Demo is labelled; live needs a configured key where applicable.
From run to grade in 3 steps
Describe
Describe the agent and sample run.
Grade
Get a rubric-based grade draft.
Act
Fix gaps — humans own the release gate.
Describe. Grade. Act.
Decision-support grades — not a pass certificate.
One-off checklists
Rubrics die when the harness moves.
Portable rubrics — Criteria that travel.
Try it on this page
or generate below.
Saved history
No testimonials yet, by design
We don't publish customer counts or invented quotes. Run the tool and read a real evaluation — that's our proof.
Score traces on three axes
Quality, faithfulness, and task success — try a labelled demo without a paid plan.
Pass rates you can compare
Each run returns a scorecard, run id, and ruleset version so releases stay comparable.
Honest eval limits
Free 3/day · 15/mo; Pro 40/day · 300/mo. No hidden caps. Demo mode is labelled. Not a pass certificate.
Honest boundaries
AgentGrade drafts a grade. No certification agents never fail.
Simple plans that scale
All paid plans include
- Grade studio
- Portable rubrics
- Trace scorecards (quality, faithfulness, task success)
- Email support on paid plans — no hidden add-ons for core studio use
- Limited daily AI runs (3/day · 15/mo)
- Studio demo (no extra LLM fee)
- AI included: 40 runs/day · 300/mo (fair use)
- No separate ChatGPT subscription required
- Priority support · 2 months free with yearly
Feature comparison
What each plan includes — quotas reflect the enforced fair-use limits.
| Dimension | Free | Pro | Enterprise |
|---|---|---|---|
| Usage | |||
| AI runs | 3/day · 15/mo | 40/day · 300/mo | 200/day · 3,000/mo |
| Report export | Watermarked | Clean export | Clean export |
| Audit log | Not included | ✓ Included | ✓ Included |
| Product | |||
| Quality score | ✓ Included | ✓ Included | ✓ Included |
| Faithfulness check | ✓ Included | ✓ Included | ✓ Included |
| Task-success rate | ✓ Included | ✓ Included | ✓ Included |
| Regression-aware flag | ✓ Included | ✓ Included | ✓ Included |
| BYOK (your LLM key) | Not included | Not included | ✓ |
| SSO | Not included | Not included | Roadmap |
| Support & compliance | |||
| Support | Community | Priority | Custom SLA |
Already paid? Unlock Pro with your license
Licenses are issued manually after your payment is confirmed: we generate one per paid order and send it to the purchase email, typically within a few hours. Enter it below to activate Pro on this device — no account needed. Nothing yet? Write to lixingliangsy@163.com and reference your order email.
Your questions, answered
What does AgentGrade score?
Agent runs against portable rubrics — offline and online lenses. Decision-support grades, not a certification agents never fail.
Is this legal advice or a certification?
No. Outputs are decision-support grades. Have your eval owner review before you gate a release. We do not certify that agents never fail.
Can I cancel anytime?
Yes. Self-serve plans cancel anytime; access continues until period end.
Do I need a credit card to start?
No. Start with email signup or Demo mode in the studio, then upgrade when ready.
Do I need my own ChatGPT / OpenAI subscription?
No for Free/Pro. AI runs are included in your plan (fair use) via our platform key. Enterprise can optionally bring your own OpenAI key (BYOK) — configure it at /settings (key stays server-side only).
What is Fair Use / what if I hit the AI limit?
Free is 3 runs/day and 15/month. Pro is 40 runs/day and 300/month. If you hit the fair-use limit, Studio returns a labelled demo result until the daily or monthly reset — or use Enterprise BYOK for heavier volume.
Is checkout secure?
Payments are processed by Waffo Pancake (merchant of record).
What happens after I pay?
You receive access confirmation; fulfillment is tracked via webhook + order logs.
Quick answers
- Quality score
- Faithfulness check
- Task-success rate
- Exportable eval rubric
- Pricing starts at $0 (Free).
More questions
What inputs?
Agent description plus sample run / eval notes.
Certification?
No. Decision-support grades — not a pass certificate.
Offline vs online?
Offline packs stress scenarios; online watches live drift.
Demo vs live?
Demo is labelled; live narrative needs a configured AI key.
Who is it for?
Teams shipping agents who need rubric-based reviews.
Primary sources?
Agent eval practice notes — decision-support framing.
How it compares
| Dimension | Manual | AgentGrade |
|---|---|---|
| Rubric | Ad-hoc vibe checks | Repeatable quality + faithfulness scores |
| Consistency | Varies by reviewer | Same eval pack every run |
| Output | Slack opinions | Exportable scorecard |
| Best for | Full offline benchmark suite | First-pass agent quality triage |
When not to use: Do not treat AgentGrade as a production SLA certificate — it is evaluation decision-support.
- Eval rubrics that travel — Scores you can reuse across prompts.
- Wave 1 launch — AgentGrade in the agent ops cluster.
- Offline eval vs online monitoring — When packs and live traces answer different questions.
People also search
Click a tab to expand it; click again to hide. All content stays on the page for search engines.
Start smarter today
Join builders using AgentGrade. Free to try — no card needed.
Trusted · Cancel anytime