jev-orderby-bench

What Jev does in jev-orderby-bench

The project describes itself this way: Does ORDER BY over a Jev probability put rows in a defensible order? Independent ranking, calibration and invariant measurements of TypeSafe AI's Jev: passes six pre-registered gates on 360 labeled rows, fails four of six on graded product relevance. In the Benchmarks & research category, Jev typically decides labels and probabilities on fixed test sets, so accuracy, calibration and latency can be measured.

Verification

Verified. We confirmed this repository and its Jev relevance through the GitHub API and the source lists below. A specific file link has not been recorded yet.

Topics

• benchmark
• calibration
• duckdb
• information-retrieval
• jev
• llm-evaluation
• python
• ranking
• reproducible-research
• typesafe

How Benchmarks & research projects use Jev

Benchmarks and research projects test Jev rather than just use it. They compare it with language models and other classifiers, check whether its probabilities are well calibrated, and explore where the decision-model approach breaks down.

Related Jev projects in Benchmarks & research

1. jev-calibration-auditjujumilk3/jev-calibration-auditAudits Jev calibration, option-wording effects and Korean judgments through public APIs and datasets.★ 0
   Python
2. jev-transcript-screenerkeduseworku/jev-transcript-screenerDemo and evaluation harness that uses Jev to rank solver transcripts for review by a more costly judge.★ 0
   Python
3. jev-vs-clefrmax-ai/jev-vs-clefLaunch-window comparison harness for Jev and Cloudflare Clef decision models on synthetic decision tasks.★ 0
   Python
4. jev-comparewestsmith-open/jev-compareSide-by-side Jev and Gemini evaluation harness comparing typed decisions, requests, latency, cost and calibration.★ 0
   HTML
5. decision-benchHanno-Labs/decision-benchOpen benchmark runtime for document-grounded decision models.★ 1
   Python
6. jev-synergy-screeningPistachioAIHQ/jev-synergy-screeningA Jev title-and-abstract screening experiment compared with Cohen Abstract Triage labels for an ADHD review.★ 1
   Python

See all 107 Benchmarks & research projects →

Where we found it

• heyjunpenn/awesome-jev
• yibie/awesome-jev