jev-benchmark

What Jev does in jev-benchmark

The project describes itself this way: Benchmarks and a playground for TypeSafe's Jev (System One) model: chess, and who-is-the-player-talking-to for speech-to-text game NPCs. In the Benchmarks & research category, Jev typically decides labels and probabilities on fixed test sets, so accuracy, calibration and latency can be measured.

Verification

Verified. We confirmed this repository and its Jev relevance through the GitHub API and the source lists below. A specific file link has not been recorded yet.

How Benchmarks & research projects use Jev

Benchmarks and research projects test Jev rather than just use it. They compare it with language models and other classifiers, check whether its probabilities are well calibrated, and explore where the decision-model approach breaks down.

Also listed in: Voice & Conversation, Games & simulations.

Related Jev projects in Benchmarks & research

1. jev-lmy0usaf/jev-lmA word-level generation experiment that asks Jev to select words or verify locally drafted continuations.★ 6
   TypeScript
2. daf-jevdocxology/daf-jevA Python toolkit for Jev requests, batch evaluation, calibration and MCP access.★ 5
   Python
3. jev-little-airwayslbotinelly/jev-little-airwaysAn island-airport simulator using Jev for routes, yielding, emergency broadcasts and landing order.★ 5
   HTML
4. jev-does-not-play-diceKantaHayashiAI/jev-does-not-play-diceExperiments on Jev’s probability calibration, uncertainty reporting, and forecast probability preservation.★ 4
   JavaScript
5. jev-for-engineersFoadsf/jev-for-engineersEight mechanical and electrical engineering experiments using Jev for task routing, log checks and component selection.★ 3
   Python
6. jev-behavior-studyRINNECODER/jev-behavior-studyAn independent Jev 1.13.0 behavior study recording successes and failures across question framing, input conditions and games.★ 3
   Python

See all 107 Benchmarks & research projects →

Where we found it

• heyjunpenn/awesome-jev