autotrust/AutoTrust-JEV-27B-v1.1
AI model · https://huggingface.co/autotrust/AutoTrust-JEV-27B-v1.1
First seen by Oriel on .
AI summary
AutoTrust-JEV-27B v1.1 assigns probabilities to supplied answer options and reports a 67.31 skill index across 37 Decision Index 0.3 benchmarks, with strongest results in tools and automation.
- The completed local evaluation covered 140,178 scoreable requests; results are self-reported and pending maintainer verification.
- Tools & Automation scored 85.19 skill points, while Knowledge & Reasoning scored 55.16 and Arts & Human Taste scored 51.08.
- The model outputs option probabilities for text-only choice and yes/no questions; its tool-selection adapters are not end-to-end autonomous-agent evaluations.
- HLE was close to chance, and the page cautions that the evaluation does not establish probability calibration or suitability for high-stakes use.
- Evaluation used vLLM 0.31.0 on one NVIDIA B300; reported median latency was 64.8 ms per request under concurrent local testing. The model is licensed Apache-2.0.
#language models #decision models #benchmarking #vllm
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Likes
Total likes reported by the source.
- Hugging Face Models: 1,406, latest observation ; +140 over observed period since
Downloads (30d)
Rolling 30-day downloads. A change in this value is not new downloads during the observed period.
- Hugging Face Models: 140,007, latest observation ; +6,895 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Models | autotrust/AutoTrust-JEV-27B-v1.1 | https://huggingface.co/autotrust/AutoTrust-JEV-27B-v1.1 Read discussion |