Oriel — AI & technology worth noticing, every day.

autotrust/AutoTrust-JEV-27B-v1.1

AI model · https://huggingface.co/autotrust/AutoTrust-JEV-27B-v1.1

First seen by Oriel on .

AI summary

AutoTrust-JEV-27B v1.1 assigns probabilities to supplied answer options and reports a 67.31 skill index across 37 Decision Index 0.3 benchmarks, with strongest results in tools and automation.

  • The completed local evaluation covered 140,178 scoreable requests; results are self-reported and pending maintainer verification.
  • Tools & Automation scored 85.19 skill points, while Knowledge & Reasoning scored 55.16 and Arts & Human Taste scored 51.08.
  • The model outputs option probabilities for text-only choice and yes/no questions; its tool-selection adapters are not end-to-end autonomous-agent evaluations.
  • HLE was close to chance, and the page cautions that the evaluation does not establish probability calibration or suitability for high-stakes use.
  • Evaluation used vLLM 0.31.0 on one NVIDIA B300; reported median latency was 64.8 ms per request under concurrent local testing. The model is licensed Apache-2.0.

#language models #decision models #benchmarking #vllm

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Models: 1,406, latest observation ; +140 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Models: 140,007, latest observation ; +6,895 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Modelsautotrust/AutoTrust-JEV-27B-v1.1https://huggingface.co/autotrust/AutoTrust-JEV-27B-v1.1 Read discussion