Oriel — AI & technology worth noticing, every day.

autotrust/AutoTrust-Decider-Pro-Vision-27B-v1.1

AI model · https://huggingface.co/autotrust/AutoTrust-Decider-Pro-Vision-27B-v1.1

First seen by Oriel on .

AI summary

AutoTrust-JEV-27B v1.1 is a text decision model that achieved a self-reported 67.31 public skill index across 37 Decision Index 0.3 benchmarks.

  • Tools & Automation led its five evaluated areas with 85.19 skill points, including BFCL at 96.25 and CLINC150+OOS at 97.06.
  • The evaluation completed 140,178 scoreable requests locally using vLLM 0.31.0 on one NVIDIA B300, with verification by maintainers still pending.
  • The decision engine restricts output to one option-code token and converts processed log-probabilities into probabilities for text-only choice and yes/no questions.
  • Median local latency was 64.8 ms, mean latency was 178.8 ms, and p95 was 800.5 ms under 16-client concurrency.
  • Results do not establish probability calibration, autonomous-agent performance, high-stakes suitability, or deployment readiness; representative application validation remains necessary.

#decision models #llm evaluation #vllm #classification

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Models: 1,515, latest observation ; +107 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Models: 147,541, latest observation ; +7,534 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Modelsautotrust/AutoTrust-Decider-Pro-Vision-27B-v1.1https://huggingface.co/autotrust/AutoTrust-Decider-Pro-Vision-27B-v1.1 Read discussion