Oriel — AI & technology worth noticing, every day.

google/physical-engineerings-last-exam

Dataset · https://huggingface.co/datasets/google/physical-engineerings-last-exam

Physical-Engineering’s Last Exam (Dev Set) Physical-Engineering’s Last Exam (PLE) is a benchmark of long-horizon engineering tasks built by licensed practicing engineers across 4 disciplines. In each task, an agent works inside a realistic project workspace containing drawings, specifications, spreadsheets, solver scripts, and CAD/BIM or simulation tools to produce engineering deliverables: code, calculations, models, schedules, and technical reports. This repository contains… See the full description on the dataset page: https://huggingface.co/datasets/google/physical-engineerings-last-exam.

First seen by Oriel on .

AI summary

Physical-Engineering’s Last Exam (PLE) is a gated benchmark evaluating agents on long-horizon, realistic engineering projects across structural, civil, electrical, and mechanical disciplines.

  • The public development set contains 29 of 85 tasks, created by licensed practicing engineers and packaged as self-contained Harbor task folders.
  • Agents manipulate drawings, specifications, spreadsheets, solver scripts, and CAD/BIM or simulation tools to produce code, calculations, models, schedules, and reports.
  • Each task permits six hours, restricts internet access to model API calls, and receives a 0-to-1 score from an isolated verifier container.
  • The dev set includes 4 structural, 9 civil, 9 electrical, and 7 mechanical tasks; reported scores average three runs across all 29 tasks.
  • The recommended competition runner downloads the dataset, enforces limits, builds task-specific verifier images, grades tasks, and exports submission artifacts; standalone Harbor supports 12 tasks directly.

#ai agents #engineering #benchmarks #harbor

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Datasets: 81, latest observation ; +70 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Datasets: 127,161, latest observation ; +125,865 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Datasetsgoogle/physical-engineerings-last-examhttps://huggingface.co/datasets/google/physical-engineerings-last-exam Read discussion