Oriel — AI & technology worth noticing, every day.

facebook/hackbench

Dataset · https://huggingface.co/datasets/facebook/hackbench

HackBench HackBench is a benchmark for numbered command localization of reward hacking in long-horizon ML research agent trajectories. Each record contains the original trajectory, a ready-to-use numbered monitor input, and audited gold command indices. This release contains 169 HACK records across 13 tasks: 41 natural and 128 synthetic hacking examples. Source agent Records GPT-5.5 102 GLM-5.2 67 Total 169 Project: RAM. Blog Post: HackBench: Detecting and… See the full description on the dataset page: https://huggingface.co/datasets/facebook/hackbench.

First seen by Oriel on .

AI summary

HackBench is a benchmark for detecting and localizing reward-hacking commands in long-horizon ML research agent trajectories, containing 169 audited HACK records across 13 tasks.

  • The release includes 41 natural and 128 synthetic examples generated by GPT-5.5 and GLM-5.2.
  • Each JSONL record provides the original trajectory, numbered monitor input, audited gold command indices, task instructions, gold command strings, and source-agent settings.
  • Reference preprocessing removes historical detector feedback, retains blocked attempts, and labels executable command blocks with one-based [CMD n] identifiers.
  • Original transcripts range from 73,412 to 3,834,163 Unicode characters, while the complete JSONL file is 408,528,473 bytes.
  • The dataset has only a test split; evaluations should average per-trajectory precision, recall, and F₁, while reporting any chunking or long-context strategy.

#reward hacking #ml agents #benchmarking #long context

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Datasets: 18, latest observation ; +4 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Datasets: 125, latest observation ; +37 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Datasetsfacebook/hackbenchhttps://huggingface.co/datasets/facebook/hackbench Read discussion