facebook/hackbench
Dataset · https://huggingface.co/datasets/facebook/hackbench
HackBench HackBench is a benchmark for numbered command localization of reward hacking in long-horizon ML research agent trajectories. Each record contains the original trajectory, a ready-to-use numbered monitor input, and audited gold command indices. This release contains 169 HACK records across 13 tasks: 41 natural and 128 synthetic hacking examples. Source agent Records GPT-5.5 102 GLM-5.2 67 Total 169 Project: RAM. Blog Post: HackBench: Detecting and… See the full description on the dataset page: https://huggingface.co/datasets/facebook/hackbench.
First seen by Oriel on .
AI summary
HackBench is a benchmark for detecting and localizing reward-hacking commands in long-horizon ML research agent trajectories, containing 169 audited HACK records across 13 tasks.
- The release includes 41 natural and 128 synthetic examples generated by GPT-5.5 and GLM-5.2.
- Each JSONL record provides the original trajectory, numbered monitor input, audited gold command indices, task instructions, gold command strings, and source-agent settings.
- Reference preprocessing removes historical detector feedback, retains blocked attempts, and labels executable command blocks with one-based [CMD n] identifiers.
- Original transcripts range from 73,412 to 3,834,163 Unicode characters, while the complete JSONL file is 408,528,473 bytes.
- The dataset has only a test split; evaluations should average per-trajectory precision, recall, and F₁, while reporting any chunking or long-context strategy.
#reward hacking #ml agents #benchmarking #long context
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Likes
Total likes reported by the source.
- Hugging Face Datasets: 18, latest observation ; +4 over observed period since
Downloads (30d)
Rolling 30-day downloads. A change in this value is not new downloads during the observed period.
- Hugging Face Datasets: 125, latest observation ; +37 over observed period since
Aliases
| Source | External ID | URL |
|---|---|---|
| Hugging Face Datasets | facebook/hackbench | https://huggingface.co/datasets/facebook/hackbench Read discussion |