Oriel — AI & technology worth noticing, every day.

prometheus-eval/into-the-unknown

Dataset · https://huggingface.co/datasets/prometheus-eval/into-the-unknown

Into the Unknown Dataset Summary Into the Unknown tests whether AI agents can build on a scientist's work and write a follow-up paper. Each of its 15 tasks gives an agent a published (or recently submitted) paper and a follow-up research direction proposed by the paper's own author; the agent must carry out the research and write a follow-up paper that is honest, complete, and meets the scientist's bar. This dataset holds the 15 tasks and everything nine agents produced on them:… See the full description on the dataset page: https://huggingface.co/datasets/prometheus-eval/into-the-unknown.

First seen by Oriel on .

AI summary

Into the Unknown is a benchmark testing whether AI agents can conduct author-proposed follow-up research and produce honest, complete scientific papers.

  • The dataset contains 15 tasks, each pairing an original paper with a follow-up research direction proposed by its author.
  • Nine agents produced 135 submissions, including papers, full execution trajectories, and written files; four GLM-5.3 runs produced no PDFs.
  • The benchmark includes original papers, task specifications, agent outcomes, trajectories, filesystem archives, indexes, rubrics, and grader tooling.
  • Agents are evaluated on open-ended research ability, with leaderboard results, rubric verdicts, and instructions for submitting new papers.
  • Tasks span published papers from 2024 or earlier and papers submitted in 2026, across different domains, venues, and laboratories.

#ai agents #scientific research #benchmarks #hugging face datasets

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Datasets: 11, latest observation ; +1 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Datasets: 2,096, latest observation ; +107 over observed period since

Aliases

SourceExternal IDURL
Hugging Face Datasetsprometheus-eval/into-the-unknownhttps://huggingface.co/datasets/prometheus-eval/into-the-unknown Read discussion