Oriel — AI & technology worth noticing, every day.

CrossSWE/CrossSWE-Bench

Dataset · https://huggingface.co/datasets/CrossSWE/CrossSWE-Bench

CrossSWE-Bench Software Engineering Across Repository Boundaries Dataset · Quick Start · Evaluation · Data Format CrossSWE-Bench is a benchmark of 171 cross-repository software engineering tasks drawn from real upstream projects. Tasks require coordinated changes across related repositories, such as a library and its consumer, an API and its implementation, or a build tool and its dependency. Each instance includes its problem statement, pinned repository revisions, reference… See the full description on the dataset page: https://huggingface.co/datasets/CrossSWE/CrossSWE-Bench.

First seen by Oriel on .

AI summary

CrossSWE-Bench is a benchmark of 171 real-world software engineering tasks that require coordinated changes across related repositories, with pinned environments and task-specific evaluation materials.

  • Tasks span 350 repository entries across 165 distinct upstream repositories and include problem statements, pinned revisions, reference patches, tests, and evaluation contracts.
  • The dataset provides immutable Docker image references, allowing users to pull and open task environments without reconstructing them.
  • Evaluation keeps reference patches and evaluator-only tests outside the agent workspace and uses task-specific test materials and adapters.
  • The release records public-test, editable-source rebuild, image-audit, and formal endpoint evidence; some outcomes are task-caused failures or unexecuted cases, not passes.
  • Licensing is mixed: benchmark contributions have a limited CC-BY-4.0 grant, while upstream code and derived materials retain their original terms.

#software engineering #benchmark datasets #docker #repository tooling

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Likes

Total likes reported by the source.

  • Hugging Face Datasets: 14, latest observation ; +3 over observed period since

Downloads (30d)

Rolling 30-day downloads. A change in this value is not new downloads during the observed period.

  • Hugging Face Datasets: 1,912, latest observation ; +1,798 over observed period since

Aliases

SourceExternal IDURL
Hugging Face DatasetsCrossSWE/CrossSWE-Benchhttps://huggingface.co/datasets/CrossSWE/CrossSWE-Bench Read discussion