What Are LLM Agents Actually Learning on Low-Budget Molecular Optimization Benchmarks?
Paper · https://arxiv.org/abs/2611.08522
arXiv:2611.08522v1 Announce Type: new Abstract: Recent work reports large gains for LLM agents on low-budget molecular optimization. We measure the contributions of online optimization, retrievable task information, and permissive oracle landscapes. On PMO at 50 oracle calls, a GPT-4o mini agent's sum AUC falls from 14.44 to 4.85 after task names and curated reference compounds are hidden. An identity-blinded condition that states only the oracle form (e.g., similarity to a hidden reference) scores 5.39, while a deterministic retrieval-only baseline with no LLM reaches 12.16 (84% of the full agent). A full 23-task 2x2 ablation and a three-seed replication on eight tasks support this interpretation. On a protein-targeted benchmark with an AutoDock Vina composite oracle, a single-edit random walk is a strong lower bound; across three seeds it obtains composite 0.636 +/- 0.008 and docking improvement -1.11 +/- 0.04 kcal/mol. As a positive control, a retrieval-free population agent exceeds REINVENT at a matched 50-call AUC budget, with its strongest result at 100 calls. These results motivate two inexpensive sanity checks: retrieval-safe PMO prompts and a random-walk floor for docking-based optimization.
First seen by Oriel on .
AI summary
The paper finds that reported LLM-agent gains on low-budget molecular optimization largely reflect retrievable task information and permissive oracle landscapes rather than online optimization.
- On PMO with 50 oracle calls, hiding task names and curated reference compounds reduces a GPT-4o mini agent’s sum AUC from 14.44 to 4.85.
- A deterministic retrieval-only baseline without an LLM scores 12.16, reaching 84% of the full agent, while an identity-blinded condition scores 5.39.
- For an AutoDock Vina composite oracle, a single-edit random walk achieves 0.636 +/- 0.008 composite and -1.11 +/- 0.04 kcal/mol docking improvement.
- A retrieval-free population agent exceeds REINVENT at a matched 50-call AUC budget, with its strongest result at 100 calls.
- The authors recommend retrieval-safe PMO prompts and a random-walk floor as inexpensive sanity checks for docking-based optimization benchmarks.
#llm agents #molecular optimization #autodock vina #benchmarking
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| arXiv AI Papers | oai:arXiv.org:2611.08522v1 | https://arxiv.org/abs/2611.08522 Read discussion |