Oriel — AI & technology worth noticing, every day.

What Are LLM Agents Actually Learning on Low-Budget Molecular Optimization Benchmarks?

Paper · https://arxiv.org/abs/2611.08522

arXiv:2611.08522v1 Announce Type: new Abstract: Recent work reports large gains for LLM agents on low-budget molecular optimization. We measure the contributions of online optimization, retrievable task information, and permissive oracle landscapes. On PMO at 50 oracle calls, a GPT-4o mini agent's sum AUC falls from 14.44 to 4.85 after task names and curated reference compounds are hidden. An identity-blinded condition that states only the oracle form (e.g., similarity to a hidden reference) scores 5.39, while a deterministic retrieval-only baseline with no LLM reaches 12.16 (84% of the full agent). A full 23-task 2x2 ablation and a three-seed replication on eight tasks support this interpretation. On a protein-targeted benchmark with an AutoDock Vina composite oracle, a single-edit random walk is a strong lower bound; across three seeds it obtains composite 0.636 +/- 0.008 and docking improvement -1.11 +/- 0.04 kcal/mol. As a positive control, a retrieval-free population agent exceeds REINVENT at a matched 50-call AUC budget, with its strongest result at 100 calls. These results motivate two inexpensive sanity checks: retrieval-safe PMO prompts and a random-walk floor for docking-based optimization.

First seen by Oriel on .

AI summary

The paper finds that reported LLM-agent gains on low-budget molecular optimization largely reflect retrievable task information and permissive oracle landscapes rather than online optimization.

  • On PMO with 50 oracle calls, hiding task names and curated reference compounds reduces a GPT-4o mini agent’s sum AUC from 14.44 to 4.85.
  • A deterministic retrieval-only baseline without an LLM scores 12.16, reaching 84% of the full agent, while an identity-blinded condition scores 5.39.
  • For an AutoDock Vina composite oracle, a single-edit random walk achieves 0.636 +/- 0.008 composite and -1.11 +/- 0.04 kcal/mol docking improvement.
  • A retrieval-free population agent exceeds REINVENT at a matched 50-call AUC budget, with its strongest result at 100 calls.
  • The authors recommend retrieval-safe PMO prompts and a random-walk floor as inexpensive sanity checks for docking-based optimization benchmarks.

#llm agents #molecular optimization #autodock vina #benchmarking

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
arXiv AI Papersoai:arXiv.org:2611.08522v1https://arxiv.org/abs/2611.08522 Read discussion