SteelAgent: LLM-Orchestrated, Physics-Informed Steel Property Prediction with Audited Generalization and Calibrated Uncertainty
Paper · https://arxiv.org/abs/2611.06953
arXiv:2611.06953v1 Announce Type: new Abstract: Steel property models can perform well on random test splits even when they extrapolate poorly to unfamiliar alloys. SteelAgent couples metallurgical descriptors and machine-learning models with a language-model interface for property prediction, uncertainty reporting and inverse design. We examine how the choice of held-out unit changes the conclusions drawn from SteelBench tensile-strength evaluations. For gradient boosting, pooled out-of-fold $R^2$ fell from 0.859 on a random split to 0.113 when steel families were held out. Empirical coverage of nominal 90% prediction intervals ranged from 91.0% under random splitting to 37.6% on held-out sources. The effect of physics-derived features depended on the evaluation. They increased gradient-boosting $R^2$ by 0.2534 in a 17-family holdout analysis and decreased it by 0.358 in a separate four-grade laboratory analysis. We report group-level errors and a synthetic regression experiment illustrating how calibration can deteriorate on held-out groups. These results motivate reporting the evaluation population alongside a property estimate or compliance score.
First seen by Oriel on .
AI summary
SteelAgent combines metallurgical descriptors, machine learning, and a language-model interface, showing that steel-property accuracy and uncertainty calibration can deteriorate sharply when evaluating unfamiliar steel families.
- SteelAgent supports property prediction, uncertainty reporting, and inverse design through a language-model interface coupled to metallurgical descriptors and machine-learning models.
- Gradient boosting achieved pooled out-of-fold R² of 0.859 on random splits but only 0.113 when steel families were held out.
- Nominal 90% prediction-interval coverage ranged from 91.0% under random splitting to 37.6% on held-out sources.
- Physics-derived features improved gradient-boosting R² by 0.2534 in a 17-family holdout but reduced it by 0.358 in a separate four-grade laboratory analysis.
- The results motivate reporting the evaluation population alongside each property estimate or compliance score.
#steel #machine learning #uncertainty calibration #inverse design
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| arXiv AI Papers | oai:arXiv.org:2611.06953v1 | https://arxiv.org/abs/2611.06953 Read discussion |