How Amazon Marketplace evaluates content quality with Amazon SageMaker AI
Learn how Amazon Marketplace evaluates A+ Content quality at scale on Amazon SageMaker AI by rendering each product page as a single snapshot, scoring it with a vision-language model fused with OCR readability signals in a CatBoost classifier, and using SHAP to turn each decision into prioritized, actionable recommendations for selling partners.
First seen by Oriel on .
AI summary
Amazon Marketplace describes a SageMaker AI pipeline that combines holistic VLM assessments, OCR readability features, and CatBoost to evaluate A+ Content, achieving 0.94 accuracy and 0.95 F1 on a 5,478-page dataset.
- The system renders each product detail page as one snapshot, then scores Visual Storytelling, Feature Presentation, A+ Layout, and Conversion Optimization with a rubric-prompted VLM.
- OCR adds Text Density, Character Density, Spacing, and Overall Readability signals; combining these with VLM scores improves accuracy by 0.05 over VLM-only CatBoost.
- Five annotators rated 5,478 pages across 10 categories; consensus labels were nearly balanced, and annotator agreement with consensus ranged from 0.845 to 0.915 Cohen’s kappa.
- SHAP identifies features driving each prediction, and an LLM turns the weakest contributions and VLM rationales into prioritized recommendations; revised content scores rose by an average of 1.4 points out of 10.
- Only CatBoost is retrained as annotations or criteria change, avoiding foundation-model fine-tuning; the post reports a separate 499-project experiment with two VLM variants, but its final discussion is truncated.
#amazon sagemaker ai #vision-language models #ocr #catboost
Generated by gpt-5.6-sol. Check the original for details.
Aliases
| Source | External ID | URL |
|---|---|---|
| AWS Machine Learning Blog | e8c83ec5b92498185c84027903905490ba326750 | https://aws.amazon.com/blogs/machine-learning/how-amazon-marketplace-evaluates-content-quality-with-amazon-sagemaker-ai/ Read discussion |