Oriel — AI & technology worth noticing, every day.

Knowledge Graphs Quality Assessment in the Era of LLMs: A Conceptual Framework

Paper · https://arxiv.org/abs/2610.31684

arXiv:2610.31684v1 Announce Type: new Abstract: Knowledge Graphs (KGs) have been popularized during the last few decades and are increasingly constructed and consumed by Large Language Models (LLMs). Various applications therefore rely on KGs to provide concise, complete, accurate, faithful, fresh, and traceable answers. However, what is the quality of those KGs? In which cases should a KG be used, and can it be trusted to ground an LLM? How might it be evaluated? We reviewed the literature on quality assessment (QA) of data, information, linked data, KGs, and the intersection of KGs and LLMs. We extended the current state-of-the-art frameworks to a catalog of 24 quality dimensions (QDs) and various quality metrics (QMs). Furthermore, we propose a general-purpose, customizable to a domain or task, and practical QA framework for assessing the quality of KGs.

First seen by Oriel on .

AI summary

The paper proposes a practical, customizable framework for assessing knowledge graph quality as KGs are increasingly used to ground LLM answers.

  • The authors reviewed quality assessment research on data, information, linked data, KGs, and the intersection of KGs and LLMs.
  • They extend existing frameworks into a catalog of 24 quality dimensions and associated quality metrics.
  • The framework is intended to be adaptable to specific domains or tasks.
  • The motivation is to evaluate whether KGs provide concise, complete, accurate, faithful, fresh, and traceable answers.

#knowledge graphs #llm evaluation #data quality

Generated by gpt-5.6-sol. Check the original for details.

Aliases

SourceExternal IDURL
arXiv AI Papersoai:arXiv.org:2610.31684v1https://arxiv.org/abs/2610.31684 Read discussion