AI

Construction of Paired Knowledge Graph – Text Datasets Informed by Cyclic Evaluation

1 Mins read

[ad_1]

Datasets that pair Knowledge Graphs (KG) and text together (KG-T) can be used to train forward and reverse neural models that generate text from KG and vice versa. However models trained on datasets where KG and text pairs are not equivalent can suffer from more hallucination and poorer recall. In this paper, we verify this empirically by generating datasets with different levels of noise and find that noisier datasets do indeed lead to more hallucination. We argue that the ability of forward and reverse models trained on a dataset to cyclically regenerate source KG or text is a proxy for the equivalence between the KG and the text in the dataset. Using cyclic evaluation we find that manually created WebNLG is much better than automatically created TeKGen and T-REx. Informed by these observations, we construct a new, improved dataset called LAGRANGE using heuristics meant to improve equivalence between KG and text and show the impact of each of the heuristics on cyclic evaluation. We also construct two synthetic datasets using large language models (LLMs), and observe that these are conducive to models that perform significantly well on cyclic generation of text, but less so on cyclic generation of KGs, probably because of a lack of a consistent underlying ontology.

[ad_2]
Source link

Related posts
AI

GENAUDIT: A Machine Learning Tool to Assist Users in Fact-Checking LLM-Generated Outputs Against Inputs with Evidence

2 Mins read
[ad_1] With the recent progress made in the field of Artificial Intelligence (AI) and mainly Generative AI, the ability of Large Language…
AI

This AI Paper from the University of Oxford Proposes Magi: A Machine Learning Tool to Make Manga Accessible to the Visually Impaired

2 Mins read
[ad_1] In storytelling, Japanese comics, known as Manga, have carved out a significant niche, captivating audiences worldwide with their intricate plots and…
AI

The Dawn of Grok-1: A Leap Forward in AI Accessibility

2 Mins read
[ad_1] In an era where the democratization of artificial intelligence technology stands as a pivotal turning point for innovation across industries, xAI…

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *