TRIPLET Challenge @ ISWC2025

Organized by nandana - Current server time: July 2, 2025, 2:49 a.m. UTC

Previous

Task 2: Extraction
June 15, 2025, midnight UTC

Current

Task 3: Consistency
June 15, 2025, midnight UTC

End

Competition Ends
Never

Task 1: Relatedness

Start: June 15, 2025, midnight

Description: The goal is to assess the relatedness between tables and textual passages (within documents and across documents). For example, in the figure below, a participant tool should identify the relatedness between the sentence containing the entity “AirPort Extreme 802.11n” (highlighted in yellow) and the data table providing information about output power and frequency for this entity. Participants will be provided with tables and textual passages that would need to be ranked. The evaluation will use metrics such as P@k, R@k and F1@k.

Task 2: Extraction

Start: June 15, 2025, midnight

Description: The goal is to automatically extract knowledge jointly from tables and related texts. For example, in the figure below, a participant tool should extract triples composed of mentions located partly in the text and partly in the table and disambiguated with entities and predicates identified in the Wikidata reference knowledge graph. Knowledge extraction will be carried out in Open Extraction Information mode, using the Wikidata knowledge graph as a reference, but with the possibility that certain predicates or entities may not be present in the graph. Similar to the Text2KGBench evaluation, and because the set of triples are not exhaustive for a given sentence, to avoid false negatives, we will follow a locally closed approach by only considering the relations that are part of the ground truth. The evaluation will then use metrics such as P, R and F1.

Task 3: Consistency

Start: June 15, 2025, midnight

Description: The goal is to check the consistency of knowledge extracted from tables and texts with existing triples in the Wikidata knowledge graph. Different kind of inconsistencies will be considered in this task.

Competition Ends

Never

You must be logged in to participate in competitions.

Sign In