Instructions for prediction file submission at Codalab shared-task server:
1. The submission NEEDS to be a .zip, with a .jsonl predicted results file as its ONLY content.
2. Filenames could be anything but should conform to "samplefilename.jsonl".
3. Please DO NOT include anything other than your only .jsonl prediction result file.
4. Please ensure ALL the test samples (meme image) AND the corresponding entities enlisted in the unseen set, are considered for prediction and appear in your submission file. Please, do NOT leave out any entity unpredicted.
5. Each team should submit the output in JSON line format for the final evaluation:
{
‘image’ : ‘image_name’,
‘hero’ : [entity_list],
‘villain’ : [entity_list],
‘victim’ : [entity_list],
‘other’ : [entity_list]
}
For the samples depicted for the unseen test set, the submission format would look like:
{
‘image’ : memes_1486.png’,
‘hero’ : ['Donald Trump'],
‘villain’ : ['Joe Biden'],
‘victim’ : [],
‘other’ : ['Democratic National Convention (DMC)’, 'Republican National Convention (RNC)']
}
{
‘image’ : ‘image_2.png’,
‘hero’ : [‘Vladimir Putin’],
‘villain’ : [],
‘victim’ : [],
‘other’ : [‘the world’, ‘Salman Khan’, ‘vaccine’]
}
In case of multiple submissions by a team, we shall consider the best submission prior to the deadline for the final evaluation. No exceptions shall be made.