The classes are Bulla, Cardiomyopathy, Hilum, Osteopenia, and Scoliosis. Without giving much detail about my approach, I'm checking my performance on these classes locally with the assistance of GPT-4 (see https://aclanthology.org/2023.emnlp-main.891.pdf), and according to my local evaluations my performance on these classes should be much higher than what I'm getting according to the system. I'm suspecting the main reason is that there is a mismatch between GPT-4 and the automatic labeler used by the challenge. Hence, it would be helpful if a clear definition of each class is provided, together with examples of reports from MIMIC-CXR illustrating positive and negative cases for each class. In particular, I noticed that Hilum is not an abnormality but an anatomical location, so I'm reinterpreting this class as "hilar abnormalities seen", but maybe that's not what the organizers of the challenge meant by it. I've also noticed many cases of disagreement between GPT-4 and the official labels for the other classes in Tasks 1 and 2, but for the purposes of this thread let's focus on the 5 classes of Task 3.
Thanks in advance.
Posted by: pamessina @ July 18, 2024, 4:43 p.m.