Since it is contrastively trained on MIMIC-CXR train+val according to the paper and this competition terms and conditions.
"You cannot train on images in this competition's development and test sets. If we find any evidence that you have trained on MIMIC-CXR images in the competition's unlabeled development and test sets, you will be disqualified and not permitted to participate in the ICCV CVAMD 2023 workshop.
You are permitted to train on external data and use pretrained models, so long as they were not trained on images in this competition's development and test sets. For example, this means that you can use ImageNet-pretrained models, but you likely cannot use models pretrained on MIMIC-CXR since those models were almost certainly trained on images in this competition's development and/or test sets (we use different splits from the official MIMIC-CXR-JPG train/validation/test split).
All external data and pretrained models used must be publicly available. Please provide links or other attribution to these resources either in your code or ICCV CVAMD submission describing your competition solution."
The MedCLIP, GloRia or similar which were trained on RoCo or CheXpert are eligible....
While ConViRT is not...
And to be eligible... a manually trained CheXZero on only the train set of this competition is needed....
Posted by: peratham.bkk @ July 9, 2023, 2:23 p.m.In fact, even though having some dataset leaks, CheXZero (zero-shot) and MedCLIP (also zero-shot; pretrained on both mimic-cxr + CheXpert) perform poorly in this competition data.
These models tend to have inadequate knowledge of disease classes especially rare ones.
Yes, that's correct. Any model pretrained on MIMIC-CXR is almost certainly ineligible since we use a different training set from the official splits and from other papers. Other chest X-ray datasets are fair game though!
Greg Holste | CXR-LT Organizer
Posted by: gholste @ July 9, 2023, 6:01 p.m.