As I understood the objective of the task B. We CANNOT finetune and get any finetuned models on str and sts datasets. So our solution should contain some LM pretrained on MLM, basically any model not finetuned on downstream tasks.
I want to clear that from organizers. And hos is it going to be checked that the weights for the model weren't obtained after finetuning on any downstream task.