The data in the public and private test sets is partitioned into 3-minute "windows", every window contains 360 entries, one entry every 0.5 seconds, and contains RSSI values from one device only. Participants’ models are required to predict a single prediction of occupancy or people counting (depending on the given track) per every 3-minute window.
Participants have asked us:
How exactly do you define occupancy / people counting in the cases where those windows are not homogenous, i.e. the "ground truth" of occupancy / number of people in the room has different values for different entries inside the same window.
Answer:
Occupancy definition: a window is defined as occupied if there are at least 20 consecutive entries (10 seconds) with one person or more, labeled in the data (ground truth). Otherwise, it’s defined as empty.
Number of people definition: a window is defined as having 3/2/1/0 people in the room if the maximal labeled number of people that appear in at least 20 consecutive entries (10 seconds) is 3/2/1/0 correspondently.
Please note that in the provided baseline model, we assumed homogenous windows (occupied by the same number of people or empty in all window entries) - which is a simplistic and naïve assumption. We recommend the participants implement a more accurate model to support heterogeneous windows as well.
Posted by: MAFAT_Challenge @ May 16, 2022, 12:17 p.m.Thanks I really appreciate this reply as it both explicitly clarifies the way in which you define occupancy and also answers a lot of questions regarding the same.
I assume when the curtain falls you will use this definition to score the private LB.
However I can't help wondering if you use this exact same definition in calculating the public LB scores at the moment?
Posted by: skaak @ May 25, 2022, 10:36 a.m.Hi,
Yes, We use the same definition in calculating the public LB scores.
MAFAT Challenge Team
Posted by: MAFAT_Challenge @ May 26, 2022, 6:34 a.m.After some questions were raised by participants, we would like to help by providing an example that will further demonstrate the definition of the label for a window.
To make it easier for the reader, we concatenated lists’ labels (0/1/2/3) sequences so that the length will be 360.
Small scale example: np.array(([0] * 5) + ([1] * 3) is [0,0,0,0,0,1,1,1]Now, suppose there are five windows, and these are their Num_People:
Num_People_win1 = np.array(([0] * 100) + ([1] * 19) + ([2] * 2) + ([0] * 239))
Num_People_win2 = np.array(([0] * 200) + ([1] * 19) + ([0] * 5) + ([1] * 20) + ([0] * 116))
Num_People_win3 = np.array(([1] * 100) + ([2] * 21) + ([0] * 220)+([3]*19))
Num_People_win4 = np.array(([2] * 90) + ([1] * 90) + ([3] * 90) + ([0] * 90))
Num_People_win5 = np.array(([0] * 50) + ([1] * 19) + ([0] * 100) + ([3] * 15) +([2]*3)+ ([0]*173))
In track1 the labels for that windows:
1
1
1
1
0
In track2 the labels for that windows:
0
1
2
3
0
MAFAT Challenge Team