FYI, in some of the tests in scenarios the random algorithm did better than the baseline shortest path. Here are two examples:
Test18/Level0 (random: 2690.279 baseline: 2710.722)
Test18/Level1 (random: 2604.195 baseline: 2689.439)
This situation means that scores that are less than (better) than both will have a negative normalized score.
Here's a full list of tests in scenarios that will produce reversed normalized scores:
Thanks for bringing this issue up. We're looking into this.Posted by: ccafeccafe @ Jan. 20, 2023, 4:13 p.m.
jkolen, thanks so much for letting us know, and for taking the time to write out the details.
You are certainly right - we missed this issue. We fixed it in a couple of ways:
1) We used a different baseline (similar to the original, but with a few differences). We will share this baseline next week.
2) We were still seeing the issue in a couple of levels. The baseline solution will sit and wait for routes to become available, and it appears that in some cases a random movement pattern actually beats this. After some discussion, we decided to simply omit these levels.
We pushed an updated set of scenarios to https://airliftchallenge.com/scenarios/airlift_test_scenarios.zip (we also made changes to the parameters of the scenarios to improve scoring).Posted by: abeckus @ Jan. 22, 2023, 1:23 a.m.