Hi,
while running the ShortestPath baseline, I noticed some weird behavior: It seems like the agent is actually able to pick up more than it should be capable based on its actual max_weight.
I am using run_custom_scenario with the following to get a single airplane and 20 cargos:
env = AirliftEnv(
world_generator=AirliftWorldGenerator(
plane_types=[PlaneType(id=0, max_range=1.0, speed=0.01, max_weight=5)],
airport_generator=RandomAirportGenerator(
max_airports=10,
make_drop_off_area=True,
make_pick_up_area=True,
num_drop_off_airports=1,
num_pick_up_airports=1,
mapgen=map_generator,
),
route_generator=RouteByDistanceGenerator(
route_ratio=2,
),
cargo_generator=StaticCargoGenerator(
num_of_tasks=20,
max_weight=3,
soft_deadline_multiplier=10,
hard_deadline_multiplier=20,
),
airplane_generator=AirplaneGenerator(num_of_agents=1),
max_cycles=max_cycles
)
)
Running this will result in the plane doing only two flights, with the last flight actually transporting 19 cargos.
I confirmed this via debugging: After issuing the action to load all the cargo, if you step further you will get the following observation back, where current_weight > max_weight.
{'cargo_onboard': [12, 8, 15, 6, 11, 16, 3, 17, 18, 19, 0, 5, 2, 1, 10, 14, 9, 13, 4], 'state': <PlaneState.MOVING: 2>, 'plane_type': 0, 'current_weight': 31, 'max_weight': 5, ..}
Note, that this behaviour seems to be based on the following part of the ShortestPath Agent which just loads everything and the environment does not seem to check the actual max_weight:
for cargo in cargo_info_copy:
assigned_cargo.append(cargo.id)
pending_cargo.remove(cargo)
num_cargo_assigned += 1
cargo_info_list.remove(cargo)
# Ideally break out of this loop right as we reach the max airplane weight. but I've
# been getting better results just assigning all of them.
if num_cargo_assigned == max_airplane_weight:
continue
# break
Could you confirm whether this is a bug or whether I misunderstood something?
Posted by: jok4pt @ March 4, 2024, 11:28 a.m.I just noticed that the formatting in this forum really does not work well for code, so I posted it again on the github page. I hope this is okay. https://github.com/airlift-challenge/airlift/issues/1
Posted by: jok4pt @ March 4, 2024, 11:32 a.m.Hi,
The environment should not be allowing for the loading of more cargo even if the algorithm attempts to do so. This is is the load_cargo function in agents.py. I'm gonna look into this and see what else may be occurring. Thank you!
else:
if (cargo.weight + self.current_cargo_weight) <= self.max_loaded_weight:
self.current_airport.remove_cargo(cargo)
self.cargo_being_loaded.add(cargo)
else:
warnings.append(
"Unable to load Cargo ID: " + str(cargo.id) + " due to exceeding allowed airplane weight limit")
Thanks for your answere. I debugged at the code you highlighted and found the following:
While you do check: if (cargo.weight + self.current_cargo_weight) <= self.max_loaded_weight
it seems like self.current_cargo_weight does not change during the for loop, meaning that it is always 0, and therefore you can load any amount of cargo.
Posted by: jok4pt @ March 4, 2024, 1:54 p.m.I was able to confirm your bug report and this isn't working as intended. A fix will be rolled out soon. My recommendation is to ensure your algorithm checks for the max_weight for now (just to make sure it doesn't break after the bug fix).
Thank you
Posted by: adelanovic @ March 4, 2024, 5:31 p.m.Hi,
Just an update for ya. This was fixed and is live. Previous submissions will be re-run just to make sure they didn't attempt to load more cargo than allowed.
Thank you for finding this for us.
Adis
Posted by: adelanovic @ March 5, 2024, 9:02 p.m.Glad I could help. Thanks for fixing it so quickly!
Posted by: jok4pt @ March 6, 2024, 8:24 a.m.