I am writing to inquire about the camera poses in the provided ICCV2023 challenge dataset. (To NeRF or not to NeRF)
I have been working on the challenge and attempting to obtain camera poses from the given data. However, I encountered some difficulties when using the COLMAP program. It seems that COLMAP failed to produce the desired results, possibly due to the black background in the images, which might have hindered the identification of good image pairs and consequently led to inaccurate pose estimation.
To better understand the camera poses in the dataset, I would like to clarify whether the camera extrinsic and intrinsic parameters were extracted using the COLMAP program itself or if they were obtained from the fixed cameras used during the capture process.
I appreciate your assistance in this matter and any information you can provide regarding the camera pose acquisition process would be highly valuable. Please let me know if there are any specific instructions or alternative methods I should consider to accurately determine the camera poses.
Sincerely,
Hyeseong Kim
Hi Hyeseong,
Thank you for your question. The camera poses [R,t] (extrinsic matrices) are provided in the dataset. For example, the Blender format pose is given as json_data[‘transform_matrix’] in the transforms_{train, val}.json file. Additionally, the camera intrinsic matrix, which consists of the focal length (fx) and image centre (cx, cy), can be obtained using the following equations:
W, H = 3000, 4096
fx = 0.5 * W / np.tan(0.5 * json_data[‘camera_angle_x’])
cx, cy = W/2, H/2
intrinsic_matrix = np.array([[fx, 0, cx], [0, fx, cy], [0, 0, 1]])
To give you more information, we used fixed positions for the cameras. At every capture date, we followed camera calibration, color correction, image undistortion and camera pose format conversion (to either LLFF or Blender format) processes. We never used COLMAP to produce camera poses. Please use our script (tip1_visualise_camera_poses.py) in the starting kit to visualize all the camera poses in OpenCV format.
You can find more details in following pages:
- Focal length: https://codalab.lisn.upsaclay.fr/forums/13233/1773/
- Camera pose format (e.g., Blender to OpenCV) conversion: https://codalab.lisn.upsaclay.fr/forums/13233/1755/
- Data capture in 4 dates: https://codalab.lisn.upsaclay.fr/competitions/13273#participate
- Camera pose visualisation script: https://codalab.lisn.upsaclay.fr/competitions/13273#learn_the_details-submission_format
Can you please provide the camera sensor details? I am trying to calculate the focal length in mm using F(mm) = F(pixels) * SensorWidth(mm) / ImageWidth (pixel). I believe the above equation (fx = 0.5 * W / np.tan(0.5 * json_data[‘camera_angle_x’])) provides F(pixels) and W is the ImageWidth (pixel). So I would require SensorWidth(mm) to get the focal length in mm.
Thank you
Posted by: rkumar38 @ July 1, 2023, 11:46 p.m.Hi,
Regarding the cameras specifications, we used the Basler boA4112-68cc cameras with the Sony IMX253 CMOS sensor.
The Sensor is 14.1 mm x 10.3 mm.
More information for the cameras and the sensor can be found in the following link:
https://www.baslerweb.com/en/products/cameras/area-scan-cameras/boost/boa4112-68cc
Let me know if you need anything else,
Thanos