CalibProj: Automatic Calibration of a Multi-Camera System
with Limited Overlapping Fields of View

for 3D Surgical Scene Reconstruction

TODO

IPCAI 2025 Extended Oral

Tim Flückiger1,2*, Jonas Hein1,2, Valery Fischer1, Philipp Fürnstahl1, Lilian Calvet1
1Research in Orthopedic Computer Science, University Hospital Balgrist, University of Zurich, Switzerland, 2Computer Vision and Geometry Group, ETH Zurich, Switzerland,
*Corresponding author
Paper Code Poster
TODO

Automatic external calibration of a multi-camera system using the projections of multi-scale markers (MSMs) on the operating room floor.

We present CalibProj, a Python toolkit for automatically calibrating the positions and orientations of multiple cameras, even when they have very limited field of view overlap or they have very different zoom levels. Unlike traditional methods, this approach requires no manual intervention, calibration board, or prior calibration expertise. CalibProj uses a ceiling-mounted projector to display specially designed multi-scale markers (MSMs)—2D patterns at varying sizes—onto the scene. These projected markers can be seen from multiple angles and distances, enabling cameras with different viewpoints and zoom settings to detect and extract accurate feature correspondences. Importantly, the projector itself does not need to be calibrated. Such a system is especially useful for:
  • Camera networks with different fields of view (e.g., in an operating room)
  • Multi-zoom or pan-tilt-zoom (PTZ) camera installations
  • Surveillance systems with cameras mounted far apart
Example viewpoints (left: GoPro Hero 10 Black, right: ceiling-mounted Canon CR-N300 with strong optical zoom) with different camera locations and zoom levels inducing an appearing scale factor of up to 19x on the surgical scene.

Calibration Setup

TODO
Overview of the proposed setup. Our multi-camera calibration system employs an entry-level projector (P, projection in blue). Cameras are highlighted in red. In this example, GoPro cameras placed in far field and in near field on surgical lamps, and a zoomed in Canon CR-N300 (C3) are employed. Images from a farfield camera (C4, top) and a near field camera (C3, bottom) during the projection of the proposed MSM. Top: MSM is successfully detected (green cross) in the far-field view for a large scale projection (left) and not detected otherwise (right). Bottom: MSM is not detected in the near-field view for a large scale projection (left) and successfully detected otherwise (green cross in the right image), therefore providing one point correspondence between C3 and C4 through their shared imaged center. This operation is repeated, providing a set of dense and uniformly distributed point correspondences.


Demo

The projection of 12,800 markers, resulting in a dense grid of 3,200 multi-scale marker (MSM) centers (shown in red in the video) used as inter-view correspondences, takes 40 seconds. The central viewpoint, captured by a Canon CR-N300 with high optical zoom, exhibits a relative scale factor of up to 19× compared to other far-field cameras. Despite this, the use of projected multi-scale markers enables the recovery of accurate inter-view correspondences.

Application to 3D Human Pose Reconstruction

TODO

Top: Examples of camera viewpoints used for 3D surgical scene reconstruction. From left to right: far-field camera C1 (ceiling-mounted GoPro), near-field camera C2 (surgical lamp-mounted GoPro), and near-field camera C3 (ceiling-mounted Canon CR-N300), zoomed in on the surgical field.

Bottom: Inter-viewpoint correspondences provided by our multi-camera calibration system, observed from the same viewpoints. From left to right: all correspondences seen by C1 (blue, green, and red), correspondences between C1 and C2 (green), and correspondences between C1 and C3 (red). The surgeon’s body and hand poses, represented by blue lines in the first row, are successfully reconstructed using the estimated camera poses.



Application of our calibration method to 3D-SSR. Left: input videos from far-field camera (GoPro Hero 10 Black). Right: the surgeon’s body poses successfully reconstructed using camera poses computed with the proposed MSM-based calibration method. The reconstructed objects are integrated into the OR digital twin.

BibTeX

@misc{flueckiger2025automatic,
      title={Automatic Calibration of a Multi-Camera System with Limited Overlapping Fields of View for 3D Surgical Scene Reconstruction}, 
      author={Tim Flückiger and Jonas Hein and Valery Fischer and Philipp Fürnstahl and Lilian Calvet},
      year={2025},
      eprint={2501.16221},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2501.16221}, 
}

Acknowledgements

This work has been supported by the OR-X - a swiss national research infrastructure for translational surgery - and associated funding by the University of Zurich and University Hospital Balgrist.