Creating the Perfect Panoramic Video – A Collaboration between Disney, Lumenera and Norpix

The Disney Research team developed an algorithm to correct for parallax in video panoramas. Norpix were approached by the team to create a system to help test this algorithm. For this research, the Disney team needed a system that had the ability to capture high-resolution video with multiple cameras at 60 fps. To create a panorama beyond the resolution of a single camera, multiple views from the camera array had to be attached. These views had overlapping visual fields, but were not positioned precisely.

Challenges

Stitching together a series of images to produce a panorama video can result in various image imperfections, such as loss of image resolution, ghosting, distorted images, and blurred images. There are also issues relating to parallax and image warping. Parallax refers to the difference in an object’s position caused by varied site lines, in this case there were 16 different camera views, and the timing difference between cameras results in image wrapping. A computer was also needed to deal with the uncompressed high resolution images produced by each camera, using software that is suitable for multiple camera image streaming capture.

Solution

To facilitate multi- camera recording, Norpix have developed a system using its software and hardware with Lumenera cameras (Figure 1). Disney Research produced the stitching method used to piece together separate video streams into a seamless and single output panorama (Figure 2). Each camera was fastened to a Norpix SmallPix 2 portable solid state drive computer, with Norpix StreamPix software to meet all the challenging requirements of the system. Norpix’s StreamPix Remote software is installed on a remote computer on a LAN, and this controls all of the nine computers.

Norpix created a system for multi-camera recording using their hardware and software along with Lumenera cameras.

Figure 1. Norpix created a system for multi-camera recording using their hardware and software along with Lumenera cameras.

The above photo is a crop from a 20 megapixel panorama created from an unstructured array consisting of sixteen Lumenera Lt425C USB 3.0 cameras.

Figure 2. The above photo is a crop from a 20 megapixel panorama created from an unstructured array consisting of sixteen Lumenera Lt425C USB 3.0 cameras.

Figure 3 shows the camera array built by Disney Research to capture the panoramic videos. The research team demonstrated the flexibility of its method by capturing panoramas without placing cameras in accurate positions. The images captured by the cameras were automatically analyzed to determine the placement and alignment of every camera. As a result, the need for calibration is eliminated and the cameras can be positioned in a flexible manner.

The camera array Disney Research constructed for capturing the panoramic videos.

Figure 3. The camera array Disney Research constructed for capturing the panoramic videos.

The operator, from this workstation, observed live sample feeds delivered from all of the eighteen Lumenera cameras and also monitored the capture process. NorPix Pulse Generator & Digital Signal Amplifiers were used to trigger the cameras.

Sixteen Lumenera Lt425 cameras were selected for the following reasons:

  • Software and Hardware triggerable
  • Power supplied over USB 3.0 interface – No external power supply
  • High speed USB 3.0 interface for simplified connectivity and fastest image delivery
  • The required frame rate for this application was 60 fps. 90 fps can be achieved with full resolution by Lt425
  • 1” CMOS sensor with a fully electronic global shutter that captures high-speed and excellent quality images with zero blur
  • 128 MB RAM frame buffer for delivering images in a reliable and fast swift manner

Conclusion

Using a transfer rate of 240 MBps or 1.92 Gbps, high resolution, uncompressed images from all of the cameras were sent into the solid state drive. Images with excellent quality were delivered at fast frame rates by the Lt425 cameras, which used the CMOSIS CMV4000 sensors. Lumenera’s efficient drivers, USB 3.0 interface, and 128 MB frame buffer helped to deliver all of the captured images to the host computer hard drive without any exceptions.

Norpix selected Lumenera’s Lt425 cameras after running tests with other competitive solutions that turned out to be unsuccessful. Alternate USB 3.0 and Dual GigE solutions failed to deliver images in a reliable manner while trying to transfer the huge data of high speed and zero latency to the computer drives.

This information has been sourced, reviewed and adapted from materials provided by Lumenera Corporation.

For more information on this source, please visit Lumenera Corporation.

Ask A Question

Do you have a question you'd like to ask regarding this article?

Leave your feedback
Submit