A new camera setup was used to film the harvest ants
Nature Picture Library/Alamy
Video from 54 cameras is stitched together using AI to produce an incredibly detailed 3D video. This technique has been used to photograph many small creatures at once.
To understand why large groups of small animals such as ants or flies behave the way they do, researchers want to find out what every individual in the group is doing at each moment. . But recording devices can typically capture one animal in great detail, or many with no detail, says Kevin Chow of the University of California, Berkeley. He and his colleagues have now combined 54 cameras to create a device that captures detail very quickly in both cases.
They placed individual harvesting ants (Pogonomyrmex barbatus), zebrafish larvae (Danish Lelio) and Drosophila (fruit fly high day) in a shallow box 25 cm below the 6 x 9 grid, there was a small camera, each less than 1 mm high and wide, illuminated by an LED light. All cameras recorded video simultaneously, and researchers trained a neural network to stitch the recordings together.
Zhou said the AI learned something similar to how smartphones create panoramas from a series of images, but in a more complex setting and in 3D. “Our brains do something similar, but the challenge here was his 54 eyes,” he says.
The finished video was displayed at approximately 5 gigapixels per second, or 100 times the resolution of high-definition television. It captures movements that are too fast for the human eye to see.
The research team observed how 40 zebrafish larvae leaned up and down as they approached food, and 10 ants bent one leg joint at a time. Other teams have previously studied single animals in such detail, but the new device could help determine whether many animals in a group do so.
Mark Levoy of Stanford University in California said the device’s inability to capture creatures smaller than ants and flies may be a limitation of the device. For example, he says, microscopic organisms are often translucent, which can hinder AI stitching methods.
Zhou said the team is now working on adapting the device to small objects, such as single cells, to be able to image internal processes in small animals injected with special glowing compounds.
topic:
