As AI makes its way out of the chat box and into the living room, it needs a better understanding of space and objects. To take that work further, the Allen Institute for Artificial Intelligence has created a huge and diverse database of his 3D models of everyday objects, so simulating AI models could be much closer to reality.
A simulator is essentially a 3D environment intended to represent a real-world place that a robot or AI needs to navigate or understand. But unlike modern console games, for example, training simulators aren’t photorealistic and often lack detail, variation, and interactivity.
The Objaverse, as awkwardly but somehow pleasantly named, aims to remedy this with a collection of over 800,000 (and growing) 3D models containing all sorts of metadata. Representations range from different types of food, to tables and chairs, to appliances and gadgets. Relatively ordinary objects you might see in your home, office, or restaurant are displayed here.
It aims to replace aging object libraries such as ShapeNet, an old standby database, with about 50,000 low-detail models. If the only “lamp” your AI sees is a generic lamp with no pattern or color, how can it recognize funky cut glass or a lamp with a completely different shape? Object variations are included so the model can learn what defines an object despite their differences.
Sure, an AI assistant might not need to identify a bookshelf as “medieval” or not, but it certainly needs to be able to tell the difference between peeled and unpeeled bananas. But you never know what could go wrong.
The use of photorealistic imagery (obviously captured with photogrammetry) also brings a level of versatility and realism that is evident in retrospect. Sure, all beds look pretty much the same, but what about an unmade bed?
It’s also helpful to have an object to animate to do the “main thing” if you want. Knowing what a refrigerator, cabinet, book, laptop, or garage door looks like is one thing, but how does it get from A to B? Simply It may sound like it, but if the AI model is not provided with this information, it is unlikely to invent or intuitively understand.
You can read more about the features and details of this huge dataset in the AI2 paper describing it. Researchers can get started today for free on Hugging Face.
