If AI wants to make its way out of the chat box and into our living rooms, it will have to better understand spaces and objects. To further that work, the Allen Institute for AI created a gigantic and diverse database of 3D models of everyday objects, so simulations for AI models can be that much closer to reality.
Simulators are basically 3D environments intended to represent real places that a robot or AI may need to navigate or understand. But unlike, say, a modern console game, training simulators are far from photo-realistic and often lack detail, variety, or interactivity.
Front, as it’s awkwardly but somehow pleasantly called, aims to improve upon this with its collection of over 800,000 (and growing) 3D models with all kinds of metadata. The things depicted range from foods to tables and chairs to appliances and gadgets. Every relatively ordinary object you would expect to find in a home, office or restaurant is represented here.
It is intended to replace outdated object libraries such as ShapeNet, an old standby database containing about 50,000 less detailed models. If the only “bulb” your AI has ever seen is a generic bulb with no pattern or color, how can you expect it to recognize a funky cut glass or a bulb with a completely different shape? Objaverse contains variations of regular objects so that the model can learn what defines them despite their differences.
Of course, it’s probably not necessary for your AI assistant to label a bookcase as “medieval” or not, but it certainly needs to know the difference between a peeled and an unpeeled banana. But you never know what matters.
The use of photo-realistic imagery (captured via photogrammetry, it’s clear) also brings a level of diversity and realism that’s obvious in hindsight. Sure, all beds look pretty much the same, but what about unmade beds? All different!
It’s also helpful to have objects that also animate to do their “main thing”. Knowing what a fridge, cupboard, book, laptop or garage door looks like closed is one thing and open is another, but how does it get from A to B? It sounds simplistic, but if AI models don’t get this information, they probably won’t invent or sense it.
You can read more about the features and details of this huge dataset in the AI2 document describing it. And if you are a researcher, you can start using it now for free through Hugging Face.
