Research on machine learning and AI, now a key technology in virtually every industry and business, is far too extensive for anyone to read. This column, Perceptron (formerly Deep Science), aims to collect some of the most relevant recent discoveries and articles – particularly in the field of, but not limited to, artificial intelligence – and explain why they matter.
This week in AI, engineers at Penn State announced that they have created a chip that can process and classify nearly two billion images per second. Carnegie Mellon meanwhile has signed a $10.5 million contract for the US military to expand the use of AI in predictive maintenance. And at UC Berkeley, a team of scientists is applying AI research to solve climate problems, such as: concept snow as a water source.
Penn State’s work focused on overcoming the limitations of traditional processors when applied to AI workloads — specifically, recognizing and classifying images or the objects within them. Before a machine learning system can process an image, it must be captured by a camera’s image sensor (assuming it’s a real image), converted by the sensor from light into electrical signals, and then converted back into binary data. Only then can the system ‘understand’ the image enough to process, analyze and classify it.
Penn State engineers including postdoctoral fellow Farshid Ashtiani, graduate student Alexander J. Geers, and associate professor of electrical and systems engineering Firooz Aflatouni designed a workaround that they believe removes the most time-consuming aspects of traditional chip-based AI image processing. Their 9.3-square-millimeter custom processor processes light received from an “object of interest” using what they call an “optical deep neural network.”
3d rendering, computer board with circuits and chip
Essentially, the researchers’ processor uses “optical neurons” interconnected using optical wires known as waveguides to form a deep network of many layers. Information passes through the layers, with each step helping to classify the input image into one of the learned categories. Thanks to the chip’s ability to compute as light travels through it and read and process optical signals directly, the researchers claim the chip doesn’t need to store any information and can perform a full image classification in about half a nanosecond.
“We are not the first to come up with technology that directly reads optical signals,” Geers said in a statement, “but we are the first to create the complete system within a chip that is both compatible with existing technology and scalable to work with more complex data.” He expects the work to have applications in automatically detecting text in photos, helping self-driving cars recognize obstacles and other computer-related tasks.
At Carnegie Mellon, the university’s Auton Lab is focused on a different set of use cases: applying predictive maintenance techniques to everything from ground vehicles to power generators. Supported by the aforementioned contract, Artur Dubrawski, director of Auton Lab, will make an effort to conduct fundamental research to broaden the applicability of computer models of complex physical systems, known as digital twins, to many domains.
Digital twin technologies are not new. GEAWS and other companies offer products that allow customers to model digital twin of machinery. based in London SenSat creates digital twin models of locations for construction, mining and energy projects. Meanwhile, startups like Lacuna and Nexar are building digital twins of entire cities.
But digital twin technologies share the same limitations, the most important being imprecise modeling based on imprecise data. As elsewhere, the trash is in, trash is out.
To address this and other barriers to the wider use of digital twins, Dubrawski’s team is working with a range of stakeholders, such as critical care clinicians, to explore scenarios, including in healthcare. The Auton Lab aims to develop new, more efficient methods for “capturing human expertise” so that AI systems can understand contexts that are not well represented in data, as well as methods for sharing that expertise with users.
One thing AI may soon have that some people seem to lack is common sense. DARPA has funded a number of initiatives in several labs that aim to give robots a general idea of what to do when things don’t go quite right when they walk, carry something, or grasp an object.
Usually these models are quite brittle and fail miserably once certain parameters are exceeded or unexpected events occur. Teaching them “common sense” will help them be more flexible, with a general sense of how to salvage a situation. These aren’t particularly high-end concepts, just smarter ways to handle them. For example, if something falls outside the expected parameters, it can adjust other parameters to counteract it, even if they weren’t specifically designed for that.
This doesn’t mean robots will improvise everything – they just won’t fail as easily or as hard as they currently do. Current research shows that locomotion in rough terrain is better, shifting loads are better carried and unfamiliar objects are better retained when ‘common sense’ training is included.
The research team at UC Berkeley, on the other hand, focuses on one area in particular: climate change. The Berkeley AI Research Climate Initiative (BAIR) – recently launched, hosted by computer science PhD students Colorado Reed and Medhini Narasimhan and computer science doctoral student Ritwik Gupta – seeks partners between climate experts, government agencies and industry to achieve goals that make sense to both climate and AI.
One of the first projects the initiative plans to tackle will use an AI technique to combine measurements from aircraft snow sightings and freely available weather and satellite data sources. AI will help track the snow life cycle, which is currently not possible without major effort, allowing the researchers to estimate and predict how much water is in the snow in the Sierra Nevada mountains — and its impact on the flow of the snow. region can be predicted.
A press release describing the BAIR’s efforts noted that the snow condition has implications for public health and the economy. About 1.2 billion people worldwide rely on snowmelt for water consumption or other purposes, and the Sierra mountains alone provide water for more than half of California’s population.
Any technology or research done by the climate initiative will be published openly and not exclusively licensed, said Trevor Darrel, co-founder of BAIR and a professor of computer science at Berkeley.
A graph showing the CO2 emissions of various training processes for AI models.
However, AI itself also contributes to climate change, as it takes huge computing resources to train models like GPT-3 and DALL-E. The Allen Institute for AI (AI2) conducted a study about how these training periods can be done intelligently to reduce their impact on the climate. It’s no trivial calculation: where electricity comes from is in constant flux, and peak usage like a day-long supercomputer run can’t just be split up to run next week when the sun is out and solar energy is plentiful.
AI2’s work looks at the carbon intensity of training different models in different locations and times, part of a larger project at the Green Software Foundation to reduce the footprint of these important but energy-consuming processes.
Last but not least, OpenAI unveiled this week Video Preliminary Training (VPT), a training technique that uses a small amount of tagged data to teach an AI system to perform tasks such as creating diamond tools in Minecraft. VPT involves searching the web for videos and requiring contractors to produce data (eg 2,000 hours of videos with mouse and keyboard actions) and then train a model to predict actions based on past and future video frames. In the final step, the original videos from the internet are tagged with the contractor’s data to train a system to predict actions given only in previous frames.
OpenAI used Minecraft as a test case for VPT, but the company claims the approach is fairly generic — a step toward “general computer-using agents.” In any case, the model is open source, as is the contractor data that OpenAI obtained for its experiments.