DARPA Goes 'Meta' with Machine Learning for Machine Learning
June 20, 2016 | DARPAEstimated reading time: 2 minutes
Popular search engines are great at finding answers for point-of-fact questions like the elevation of Mount Everest or current movies running at local theaters. They are not, however, very good at answering what-if or predictive questions—questions that depend on multiple variables, such as “What influences the stock market?” or “What are the major drivers of environmental stability?” In many cases that shortcoming is not for lack of relevant data. Rather, what’s missing are empirical models of complex processes that influence the behavior and impact of those data elements. In a world in which scientists, policymakers and others are awash in data, the inability to construct reliable models that can deliver insights from that raw information has become an acute limitation for planners.
To free researchers from the tedium and limits of having to design their own empirical models, DARPA today launched its Data-Driven Discovery of Models (D3M) program. The goal of D3M is to help overcome the data-science expertise gap by enabling non-experts to construct complex empirical models through automation of large parts of the model-creation process. If successful, researchers using D3M tools will effectively have access to an army of “virtual data scientists.”
“The construction of empirical models today is largely a manual process, requiring data experts to translate stochastic elements, such as weather and traffic, into models that engineers and scientists can then ask questions of,” said Wade Shen, program manager in DARPA’s Information Innovation Office. “We have an urgent need to develop machine-based modeling for users with no data-science background. We believe it’s possible to automate certain aspects of data science, and specifically to have machines learn from prior example how to construct new models.”
D3M is being initiated at a time when there is unprecedented availability of data via improved sensing and open sources, and vast opportunities to take advantage of those data streams to speed scientific discovery, deepen intelligence collection, and improve U.S. government logistics and workforce management. Unfortunately, the expertise required to build useful models is in short supply. Some experts project deficits of 140,000 to 190,000 data scientists worldwide in 2016 alone, and increasing shortfalls in coming years. Also, because the process to build empirical models is so manual, their relative sophistication and value is often limited.
A recent exercise conducted by researchers from New York University illustrated the problem. The goal was to model traffic flows as a function of time, weather and location for each block in downtown Manhattan, and then use that model to conduct “what-if” simulations of various ride-sharing scenarios and project the likely effects of those ride-sharing variants on congestion. The team managed to make the model, but it required about 30 person-months of NYU data scientists’ time and more than 60 person-months of preparatory effort to explore, clean and regularize several urban data sets, including statistics about local crime, schools, subway systems, parks, noise, taxis, and restaurants.
“Our ability to understand everything from traffic to the behavior of hostile forces is increasingly possible given the growth in data from sensors and open sources,” said Shen. “The hope is that D3M will handle the basics of model development so people can apply their human intelligence to look at data in new ways, and imagine solutions and possibilities that were not obvious or even conceivable before.”
Suggested Items
Koh Young Showcases Award-winning Inspection Solutions at SMTconnect with SmartRep in Hall 4A.225
04/25/2024 | Koh Young TechnologyKoh Young Technology, the industry leader in True 3D measurement-based inspection solutions, will showcase an array of award-winning inspection and measurement solutions at SMTconnect alongside its sales partner, SmartRep, in booth 4A.225 at NürnbergMesse from June 11-13, 2023. The following offers a glimpse into what Koh Young will present at the tradeshow:
Real Time with… IPC APEX EXPO 2024: Plasmatreat: Innovative Surface Preparation Solutions
04/25/2024 | Real Time with...IPC APEX EXPOIn this interview, Editor Nolan Johnson speaks with Hardev Grewal, CEO and president of Plasmatreat, a developer of atmospheric plasma solutions. Plasmatreat uses clean compressed air and electricity to create plasma, offering environmentally friendly methods for surface preparation. Their technology measures plasma density for process optimization and can remove organic micro-contamination. Nolan and Hardev also discuss REDOX-Tool, a new technology for removing metal oxides.
KYZEN to Promote Pair of Stencil Cleaning Chemistries at SMTA Ciudad Jaurez Expo & Tech Forum
04/25/2024 | KYZEN'KYZEN, the global leader in innovative environmentally friendly cleaning chemistries, will exhibit at the SMTA Ciudad Juarez Expo & Tech Forum, scheduled to take place Thursday, May 9, 2024 at the Injectronic Convention Center in Ciudad Jaurez, Chihuahua, Mexico. During the event the KYZEN Clean Team will focus on understencil cleaning products KYZEN E5631J and CYBERSOLV C8882.
Cadence, TSMC Collaborate on Wide-Ranging Innovations to Transform System and Semiconductor Design
04/25/2024 | Cadence Design SystemsCadence Design Systems, Inc. and TSMC have extended their longstanding collaboration by announcing a broad range of innovative technology advancements to accelerate design, including developments ranging from 3D-IC and advanced process nodes to design IP and photonics.
Listen Up! The Intricacies of PCB Drilling Detailed in New Podcast Episode
04/25/2024 | I-Connect007In episode 5 of the podcast series, On the Line With: Designing for Reality, Nolan Johnson and Matt Stevenson continue down the manufacturing process, this time focusing on the post-lamination drilling process for PCBs. Matt and Nolan delve into the intricacies of the PCB drilling process, highlighting the importance of hole quality, drill parameters, and design optimization to ensure smooth manufacturing. The conversation covers topics such as drill bit sizes, aspect ratios, vias, challenges in drilling, and ways to enhance efficiency in the drilling department.