Measuring AI's Ability to Learn is Difficult
January 17, 2019 | University of WaterlooEstimated reading time: 1 minute
Organizations looking to benefit from the artificial intelligence (AI) revolution should be cautious about putting all their eggs in one basket, a study from the University of Waterloo has found.
In a study published in Nature Machine Intelligence, Waterloo researchers found that contrary to conventional wisdom, there can be no exact method for deciding whether a given problem may be successfully solved by machine learning tools.
“We have to proceed with caution,” said Shai Ben-David, lead author of the study and a professor in Waterloo’s School of Computer Science. “There is a big trend of tools that are very successful, but nobody understands why they are successful, and nobody can provide guarantees that they will continue to be successful.
“In situations where just a yes or no answer is required, we know exactly what can or cannot be done by machine learning algorithms. However, when it comes to more general setups, we can’t distinguish learnable from un-learnable tasks.”
In the study, Ben-David and his colleagues considered a learning model called estimating the maximum (EMX), which captures many common machine learning tasks. For example, tasks like identifying the best place to locate a set of distribution facilities to optimize their accessibility for future expected consumers. The research found that no mathematical method would ever be able to tell, given a task in that model, whether an AI-based tool could handle that task or not.
“This finding comes as a surprise to the research community since it has long been believed that once a precise description of a task is provided, it can then be determined whether machine learning algorithms will be able to learn and carry out that task,” said Ben-David.
The study, Learnability can be Undecidable, was co-authored by Ben-David, Pavel Hrubeš from the Institute of Mathematics of the Academy of Sciences in the Czech Republic, Shay Morgan from the Department of Computer Science, Princeton University, Amir Shpilka, Department of Computer Science, Tel Aviv University, and Amir Yehudayoff from the Department of Mathematics, Technion-IIT.
Suggested Items
Kurtz Ersa to Offer Ersa VERSAFLOW 3 – Level ll Maintenance and Application
03/27/2024 | Kurtz ErsaKurtz Ersa Inc., a leading supplier of electronics production equipment, is pleased to announce upcoming training opportunities at its facility in Plymouth, WI.
Omron to Showcase Inspection and Factory Automation Solutions at IPC APEX EXPO 2024
03/18/2024 | OmronOMRON Automation Americas, a global leader in industrial automation solutions will be exhibiting at IPC APEX 2024, taking place at the Anaheim Convention Center from April 9th to April 11th. Attendees are invited to visit Omron's booth #2647 to hear how AI powers Omron’s latest technology, in addition to learning more about our newest X-Ray inspection machine, the VT-X850. This new 3D CT X-Ray inspection system is designed specifically for electric vehicle (EV) surface mount technology (SMT) manufacturing lines.
YAMAHA to Showcase Intelligent Factory Solutions at IPC APEX 2024
03/18/2024 | Yamaha RoboticsYamaha Motor Corporation’s Robotics Division will highlight an exciting array of new machines and technologies from April 9 – 11, 2024 at the IPC APEX 2024 trade show in the Anaheim Convention Center, Booth #2820.
Seika Machinery to Present Solutions to Optimize Production Processes at APEX 2024
03/12/2024 | Seika Machinery, Inc.Seika Machinery, Inc., a leading provider of advanced machinery, materials and engineering services, is gearing up to showcase its latest advancements at Booth 4012 during the 2024 IPC APEX EXPO, slated to take place April 9-11, 2024 at the Anaheim Convention Center in California.
Koh Young’s Joel Scutchfield to Showcase Data-driven Approach to Inspection at IPC APEX EXPO
03/12/2024 | Koh YoungKoh Young, the industry leader in True3D measurement-based inspection solutions and the Premier Sponsor of IPC APEX EXPO, invites electronics manufacturers to see its innovative data-driven approach to measurement-based inspection.