A system designed by Massachusetts Institute of Technology researchers could produce far more efficient data analysis while removing most of the need for human involvement in the process.

Although computers and sensors can collect unfathomable amounts of data, human eyes are generally needed to find patterns within that information.

MIT’s Data Science Machine, however, utilized algorithms to successfully — and quickly — analyze Big Data without human intervention. The system successfully outperformed hundreds of human teams in recent data science competitions.

"We view the Data Science Machine as a natural complement to human intelligence," said Max Kanter, whose master's thesis provided the basis for the project.

The Data Science Machine can track correlations between entries throughout a database, as well as limiting categories such as times or labels.

The system subsequently produces a series of patterns to feature, then narrows that list by further identifying correlations and implementing sample data.

MIT researchers entered the system in three different data science competitions featuring a total of 906 human teams. The Data Science Machine finished ahead of 615 teams, and its predictions were 87 percent, 94 percent and 96 percent as accurate as the winning entries.

Moreover, while the human teams worked on their prediction algorithms for months, MIT's system took two to 12 hours for each submission.

“There’s so much data out there to be analyzed. And right now it’s just sitting there not doing anything," Kanter said. "So maybe we can come up with a solution that will at least get us started on it, at least get us moving."

The system will be presented this week at the IEEE International Conference on Data Science and Advanced Analytics in Paris.

By: DocMemory
Copyright © 2023 CST, Inc. All Rights Reserved