Artificial intelligence (AI) and machine learning have seen rapid advancements in recent years, fueled by the availability of large datasets that can be used to train algorithms. From image recognition to natural language processing, data is the most vital resource that enables machines to learn and improve their performance on specific tasks.
In this article, we explore the 10 most important and frequently used data sets that have driven progress in AI and machine learning research. These datasets have enabled breakthroughs in computer vision, speech recognition, and other domains by providing raw training data as well as benchmarks to measure accuracy on common tasks.
ImageNet
ImageNet is one of the most influential datasets used in computer vision and image recognition research. First released in 2009 by researchers at Princeton University, ImageNet contains over 14 million images spread across 20,000 categories like vehicles, animals, and household objects. Researchers can use this data to train convolutional neural networks (CNNs) to accurately identify and classify images. Competitions like the annual ImageNet Large Scale Visual Recognition Challenge (ILSVRC) have relied on ImageNet to measure progress in image classification - error rates have fallen from over 25% in 2010 to under 2.5% in 2017 thanks to advances fueled by this dataset.
MNIST
The MNIST database is a benchmark dataset used for training image processing systems and particularly handwritten digit recognition. First put together in 1999, it consists of 70,000 scanned images of handwritten digits 0-9 divided into 60,000 training images and 10,000 test images. Each grayscale image is size-normalized and centered to fit in a 28x28 pixel box. MNIST’s simplicity (single channel, low resolution images) has made it popular for training basic computer vision techniques despite more complex datasets being available today. Achieving over 99% accuracy on MNIST is considered a minimum requirement for modern algorithms aiming to tackle handwritten recognition tasks.
COCO
MS COCO (Common Objects in Context) is another highly influential image dataset, particularly for the task of object detection. Released by Microsoft AI researchers in 2015, it contains over 200,000 labeled images with 1.5 million object instances segmented and labeled with image captions, object classes, segmentation masks etc. COCO focuses more on contextual images with complex real-world scenes containing common objects arranged in their natural contexts. Thanks to rich annotations spanning 80 object categories, COCO enables training of computer vision models for object detection and segmentation - crucial capabilities for applications like self-driving cars.
Wikipedia
Wikipedia, launched in 2001, has become an invaluable source of data for NLP tasks looking to train on authentic human-written content. There are over 6 million English articles on Wikipedia edited collaboratively by volunteers with rules on style and formatting. This makes it ideal to train AI systems focused on text generation, text summarization, semantic analysis and more. The constantly evolving nature also provides opportunities to study how discourse changes over time. There have been multiple datasets built around Wikipedia data including DBPedia and WebText harvested from Wikipedia text.
Amazon Reviews
Sentiment analysis aims to detect tone and emotion in textual content, and there has been an explosion of sentiment analysis datasets built from online reviews on sites like Amazon. These contain hundreds of thousands of reviews from Amazon shoppers alongside star ratings that can act as targets for supervised training of machine learning models. Enabling systems to gauge sentiment from textual data alone opens up many opportunities for analyzing customer feedback, monitoring brand sentiment across social channels and more. Popular sentiment analysis benchmarks derived from Amazon reviews include the Amazon Reviews dataset from Julian McAuley at UCSD and the Amazon Product Reviews dataset from Stanford Network Analysis Project (SNAP).
Enron
Email Dataset Released in 2015, the Enron Email Dataset contains 0.5 million real emails from 150 senior Enron executives involved in the famous fraud scandal. This makes it uniquely interesting for training AI systems focused on topics and sentiment analysis related specifically to business contexts. There is also opportunity for insights into fraud detection based on abnormalities in communication and relationships. Pre-configured Enron datasets formatted for machine learning tasks have been released on Kaggle, expanding accessibility for students and researchers aiming to work with this unusual email trove. The availability of genuine internal business communications data is relatively scarce, driving interest in the Enron corpus.
Common Crawl
Common Crawl is a massive archive of web crawl data that has been invaluable for training AI models focused on natural language processing and understanding. This non-profit organization runs thousands of servers and crawls over 25 billion web pages per month, storing petabytes of data from the constantly evolving web. This is then made available for researchers and organizations through their AWS and Google Cloud partnerships. Within Common Crawl datasets we can find text and images sourced from all corners of the internet, capturing an up-to-date snapshot. The heterogeneity and authenticity of Common Crawl sourced data has fueled advances in fundamental NLP models including word embeddings and language models such as BERT and GPT-3.
LibriSpeech
For speech recognition tasks and audio data, LibriSpeech has become a standard open source benchmark dataset for training AI systems. Based on public domain audio books from LibriVox, LibriSpeech contains 1000 hours of 16khz read English speech from 2,484 speakers. Since releases in 2015, it has been augmented with multiple larger variants scaling up to 60,000 hours. Transcripts of the corresponding spoken passages are provided covering a vocabulary of over 200k words. LibriSpeech enabled ASR models to take advantage of transfer learning - models like Wav2Vec 2.0 use self-supervised pretraining on other languages before fine-tuning on English data which gives dramatic boosts on this benchmark.
Sports-1M
With increasing consumer demand for analyzing video content, Sports-1M has emerged as a crucial dataset for sports-related AI tasks. Released in 2014, it contains 1 million YouTube sports video clips totaling 20k hours in 487 sports categories. Annotations describe the sports category along with automatic speech transcripts. Primary use cases include video classification, multimodal analysis leveraging alignments between video, audio and text, and generating natural language descriptions of events in videos. Reflecting real-world imperfections, Sports-1M contains noisy imperfect annotations presenting challenges for researchers. Nevertheless it offers an invaluable starting point before attempting to tackle describing open-ended ‘in-the-wild’ sports video footage with all its complexity.
Waymo Open Dataset
Finally, self-driving vehicle development has been fueled by datasets providing sensors feeds from actual vehicles driving real routes in urban environments. The extremely demanding accuracy requirements means diverse driving scenarios must be experienced. The Waymo Open Dataset released in 2019 was a landmark offering 1,000 driving segments with camera, lidar and sensor data totaling 100,000 unique frames annotated with 2D bounding boxes for detection. The commercial sensitivities around autonomous vehicles means this kind of data is scarcely available to researchers outside company teams - making Waymo Open an invaluable (though still limited by size) resource for experimenting with approaches that integrate sensor fusion across modalities when training robocars. As self-driving matures and expands out of protected areas, demand for ever more varied autonomous driving data will continue.
This article summarizes 10 of the most widely adopted data sets that have fueled major leaps forward in AI capabilities - spanning core fields like computer vision, natural language processing, speech recognition and others. As algorithms and compute resources continue advancing rapidly, sourcing ever larger, diverse and realistic benchmark datasets is becoming the key bottleneck holding back progress. Going forward, we can expect demand for data across specialized domains from robotics to healthcare that reflects messy imperfect real-world scenarios with all their complexity. Those unlocking access to data with proper consent and privacy controls in place stand to accelerate advances in AI with broad benefits to society.
Comments