The "Hello World" of machine learning, the MNIST database turns 20

Let's not quibble about getting the exact date of this anniversary correct. I was compelled to write because the Association for Computing Machinery (ACM) just recently awarded Yoshua Bengio, Geoffrey Hinton and Yann LeCun with the Turing Award for 2018, the most prestigious accolade in computer science, for the innovations they have made in Artificial Intelligence. Specifically, this "Nobel Prize of computing" was in recognition of their conceptual and engineering breakthroughs that have made deep neural networks a critical component of computing.

But what is the MNIST database?

The MNIST database is a freely downloadable collection of tens of thousands of handwritten digits. You can see part of the database in the background of the picture above. You can also see the key publication which first introduced the MNIST database to the scientific community. Two of the four authors of that paper are the recipients of the aforementioned Turing Award. As early as 1995, the database was used to develop algorithms (i.e. "teach machines") to read numeric dollar amounts scribbled on bank checks. Systems based on those learning algorithms have been reading millions of checks per day since then. Similar systems have also helped the postal service sort mail by reading the zip code. This enabled later developments to practically solve the problem of reading house numbers from Street View imagery, giving a huge boost to city mapping technology.

Hello World!

Equally importantly, the MNIST database serves as a starting point for newcomers to machine learning to begin their exploration of the field, just as budding programmers traditionally start by writing a "Hello, World!" program. As the MNIST official website says, in a rather understated way, it is "a good database for people who want to try learning techniques and pattern recognition methods ..." The authors sign off with "Happy hacking." Since I regularly teach machine learning at the extensions of the University of California at Santa Cruz (with classes in Santa Clara) and Berkeley (both featuring classes taught by talented and sincere faculty), I can attest to the excitement experienced by students who are introduced to algorithms that make use of the MNIST database. I know some of them got quite carried away and tried to buy the MNIST T-shirt on Amazon which, alas, is unavailable at this time. My students are thrilled at their sudden comprehension of how machines can learn to do stuff. And in the case of digit recognition, learn to be as good or even better than humans! No longer is artificial intelligence a highly technical, mysterious domain reserved for the chosen few, they too can begin to participate! Soon, they busy themselves with building deep neural networks.

From my teaching experience, I can see first hand that the MNIST database provides ideal opportunities to explore an indispensable concept around which machine learning applications are built: Linear Algebra. Through exercises involving MNIST, students realize that there is something profound about linear algebra, which is at the foundation of several important machine learning algorithms.

Great ideas inspire more ideas

Once powerful algorithms to demonstrate accuracies of upwards of 99% on the MNIST database were developed, researchers have been hard at work in creating and curating other collections of images that are more challenging for recognition algorithms. This has resulted in collections designed to look like the classic MNIST dataset, while incorporating the variations and vagaries found in real data: The Fashion-MNIST dataset consists of images of apparel e.g. dresses, sandals, coats etc. The Street View House Numbers (SVHN) dataset is a real-world image dataset used to study the significantly harder, unsolved, real world problem of recognizing digits and numbers in natural scene images. The interestingly named notMNIST contains individual images of letters rendered in quirky fonts. A group seeks to introduce the machine learning community to the world of classical Japanese literature with their Kuzushiji-MNIST database. They hope to revive interest in long forgotten ancient Japanese works by keeping alive the cursive Kuzushiji script, which is no longer taught in the official school curriculum. I myself, clearly feeling the excitement, created NMNIST, a version in which the digits appear amidst distracting flotsam and jetsam. (For your convenience, this is offered as a drop-in replacement for MNIST.) And talking of far-out ideas, another group provocatively claims to have trained a moth to read MNIST digits by smell. I am kidding, but only slightly. You can read of their exploits in the paper Putting a bug in ML: The moth olfactory network.

I am aware that several researchers have pointed out that the MNIST database is being overused. Another charge is that it creates a false impression of having "solved the image recognition problem." I definitely acknowledge that it is annoying to hear of enthusiastic practitioners of machine learning who put their brilliant ideas to test on MNIST with great success, while remaining unaware of how their method performs on more realistic data. However, I recognize the joy (and celebrate my students joy) that comes from understanding and building your very own solution to a computational problem that can only be solved using machine learning. We must not forget that the "Hello World!" stage has an important role to play. I found MNIST inspiring and full of possibilities when I first encountered it. I am interested in knowing how MNIST inspired you on your initial machine learning adventures. Do write in comments and let me know.

------------------

Shashi Sathyanarayana Ph.D, the founder of Numeric Insight, Inc has several decades of expertise in scientific programming, algorithm development and teaching at universities and as a corporate trainer for companies. He is an active developer of products and creates highly efficient and effective algorithms. He enjoys creating technical vision and actively engages in understanding customer needs. Although frequently called upon to work on complex algorithms, he believes in the value of keeping it basic; he recognizes the universal customer need for intuitiveness and simplicity in a product.

Image Attributions

The check images used in this article are in the public domain on Wikimedia Commons. They have been annotated and overlaid to meet the needs of this article.

1. Cheque sample for a fictional bank in Canada. Airodyssey at English Wikipedia
2. A check from the First National City Bank of New York, signed by John F. Kennedy while serving as the 35th President of the United States. National Numismatic Collection at the Smithsonian Institution.
3. Handwritten check written by Wilbur Wright to himself and endorsed on the reverse. National Numismatic Collection at the Smithsonian Institution.
4. One of Donald Knuth's reward checks. See Knuth reward check for interesting information.