With links and brief summaries — I’ve read a lot of deep learning papers, but some stand out because of how unique and interesting their ideas are. They may not be the most cited, nor necessarily the most practical— but they definitely get the gears turning. Here they are, in no particular order. “DeepInsight: A methodology…

The Extreme Learning Machine — Some believe that the Extreme Learning Machine is one of the smartest neural network inventions ever created — so much so that there’s even a conference dedicated exclusively to the study of ELM neural network architectures. Proponents of ELMs argue that it can perform standard tasks at exponentially faster training…

The intuition behind an ingenious algorithm — Here’s a function: f(x). It’s expensive to calculate, not necessarily an analytic expression, and you don’t know its derivative. Your task: find the global minima. This is, for sure, a difficult task, one more difficult than other optimization problems within machine learning. …

With an application to machine translation — Recurrent Neural Networks (RNNs) are neural networks that can operate sequentially. Although they’re not as popular as they were even just several years ago, they represent an important development in the progression of deep learning and are a natural extension of feedforward networks. In this post, we’ll cover the following: …

Two general perspectives and some psychology — When one first encounters machine learning, one often rushes through algorithm after algorithm, technique after technique, equation after equation. But it is afterwards that one can reflect on the general trends across the knowledge that they have acquired. What it means to ‘learn’ is a very abstract concept. The goal…

From Francois Chollet: “On the Measure of Intelligence” — Throughout history, the measure of intelligence has been changed wildly to adapt to shifting thought and attitudes towards different groups of people and animals. In recent decades, the possibility of intelligence is being extended towards machines. Many have attempted to articulate criteria of intelligence in terms of language: there is…

The significance of the data distribution — A lot of discourse around deep learning would have you believe that more data is better. This is a fairly intuitive idea. Data represents the ‘real world’ in some capacity, and models training on such data learn a sampled representation of this ‘real world’. …

No-Hassle Deep Learning Model Compression — I was asked to write a review for Nota AI’s NetsPresso Compression Toolkit (NPTK). If you’re not familiar with NPTK, you can read an introduction to it and the compression methods it supports here. I was struck by the ease of using Nota’s compression toolkit to compress models. Rather than…

Nota’s (Free) NetsPresso Compression Toolkit — You’ve worked hard to build a deep learning model that performs well. Now it’s time to take it out of the massive GPU centers and into the standard everyday devices it’ll be used on. Let’s see how it performs! [System crashed] Uh-oh! Your model is too big. “Dammit,” you groan…