Is Deep Learning a Panacea?

Deep Learning is the current hot topic - unless you've been locked in a faraday cage (http://goo.gl/htfI8v) for the last year you'll have seen that. We have reached a level of compute power in which these techniques are computationally feasible, and companies large and small are looking to Deep Learning to solve their problems.

However, have the expectations outweighed the delivery? To be sure, some of the approaches within the umbrella of Deep Learning are producing some impressive results. One notable example is Google's system which can automatically pinpoint human (and feline) faces without requiring any labelled training data (http://goo.gl/FKmOIN).

At Chatterbox Labs we operate in the space of natural language processing & statistical machine learning for short text (we have backgrounds in Computational Linguistics) and are often asked if this space is going to be impacted by the advances in Deep Learning. It's worth referring now to an excellent position paper (available here: http://goo.gl/Cp99gH) by Professor Chris Manning - arguably the God Father of Natural Language Processing (see his Stanford page: http://goo.gl/YZodoV). Here he presents his case for why Deep Learning will not destroy NLP, arguing that the challenges in the field of Computational Linguistics should focus on the domain science of language technology, not the best machine learning technique per se. I strongly recommend a read of this paper.

And this brings me round to the title of this blog post: Is Deep Learning a Panacea? In a word, no. Don't get me wrong, the techniques under the umbrella of Deep Learning are bringing great scientific results to visual and auditory datasets, and in some cases textual datasets. However, it is worth remembering that:

  1. In a commercial environment, we must focus on the business goals and needs at hand
  2. Deep Learning is a collection of many techniques (mainly for unsupervised, but also supervised, learning) which all have in common the fact that they exploit multiple layers in their representations. There is not one ‘deep learning’ algorithm, but many that are suited to different tasks and domains.

It's important to note that in the same way that I do not see Deep Learning as a panacea, in no way am I advocating writing off Deep Learning as hype. Depending on the domain and business goal at hand, Deep Learning techniques can offer advantages. For example, within the domain of semi-supervised, textual machine learning there are gains to be had within feature selection over hand crafted methods.

What I believe we as an industry need is to manage expectations as to the real business wins that are possible through the use of Deep Learning techniques, potentially used alongside or combined with existing proven methods for addressing our business goals.