Popular Deep Learning Tools – a review

Popular Deep Learning Tools – a review

Deep Learning is the hottest trend now in AI and Machine Learning. We review the popular software for Deep Learning, including Caffe, Cuda-convnet, Deeplearning4j, Pylearn2, Theano, and Torch.

 comments

By Ran Bi.

Deep Learning is now of the hottest trends in Artificial Intelligence and Machine Learning, with daily reports of amazing new achievements, like doing better than humans on IQ test.

In 2015 KDnuggets Software Poll, a new category for Deep Learning Tools was added, with most popular tools in that poll listed below.

• Pylearn2 (55 users)
• Theano (50)
• Caffe (29)
• Torch (27)
• Cuda-convnet (17)
• Deeplearning4j (12)
• Other Deep Learning Tools (106)

I haven’t used all of them, so this is a brief summary of these popular tools based on their homepages and tutorials.

Theano & Pylearn2:

Theano and Pylearn2 are both developed at University of Montreal with most developers in the LISA group led by Yoshua Bengio. Theano is a Python library, and you can also consider it as a mathematical expression compiler. It is good for making algorithms from scratch. Here is an intuitive example of Theano training.

If we want to use standard algorithms, we can write Pylearn2 plugins as Theano expressions, and Theano will optimize and stabilize the expressions. It includes all things needed for multilayer perceptron/RBM/Stacked Denoting Autoencoder/ConvNets. Here is a quick start tutorial to walk you through some basic ideas on Pylearn2.

Caffe:

Caffe is developed by the Berkeley Vision and Learning Center, created by Yangqing Jia and led by Evan Shelhamer. It is a fast and readable implementation of ConvNets in C++. As shown on its official page, Caffe can process over 60M images per day with a single NVIDIA K40 GPU with AlexNet. It can be used like a toolkit for image classification, while not for other deep learning application such as text or speech.

Torch & OverFeat:

Torch is written in Lua, and used at NYU, Facebook AI lab and Google DeepMind. It claims to provide a MATLAB-like environment for machine learning algorithms. Why did they choose Lua/LuaJIT instead of the more popular Python? They said in Torch7 paper that “Lua is easily to be integrated with C so within a few hours’ work, any C or C++ library can become a Lua library.” With Lua written in pure ANSI C, it can be easily compiled for arbitrary targets.

OverFeat is a feature extractor trained on the ImageNet dataset with Torch7 and also easy to start with.

Cuda:

There is no doubt that GPU accelerates deep learning researches these days. News about GPU especially Nvidia Cuda is all over the Internet. Cuda-convnet/CuDNNsupports all the mainstream softwares such as Caffe, Torch and Theano and is very easy to enable.

Deeplearning4j:

Unlike the above packages, Deeplearning4j is designed to be used in business environments, rather than as a research tool. As on its introduction, DL4J is a “Java-based, industry-focused, commercially supported, distributed deep-learning framework.”

Comparison

These tools seem to be in a friendly competition of speed and ease of use.

Caffe developers say that “Caffe is the fastest convnet implementation available.”

Torch7 is proved to be faster than Theano on most benchmarks as shown inTorch7 paper.

Soumith gave his convnet benchmarks of all public open-source implementations.

A comparison table of some popular deep learning tools is listed in the Caffe paper.

There is a thread on reddit about “best framework for deep neural nets”. DL4J also gives DL4J vs. Torch vs. Theano vs. Caffe on its website.

Related:

• R leads RapidMiner, Python catches up, Big Data tools grow, Spark ignites
• Where to Learn Deep Learning – Courses, Tutorials, Software
• CuDNN – A new library for Deep Learning

What is your favorite Deep Learning package?

 

---

 Previous post

Next post 

---

Most popular last 30 days

Most viewed last 30 days

1. Top 20 Python Machine Learning Open Source Projects - Jun 1, 2015.
2. R vs Python for Data Science: The Winner is ... - May 26, 2015.
3. R leads RapidMiner, Python catches up, Big Data tools grow, Spark ignites - May 25, 2015.
4. Top 10 Data Mining Algorithms, Explained - May 21, 2015.
5. Which Big Data, Data Mining, and Data Science Tools go together? - Jun 11, 2015.
6. 9 Must-Have Skills You Need to Become a Data Scientist - Nov 22, 2014.
7. 7 Steps for Learning Data Mining and Data Science - Oct 10, 2013.
8. Top 10 Data Analysis Tools for Business - Jun 13, 2014.

Most shared last 30 days

1. Top 20 Python Machine Learning Open Source Projects - Jun 1, 2015.
2. R vs Python for Data Science: The Winner is ... - May 26, 2015.
3. Which Big Data, Data Mining, and Data Science Tools go together? - Jun 11, 2015.
4. R leads RapidMiner, Python catches up, Big Data tools grow, Spark ignites - May 25, 2015.
5. Popular Deep Learning Tools - a review - Jun 18, 2015.
6. 150 Most Influential People in Big Data & Hadoop - May 27, 2015.
7. Exclusive Interview: Matei Zaharia, creator of Apache Spark, on Spark, Hadoop, Flink, and Big Data in 2020 - May 22, 2015.
8. 21 Essential Data Visualization Tools - May 28, 2015.