Subscribe to Get All the Blog Posts and Colab Notebooks 

3 Ways to Build Neural Networks in TensorFlow with the Keras API

3 Ways to Build Neural Networks in TensorFlow with the Keras API

Building Deep Learning models with Keras in TensorFlow 2.x is possible with the Sequential API, the Functional API, and Model Subclassing

  Figure 1. The Sequential API, The Functional API, Model Subclassing Methods Side-by-Side

If you are going around, checking out different tutorials, doing Google searches, spending a lot of time on Stack Overflow about TensorFlow, you might have realized that there are a ton of different ways to build neural network models. This has been an issue for TensorFlow for a long time. It is almost like TensorFlow is trying to find its path towards a bright deep learning environment. Well if you think about it, this is exactly what is happening and this is pretty normal for a library in its version 2.x. Since TensorFlow is so far the most mature deep learning library on the market, this is basically the best you can get.

Keras-TensorFlow Relationship

A Little Background

TensorFlow’s evolution into a deep learning platform did not happen overnight. Initially, TensorFlow marketed itself as a symbolic math library for dataflow programming across a range of tasks. Therefore, the value proposition that TensorFlow initially offered was not a pure machine learning library. The goal was to create an efficient math library so that custom machine learning algorithms that are built on top of this efficient structure would train in a short amount of time with high accuracy.

However, building models from scratch with low-level APIs repetitively was not very ideal. So, François Chollet, a Google engineer, developed Keras, as a separate high-level deep learning library. Although Keras has been capable of running on top of different libraries such as TensorFlow, Microsoft Cognitive Toolkit, Theano, or PlaidML, TensorFlow was and still is the most common library that people use Keras with.

Current Situation

After seeing the messiness around the model-building process, the TensorFlow team announced that Keras is going to be the central high-level API used to build and train models in TensorFlow 2.0. The alternative high-level API, the Estimator API, has started to lose its already-diminishing popularity after this announcement.

The Estimator API and The Keras API

 Figure 2. The Positions of the Keras API and the Estimator API within TensorFlow Diagram

Now, let’s go back to the problem: There are a lot of different methods that people build their models using TensorFlow. The main reason for this problem is TensorFlow’s failure to adopt a single Model API.

In version 1.x, for the production-level projects, the go-to model-building API was the Estimator API. But, with the recent changes, the Keras API has almost caught up with the Estimator API. Initially, the Estimator API was more scaleable, allowed multi-distribution, and had a convenient cross-platform functionality. Yet, most of these advantages of the Estimator API are now eliminated, and therefore, soon the Keras API will probably become the single standard API to build TensorFlow models.

So, in this post, we will only focus on the Keras API methods to build models in TensorFlow and there are three of them:

  • Using the Sequential API
  • Using the Functional API
  • Model Subclassing

I will make their comparison directly with their corresponding model building codes so that you can actually test them yourself. Let’s dive into coding.

Initial Code to Make the Comparison

To test these three Keras methods, we need to select a deep learning problem. Image Classification with MNIST is a very straightforward task. What we are trying to achieve is training a model to recognize handwritten digits, using the famous MNIST dataset.

Figure 3. Our Dummy Task for Benchmark Analysis: MNIST Image Classification

MNIST dataset, which stands for Modified National Institute of Standards and Technology database, is a large database of handwritten digits that is commonly used for training various image processing systems. The MNIST database contains 60,000 training images and 10,000 testing images taken from American Census Bureau employees and American high school students. You can find my separate tutorial about Image Classification if you would like to follow the full tutorial.

With the code below, we will import all layers and models so that it would not bother us in the upcoming parts. We also download the MNIST dataset and preprocess it so that it can be used in all models we will build with these three different methods. Just run the code below:

Gist 1. Necessary Imports, MNIST Loading, Preprocessing

Now, this part is out of the way, let’s focus on the three methods to build TensorFlow models.

3 Ways to Build a Keras Model

There are three methods to build a Keras model in TensorFlow:

  • The Sequential API: The Sequential API is the best method when you are trying to build a simple model with a single input, output, and layer branch. It is an excellent option for newcomers who would like to learn fast.
  • The Functional API: The Functional API is the most popular method to build Keras models. It can do everything that the Sequential API can do. Also, it allows multiple inputs, multiple outputs, branching, and layer sharing. It is a clean and easy-to-use method, and it still allows a good level of customization flexibility.
  • Model Subclassing: Model subclassing is for advanced level developers who need full control over their model, layer, and training process. You need to create a custom class defining the model, and you probably won’t need it for daily tasks. But, if you are a researcher with experimental needs, then model subclassing might be the best option for you since it would give you all the flexibility you need.
 Figure 4. Converting 2-Dim Image Array into 1 Dim Array with Flatten Layer

Let’s see how these methods are implemented. We will build a basic feedforward neural network with a single Flatten layer to convert 2-dimensional image arrays to 1-dimensional arrays and two Dense layers.

The Sequential API

In the Sequential API, we need to create a Sequential object from tf.keras.Models module. We can simply pass all the layers as a single argument in the list format as shown below. As you can see that it is very simple.

Gist 2. A Feedforward Neural Network Built with Keras Sequential API

The Functional API

With Functional API, we need to define our input separately. Then, we need to create an output object by also creating all the layers which are tied to one another and to the output. Finally, we create a Model object which would accept inputs and outputs as arguments. The code is still very clean, but we have much more flexibility in the Functional API.

Gist 3. A Feedforward Neural Network Built with the Keras Functional API

Model Subclassing

Let’s move on to model subclassing. In model subclassing, we start with creating a class extending tf.keras.Model class. There are two crucial functions in Model subclassing:

  • __init__ function acts as a constructor. Thanks to __init__, we can initialize the attributes (e.g., layers) of our model. super is used calls the parent constructor (the constructor in tf.keras.Model) and self is used to refer to instance attributes (e.g., layers).
  • call function is where the operations are defined after the layers are defined in the __init__ function.

To build the same model with Model Subclassing, we need to write much more code, as shown below:

Gist 4. A Feedforward Neural Network Built with Keras Model Subclassing

The End Code

Now that you can create the same model with three different methods, you can choose any single one of them, build the model, and run the code below.

Gist 5. Model Configuration, Training, and Evaluation

The lines above take care of model configuration, training, and evaluation. When we compare the performances of these three methods, we see that they are pretty close but slightly different.

Table 1. Performances of Different Keras Methods: The Sequential API, The Functional API, and Model Subclassing
 Figure 5. Photo by Jonathan Chng on Unsplash

Our more complex Model Subclassing method outperforms the Sequential API and the Functional API. This shows that there are slight differences in the design of these methods in the low-end as well. However, these differences are negligible.

Final Evaluations

By now, you have an idea about the similarities and differences between these three Keras methods. But, let’s summarize them all in a table:

Table 2. A Comparison of Different Methods to Build Keras Model in TensorFlow: The Sequential API, The Functional API, and Model Subclassing

In summary, if you are just starting out, stick with the Sequential API. As you dive into more complex models, try out the Functional API. If you are doing a Ph.D. or just enjoy conducting independent research, try out Model Subclassing. If you are a professional, stick with the Functional API. It will probably satisfy your needs.

Let me know your favorite method to build Keras models in the comments.

Eager Execution vs. Graph Execution in TensorFlow: Which is Better?

Eager Execution vs. Graph Execution in TensorFlow: Which is Better?

Comparing Eager Execution and Graph Execution using Code Examples, Understanding When to Use Each and why TensorFlow switched to Eager Execution | Deep Learning with TensorFlow 2.x

 Figure 1. Eager Execution vs. Graph Execution (Figure by Author)

This is Part 4 of the Deep Learning with TensorFlow 2.x Series, and we will compare two execution options available in TensorFlow:

Eager Execution vs. Graph Execution

You may not have noticed that you can actually choose between one of these two. Well, the reason is that TensorFlow sets the eager execution as the default option and does not bother you unless you are looking for trouble😀. But, this was not the case in TensorFlow 1.x versions. Let’s see what eager execution is and why TensorFlow made a major shift with TensorFlow 2.0 from graph execution.

 
Figure 2. An Analogy to Graph Execution vs. Eager Execution (Photo by James Pond on Unsplash | Photo by TVBEATS on Unsplash)

Eager Execution

Eager execution is a powerful execution environment that evaluates operations immediately. It does not build graphs, and the operations return actual values instead of computational graphs to run later. With Eager execution, TensorFlow calculates the values of tensors as they occur in your code.

Eager execution simplifies the model building experience in TensorFlow, and you can see the result of a TensorFlow operation instantly. Since the eager execution is intuitive and easy to test, it is an excellent option for beginners. Not only is debugging easier with eager execution, but it also reduces the need for repetitive boilerplate codes. Eager execution is also a flexible option for research and experimentation. It provides:

  • An intuitive interface with natural Python code and data structures;
  • Easier debugging with calling operations directly to inspect and test models;
  • Natural control flow with Python, instead of graph control flow; and
  • Support for GPU & TPU acceleration.

In eager execution, TensorFlow operations are executed by the native Python environment with one operation after another. This is what makes eager execution (i) easy-to-debug, (ii) intuitive, (iii) easy-to-prototype, and (iv) beginner-friendly. For these reasons, the TensorFlow team adopted eager execution as the default option with TensorFlow 2.0. But, more on that in the next sections…

Let’s take a look at the Graph Execution.

Graph Execution

We covered how useful and beneficial eager execution is in the previous section, but there is a catch:

Eager execution is slower than graph execution!

 Figure 3. The Graph Visualization of the Model Example Below in Tensorboard (Figure by Author)

Since eager execution runs all operations one-by-one in Python, it cannot take advantage of potential acceleration opportunities. Graph execution extracts tensor computations from Python and builds an efficient graph before evaluation. Graphs, or tf.Graph objects, are special data structures with tf.Operation and tf.Tensor objects. Whiletf.Operation objects represent computational units,tf.Tensor objects represent data units. Graphs can be saved, run, and restored without original Python code, which provides extra flexibility for cross-platform applications. With a graph, you can take advantage of your model in mobile, embedded, and backend environment where Python is unavailable. In a later stage of this series, we will see that trained models are saved as graphs no matter which execution option you choose.

Graphs are easy-to-optimize. They allow compiler level transformations such as statistical inference of tensor values with constant folding, distribute sub-parts of operations between threads and devices (an advanced level distribution), and simplify arithmetic operations. Grappler performs these whole optimization operations. In graph execution, evaluation of all the operations happens only after we’ve called our program entirely. So, in summary, graph execution is:

  • Very Fast;
  • Very Flexible;
  • Runs in parallel, even in sub-operation level; and
  • Very efficient, on multiple devices
  • with GPU & TPU acceleration capability.

Therefore, despite being difficult-to-learn, difficult-to-test, and non-intuitive, graph execution is ideal for large model training. For small model training, beginners, and average developers, eager execution is better suited.

Well, considering that eager execution is easy-to-build&test, and graph execution is efficient and fast, you would want to build with eager execution and run with graph execution, right? Well, we will get to that…

Looking for the best of two worlds? A fast but easy-to-build option? Keep reading 🙂

Before we dive into the code examples, let’s discuss why TensorFlow switched from graph execution to eager execution in TensorFlow 2.0.

Why TensorFlow adopted Eager Execution?

Before version 2.0, TensorFlow prioritized graph execution because it was fast, efficient, and flexible. The difficulty of implementation was just a trade-off for the seasoned programmers. On the other hand, PyTorch adopted a different approach and prioritized dynamic computation graphs, which is a similar concept to eager execution. Although dynamic computation graphs are not as efficient as TensorFlow Graph execution, they provided an easy and intuitive interface for the new wave of researchers and AI programmers. This difference in the default execution strategy made PyTorch more attractive for the newcomers. Soon enough, PyTorch, although a latecomer, started to catch up with TensorFlow.

Figure 4. TensorFlow vs. PyTorch Google Search Results by Google Trends (Figure by Author)

After seeing PyTorch’s increasing popularity, the TensorFlow team soon realized that they have to prioritize eager execution. Therefore, they adopted eager execution as the default execution method, and graph execution is optional. This is just like, PyTorch sets dynamic computation graphs as the default execution method, and you can opt to use static computation graphs for efficiency.

Since, now, both TensorFlow and PyTorch adopted the beginner-friendly execution methods, PyTorch lost its competitive advantage over the beginners. Currently, due to its maturity, TensorFlow has the upper hand. However, there is no doubt that PyTorch is also a good alternative to build and train deep learning models. The choice is yours…

Code with Eager, Executive with Graph

In this section, we will compare the eager execution with the graph execution using basic code examples. For the sake of simplicity, we will deliberately avoid building complex models. But, in the upcoming parts of this series, we can also compare these execution methods using more complex models.

We have mentioned that TensorFlow prioritizes eager execution. But that’s not all. Now, you can actually build models just like eager execution and then run it with graph execution. TensorFlow 1.x requires users to create graphs manually. These graphs would then manually be compiled by passing a set of output tensors and input tensors to a session.run() call. But, with TensorFlow 2.0, graph building and session calls are reduced to an implementation detail. This simplification is achieved by replacing session.run() with tf.function() decorators. In TensorFlow 2.0, you can decorate a Python function using tf.function() to run it as a single graph object. With this new method, you can easily build models and gain all the graph execution benefits.

Code Examples

This post will test eager and graph execution with a few basic examples and a full dummy model. Please note that since this is an introductory post, we will not dive deep into a full benchmark analysis for now.

Basic Examples

We will start with two initial imports:

 
timeit is a Python module which provides a simple way to time small bits of Python and it will be useful to compare the performances of eager execution and graph execution.

To run a code with eager execution, we don’t have to do anything special; we create a function, pass a tf.Tensor object, and run the code. In the code below, we create a function called eager_function to calculate the square of Tensor values. Then, we create a tf.Tensor object and finally call the function we created. Our code is executed with eager execution:

Output: tf.Tensor([ 1.  4.  9. 16. 25.], shape=(5,), dtype=float32)

Let’s first see how we can run the same function with graph execution.

Output: Tensor("pow:0", shape=(5,), dtype=float32)

By wrapping our eager_function with tf.function() function, we are capable of running our code with graph execution. We can compare the execution times of these two methods with timeit as shown below:

Output:
Eager time: 0.0008830739998302306
Graph time: 0.0012101310003345134

As you can see, graph execution took more time. But why? Well, for simple operations, graph execution does not perform well because it has to spend the initial computing power to build a graph. We see the power of graph execution in complex calculations. If I run the code 100 times (by changing the number parameter), the results change dramatically (mainly due to the print statement in this example):

Output:
Eager time: 0.06957343100020807 
Graph time: 0.02631650599960267

Full Model Test

Now that you covered the basic code examples, let’s build a dummy neural network to compare the performances of eager and graph executions. We will:

1 — Make TensorFlow imports to use the required modules;

2 — Build a basic feedforward neural network;

3 — Create a random Input object;

4 — Run the model with eager execution;

5 — Wrap the model with tf.function() to run it with graph execution.

If you are new to TensorFlow, don’t worry about how we are building the model. We will cover this in detail in the upcoming parts of this Series.

The following lines do all of these operations:

Output:
Eager time: 27.14511264399971
Graph time: 17.878579870000067

As you can see, our graph execution outperformed eager execution with a margin of around 40%. In more complex model training operations, this margin is much larger.

Final Notes

In this post, we compared eager execution with graph execution. While eager execution is easy-to-use and intuitive, graph execution is faster, more flexible, and robust. Therefore, it is no brainer to use the default option, eager execution, for beginners. However, if you want to take advantage of the flexibility and speed and are a seasoned programmer, then graph execution is for you. On the other hand, thanks to the latest improvements in TensorFlow, using graph execution is much simpler. Therefore, you can even push your limits to try out graph execution. But, make sure you know that debugging is also more difficult in graph execution.

The code examples above showed us that it is easy to apply graph execution for simple examples. For more complex models, there is some added workload that comes with graph execution.

Note that when you wrap your model with tf.function(), you cannot use several model functions like model.compile() and model.fit() because they already try to build a graph automatically. But we will cover those examples in a different and more advanced level post of this series.

Congratulations

We have successfully compared Eager Execution with Graph Execution.

Give yourself a pat on the back!

This should give you a lot of confidence since you are now much more informed about Eager Execution, Graph Execution, and the pros-and-cons of using these execution methods.

Beginner’s Guide to TensorFlow 2.x for Deep Learning Applications

Beginner’s Guide to TensorFlow 2.x for Deep Learning Applications

Understanding the TensorFlow Platform and What It Has to Offer to a Machine Learning Expert

If you are reading this article, I am sure that we share similar interests and are/will be in similar industries. So let’s connect via Linkedin! Please do not hesitate to send a contact request! Orhan G. Yalçın — Linkedin

If you have recently started learning machine learning, you might have already realized the power of artificial neural networks and deep learning compared to traditional machine learning. Compared to other models, artificial neural networks require an extra set of technical skills and conceptual knowledge.

Figure 1. Comparison of Deep Learning and the Traditional Machine Learning Approaches (Figure by Author)

The most important of these technical skills is the ability to use a deep learning framework. A good deep learning framework speeds up the development process and provides efficient data processing, visualization, and deployment tools. When it comes to choosing a deep learning framework, as of 2020, you only have two viable options :

Figure 2. PyTorch by Facebook | Figure 3. TensorFlow by Google

Well, we can compare TensorFlow an PyTorch for days, but this post is not about framework benchmarking. This post is about what you can achieve with TensorFlow.

What is TensorFlow?

TensorFlow is an end-to-end framework and platform designed to build and train machine learning models, especially deep learning models. It was developed by Google and released as an open-source platform in 2015.

The two programming languages with stable and official TensorFlow APIs are Python and C. Besides, C++, Java, JavaScript, Go, and Swift are other programming languages where developers may find limited-to-extensive TensorFlow compatibility. Most developers end up using Python since Python has compelling data libraries such as NumPy, pandas, and Matplotlib.

Why Should We Use TensorFlow?

There are several advantages of using a powerful deep learning framework, and the non-exhaustive list below points out to some of them:

  • Reduced time to build and train models;
  • Useful data processing tools;
  • Compatibility with other popular data libraries such as NumPy, matplotlib, and pandas;
  • A rich catalog of pre-trained models with TF Hub;
  • Tools to deploy trained models across different devices such as iOS, Android, Windows, macOS, and Web;
  • Great community support;
  • A desirable skill by tech companies.

A Brief History of TensorFlow

Currently, we are using the second major version of TensorFlow: TensorFlow 2.x. It took almost nine years to achieve this level of maturity. However, I can say that we are still in the beginning phase of the ultimate deep learning platform because the current trends indicate that deep learning processes will be much more streamlined in the future. Some claims that API based practices will be the standard way of using deep learning and artificial neural networks. But, let’s not get ahead of ourselves and take a look at the history of the TensorFlow platform:

The TensorFlow team deliberately uses the term platform since its deep learning library is just a part of the whole technology.

2011–2016: The Infancy and Initial Developments

I — In 2011, Google Brain developed a proprietary machine learning library for internal Google use, called DistBelief. DistBelief was primarily used for Google’s core businesses, such as Google Search and Google Ads.

I — In 2015, to speed up the advancements in artificial intelligence, Google decided to release TensorFlow as an open-source library. Tensorflow Beta was released.

I — In 2016, Google announced Tensor Processing Units (TPUs). Tensors are the building bricks of TensorFlow applications, and as the name suggests, TPUs are specially designed ASICs for deep learning operations.

Figure 4. Google’s Tensor Processing Units on Wikipedia
ASIC stands for application-specific integrated circuit. ASICs are customized for a particular use such as deep learning or cryptocurrency mining, rather than general-purpose use.

2017–2019: First Major Version and the Advancements in Cross-Platform Technologies

The Developments of 2017:

I — In February, TensorFlow 1.0 was released, setting a milestone. Before February 2017, TensorFlow was still in 0.x.x versions, the initial development process. In general, version 1.0.0 defines the public API with a stable production capability.Therefore, February 2017 was indeed a big milestone for TensorFlow.

I — Seeing the rapid advancements in mobile technologies, the TensorFlow team announced TensorFlow Lite, a library for machine learning development in mobile devices, in May 2017.

I — Finally, in December 2017, Google introduced KubeFlow. Kubeflow is an open-source platform that allows operation and deployment of TensorFlow models on Kubernetes. In other words, “the Machine Learning Toolkit for Kubernetes

The Developments of 2018:

I — In March, Google announced TensorFlow.js 1.0, which enable developers to implement and serve machine learning models using JavaScript.

I — In July 2018, Google announced the Edge TPU. Edge TPU is Google’s purpose-built ASIC designed to run TensorFlow Lite machine learning (ML) models on smartphones.

The Developments of 2019:

I — In January 2019, the TensorFlow team announced the official release date for TensorFlow 2.0.0: September 2019.

I In May 2019, TensorFlow Graphics was announced to tackle issues related to graphic rendering and 3D modeling.

2019–2020: From September 2019 Onwards: TensorFlow 2.0+

I — In September 2019, the TensorFlow team released TensorFlow 2.0, the current major version, which streamlined many of the inconveniencies of building neural networks.

I — With version 2.0, TensorFlow finally embraced Keras as the official main High-level API to build, train, evaluate neural networks.

I — TensorFlow 2.0 streamlined the data loading and processing tools and provided newly added features.

I — Eager Execution was made the default option, replacing Graph execution. This strategy was adopted because PyTorch has attracted many researchers with eager execution.

With Eager execution, TensorFlow calculates the values of tensors as they occur in your code.

As you can see, TensorFlow is much more than a deep learning library for Python. It is an end-to-end platform that you can process your data, build & train machine learning models, serve the trained models across different devices with different programming languages. Below you can see the current diagram of the TensorFlow platform:

Figure 5. The Current Diagram of the TensorFlow Platform (Figure by Author)

How Popular is TensorFlow?

As of 2020, the real competition is taking place between TensorFlow and PyTorch. Due to its maturity, extensive support in multiple programming languages, popularity in the job market, extensive community support, and supporting technologies, TensorFlow currently has the upper hand.

Figure 6. Deep Learning Framework Power Score 2018 (based on Jeff Hale’s Work) (Figure by Author)

In 2018, Jeff Hale developed a power ranking for the deep learning frameworks in the market. He weighs the mentions found in the online job listings, the relevant articles and the blog posts, and on GitHub. Since 2018, PyTorch has achieved an upward momentum, and I believe it must have a higher score by now. But, I believe TensorFlow still has superiority over PyTorch due to its maturity.

I am Convinced! What’s Next?

You have come to this point, and I hope you already developed an understanding of what TensorFlow is and how you can benefit from it. If you are convinced to learn TensorFlow, in the next posts, I will explain the topics below with actual code examples:

  • The very basics of TensorFlow: Tensors, Variables, and Eager Execution; and
  • Five major capabilities of TensorFlow 2.x that cover the entire deep learning pipeline operations.

The second post is already published:

And the third one:

You can follow my account and subscribe to my newsletter:

Final Notes

Over the years, TensorFlow turned into a big platform covering every need of machine learning experts from head to toe. There is still a long way to go, but we are far ahead compared to where we were ten years ago. Join the rise of this new technology and learn to implement your own deep learning models with TensorFlow’s help. Don’t miss out…

Finally, if you are interested in applied deep learning tutorials, check out some of my articles:

4 Reasons Why You Should Use Google Colab for Your Next Project

4 Reasons Why You Should Use Google Colab for Your Next Project

Learn whether iPython, Jupyter Notebook, and Google Colab are Rivals or Complimentary Tools; Understand Their Relationship

Figure 1. iPython, Jupyter Notebook, Google Colab Comparison (Figure by Author)

Colaboratory, or Colab for short, is a Google Research product, which allows developers to write and execute Python code through their browser. Google Colab is an excellent tool for deep learning tasks. It is a hosted Jupyter notebook that requires no setup and has an excellent free version, which gives free access to Google computing resources such as GPUs and TPUs.

In this post, we will cover three topics:

1 — Interactive Python Programming Environments: Python, Jupyter Notebook, and Google Colab;

2–4 Additional Features of Google Colab over Jupyter Notebook; and

3 — How to Create a Google Colab Notebook in 5 Easy Steps.

Since Google Colab is built on top of vanilla Jupyter Notebook, which is built on top of Python kernel, let’s look at these technologies before diving into why we should and how we can use Google Colab.

Please note that this is not a sponsored post. I am sharing my views on a product that I have been using for years.

Interactive Programming Environments

There are several tools used in Python interactive programming environments. The central technology making interaction possible is iPython. IPython is an improved shell and read–eval–print loop (REPL) for Python.

A read–eval–print loop (REPL) is a simple interactive computer programming environment that takes single user inputs, executes them, and returns the result to the user; a program written in a REPL environment is executed piecewise.

“iPython Notebook” is a product developed with iPython accessed as a “notebook” via a web browser. IPython handles two fundamental roles:

  • The Terminal IPython as a REPL; and
  • The IPython kernel, which provides computation and communication with the frontend interfaces such as iPython notebook.

Developers can write codes, take notes, and upload media to their iPython notebook. The growth of the iPython notebook project led to Project Jupyter, which contains the notebook tool and the other interactive tools for multiple languages (Julia, Python, and R). Jupyter Notebook and its flexible interface extend the notebook beyond code to visualization, multimedia, collaboration, and many other features, which creates a comfortable environment for data scientists and machine learning experts.

Let’s get into a more detailed analysis of iPython, Jupyter Notebook, and Google Colab.

iPython

iPython is a command shell and a kernel, which powers interactive Python notebooks. iPython allows programmers to run their code in a notebook environment quickly. The features that iPython provides can be summarized as follows:

  • Interactive shells (Terminal and Qt Console).
  • A web-based notebook interface with support for code, text, and media.
  • Support for interactive data visualization and GUI toolkits.
  • Flexible and embeddable interpreters to load into projects.
  • Parallel computing toolkits.

iPython Project has grown beyond running Python scripts and is on its way to becoming a language-agnostic tool. As of iPython 4.0, the language-agnostic parts are gathered under a new Project, named Project Jupyter. The name Jupyter is a reference to core programming languages supported by Jupyter, which are Julia, Python, and R. As of the implementation of this spin-off decision, iPython, now only focuses on interactive Python whereas Jupyter focuses on tools like the notebook format, message protocol, QT Console, notebook web application.

Jupyter Notebook

Figure 2. Project Jupyter Logo on Wikipedia

Project Jupyter is a spin-off open-source project born out of the iPython Project in 2014. Jupyter is forever free for all to use, and it is developed through the consensus of the Jupyter community. Several useful tools are released as part of the Jupyter Project, such as Jupyter Notebook, JupyterLab, Jupyter Hub, and Voilà. While all these tools may be used simultaneously for accompanying purposes, installing Jupyter Notebook suffices the environmental requirements for a basic machine learning project.

On the other hand, as an open-source project, Jupyter tools may be integrated into different toolsets and bundles. Instead of installing Jupyter Notebook through Terminal (for macOS and Linux) or Command Prompt (for Windows), you can use the Anaconda distribution, which will also take care of the environment installation on local machines.

If you want to take your development experience to the next level, Google Colab, which is a cloud-based Jupyter Notebook environment, is the ultimate tool.

Let’s see why:

Why Should I Use Google Colab?

There are several reasons to opt to use Google Colab instead of a plain Jupyter Notebook instance:

  • Pre-Installed Libraries
  • Saved on the Cloud
  • Collaboration
  • Free GPU and TPU Use

Let’s see these advantages in more detail:

Pre-Installed Libraries

Anaconda distribution of Jupyter Notebook shipped with several pre-installed data libraries, such as Pandas, NumPy, Matplotlib, which is awesome. Google Colab, on the other hand, provides even more pre-installed machine learning libraries such as Keras, TensorFlow, and PyTorch.

Saved on the Cloud

When you opt to use a plain Jupyter notebook as your development environment, everything is saved in your local machine. If you are cautious about privacy, this may be a preferred feature for you. However, if you want your notebooks to be accessible to you from any device with a simple Google log-in, then Google Colab is the way to go. All of your Google Colab notebooks are saved under your Google Drive account, just like your Google Docs and Google Sheets files.

Collaboration

Another great feature that Google Colab offers is the collaboration feature. If you are working with multiple developers on a project, it is great to use Google Colab notebook. Just like collaborating on a Google Docs document, you can co-code with multiple developers using a Google Colab notebook. Besides, you can also share your completed work with other developers.

For me this is a great option since I share all my Colab notebooks with my subscribers thanks to this feature. Subscribe today to get them!

Free GPU and TPU Use

I think this is an absolute no brainer to choose Google Colab instead of a local Jupyter notebook. Google Research lets you use their dedicated GPUs and TPUs for your personal machine learning projects. Speaking from experience, for some projects, the GPU and TPU acceleration make a huge difference even for some small projects. This is one of the main reasons for me to code all my educational projects on Google Colab. Besides, since it uses Google resources, the neural network optimization operations do not mess with my processors, and my cooling fan doesn’t go crazy.

In Summary

Google Colab is just a specialized version of the Jupyter Notebook, which runs on the cloud and offers free computing resources. The relationship between iPython, Jupyter Notebook, and Google Colab is shown in Figure 3.

Figure 3. The relation between iPython, Jupyter Notebook, and Google Colab (Figure by Author)

So, I assume that you are now convinced that you will use Google Colab for your next project. How are you going to set it up?

It’s fairly simple:

Google Colab Setup

The Google Setup process is relatively easy and can be completed with the following steps across all devices:

  1. Visit the Google Colab page, which will direct you to the Google Colaboratory Welcome Page.

2. Click the Sign in button on the right top.

Figure 4. A Screenshot of Google Colab Welcome Notebook

3. Sign in with your Gmail account. Create one if you don’t have a Gmail account.

Figure 5. Google Sign in Page

4. As soon as you complete the sign-in process, you are ready to use Google Colab.

5. You may easily create a new Colab notebook on this page by clicking File> New notebook.

Figure 6. A Screenshot of Empty Google Colab Notebook

Congratulations

You have successfully created a Google Colab notebook within minutes. Now you can start working on your machine learning project.

In this post, we talked about three complementary interactive programming environments. iPython provides a useful kernel to run interactive codes. Jupyter Notebook provides a beautiful notebook with cells that you can add code, media, and text. Finally, Google Colab adds collaboration, free GPU and TPU, cloud features, and additional pre-installed ML libraries.

I can think of two scenarios where you should opt for a local Jupyter Notebook instance:

1 — If you care about privacy and want to keep your code hidden, stay away from Google Colab; or

2 — If you have an incredibly powerful local machine with access to GPUs and TPUs, also local Jupyter Notebook is the way to go.

But, for almost all the other scenarios, I’d recommend Google Colab. I hope this post was helpful in understanding the relationship between iPython, Jupyter Notebook, and Google Colab notebook.