PyTorch vs TensorFlow! What to Choose?
October 05, 2020
These days Deep-Learning applications are increasing at a rapid rate, from self-driving vehicles to real-time person identification to grocery-shopping; it seems that AI will soon be an integral part of our daily lives. Deep-learning is an essential component of all state-of-the-art AI-based solutions. The development of professional and stable AI solutions is a monumental effort requiring the investment of hundreds of manhours. Consequently, it is generally quite wise to spend some time thinking about which tools to choose to develop your solutions. So today, I want to compare two of the most popular platforms being used for building AI and Deep-learning solutions: PyTorch and TensorFlow. To avoid making the information bookish, I reached out to a few practitioners who have at least a couple of years of industry experience and got their feedback about their experience with using both these platforms. Shout to Umar Khan (and his colleagues) at D-Cube Technologies and Osama Dar at Sky Electric for taking out to provide feedback. Now without further ado, let’s get to it.
For every 1 developer using PyTorch there are 3.4 developers using Tensorflow.
Clash of the Titans…!!
Whenever we consider buying or using anything, the first metric we employ (whether we like to admit it or not) is prestige. From this perspective, it wouldn’t be unfair if I stated that both PyTorch and TensorFlow have royal bloodlines since one is developed by Facebook (PyTorch) and the other by Google (TensorFlow). Given that these two tech giants are behind PyTorch and TensorFlow, it seems apt to label this a “Clash of the Titans” whose purpose to dominate the global AI landscape for as long as possible.
Let’s talk about the workflow standards they (Google and Facebook) follow!
The purpose of both TensorFlow and PyTorch is identical i.e., enable develop to build deep-learning solutions, however, their execution pipelines differ drastically from each other! In TensorFlow, the code is first compiled in a static graph-like manner, which means that at first the entire code sketch is drawn just like the static computational graphs, and at the end entire graph of code is executed when some special TensorFlow commands/functions are called like running the sessions. So, after creating and running the session, the entire sketch of code begins to run from top to bottom.
In PyTorch, the lines of code we type execute as soon as the compiler reaches them during execution; it is, therefore, more pythonic in this sense and the compiler does not wait until the end of the entire code but rather performs dynamic computations. One of the big advantages of this approach is that it makes writing programs much easier than the computational graph-based approach (employed by TensorFlow).
One commonality between both PyTorch and TensorFlow is that both have very active developer communities and solid tech teams (at Google and Facebook) due to which new updates and features are released frequently. In January 2019 Google released TensorFlow 2.0 which became officially available to the public in September 2019. This release introduced some major changes, the biggest one being its direct execution-style. This was an effort to make TensorFlow more Pythonic (along the same lines as PyTorch) and received significant attention from the AI community. However, usage-wise (at-least at this point) PyTorch is still relatively easier to use than TensorFlow. Based on input from developers, this issue persists primarily because many older code repositories are still in the older version of TensorFlow, and they become very buggy when converted to TensorFlow 2.0. Furthermore, developers also have to take care of the additional task making older repositories consistent with the new version of TensorFlow, meaning that developers often end up rewriting a lot of code again. However, if you remain completely confined within TensorFlow 2.0 you should be fine and safe (and happy too).
Which one has more Complex Code?
Just like we have different levels of programming language, we can also categorize those frameworks into high-level and low-level frameworks. PyTorch, as mentioned above, is more pythonic and has simple commands. So, it is easier to understand. Also, debugging and prototyping in PyTorch is generally quite easy. TensorFlow on the other hand is more of a low-level framework, which means that it has more advanced operations and in-depth accessibility in building models giving developers more power, but this comes at the expense of more code complexity.
I have a great idea to build an application for public use, so which framework will suit me best?
First of all, give yourself a pat on the back for coming up with a cool idea. In terms of production and application-level modeling, TensorFlow is undoubtedly ahead in the race as Google has already taken care of the deployment point from the very beginning. TensorFlow has great application support and has technologies like ‘TensorFlow Serving’ that provides high performance serving systems designed specifically for production environments’ as well as ‘TensorFlow Lite’ for mobile applications. They can be used directly to build your own custom applications easily and plug them into your production pipeline.
Still, it doesn’t mean that PyTorch has no support in this context at all! There are also numerous tools that provide an application-level building with Pytorch such as Torch Script, JIT, PyTorch Pruning, and many others.
Below are some pieces of advice from one of the ML engineers I discussed with:
The classic narrative of PyTorch being inefficient vs TensorFlow is only impactful and true if your computations are in thousands, heck not even thousands, in fact when they are in as much of a massive scale as let’s say google Vision API.
I am more of a research-oriented person and want to build models for experimenting and analysis, which framework should I use?
I personally think that Pytorch is the better choice for researchers! With its pythonic structure, it is very easy to understand and its dynamic coding enables researchers to experiment with every part of the model. In research and model analysis, it is a must to know every tiny part of code and model, so Pytorch provides a lot of flexibility! It also provides rapid prototype modeling which means you can test, and consequently publish, your ideas far more quickly than the rest of the pack (NIPS & CVPR here I come!).
Lastly, which framework has the Best Community support?
For most of the programmers and data scientists, this question is really important since the community size of particular tool/technology is generally reflective of how much support is available and how easily can one get rid of any unforeseen issues that may arise.
TensorFlow is an older framework and thus has a huge global community of developers, you will find a large number of posts on websites like Stack overflow and more learning resources related to TensorFlow. These days, I am working on optimizing AI models on the Nvidia Jetson TX2 using the official Nvidia’s optimization library TensorRT and I have come to know that, for the models built in TensorFlow, TensorRT provides a direct conversion tool for it to optimize while for PyTorch models, we have to use the intermediate step (i.e., the ONNX parser) to proceed with the optimization process.
But still, Pytorch is not far behind in this race, it too has a sizeable community which is quite helpful. Even the biggest Autonomous Car Company Tesla is using Pytorch for building its complex AI models for autonomous driving.
Both TensorFlow and PyTorch are great frameworks for learning and implementing deep learning. There is no clear-cut winner as such (apologies for the disappointment) since it really comes down to what the users are looking to do; both have their pros and cons. You are the one to decide which one will suit you more! So different people from different backgrounds will prefer one over the other depending on their specific needs.
I hope you found this blog useful. Feel free to send me your feedback and do share your own experiences with TensorFlow and PyTorch.
*Disclaimer: The views and opinions expressed in this blog are those of the author and do not necessarily reflect the official policy or position of AI Lounge.
Written By Dr. Adnan ul Hassan April 27, 2019You would hear a lot about Artificial Intelligence (AI) from media to influential people, from laymen to experts; how it is going to transform our world in future. But seldom we realize how much has it already changed our world and our daily lives, from...
Stay Up to Date With The Latest News & Updates
Subscribe AI Lounge
Enter your email address to get latest updates about AI Lounge
Follow us on social media to get latest updates about upcoming courses