Pytorch is a deep learning framework; a set of functions and libraries which allow you to do higher-order programming designed for Python language, based on . Torch is an open-source machine learning package based on the programming language . It is primarily developed by Facebook’s artificial-intelligence research group and Uber’s probabilistic programming language software is built on it. Torch Lua Pyro PyTorch is more “pythonic” and has a more consistent API. It also has native model exports which can be used to speed up inference. PyTorch shares many commands with , which helps in learning the framework with ease. ONNX numpy At its core, PyTorch provides two main features: An n-dimensional Tensor, similar to Numpy but can run on GPUs Automatic differentiation for building and training neural networks If you’re using anaconda distribution, you can install the Pytorch by running the below command in the anaconda prompt. conda install pytorch-cpu torchvision-cpu -c pytorch The rest of the article is structured as follows: What is Colab, Anyway? Setting up GPU in Colab Pytorch Tensors Simple Tensor Operations Pytorch to Numpy Bridge CUDA Support Automatic DifferentiationConclusion If you want to skip the theory part and get into the code right away, Niranjankumar-c/DeepLearning-PadhAI Colab — Colaboratory is a research tool for machine learning education and research. It’s a Jupyter notebook environment that requires no setup to use. Colab offers a free GPU cloud service hosted by Google to encourage collaboration in the field of Machine Learning, without worrying about the hardware requirements. Colab was released to the public by Google in October 2017. Google Colab Getting Started with Colab Sign in with your Google Account Create a new notebook via File -> New Python 3 notebook or New Python 2 notebook You can also create a notebook in Colab via Google Drive Go to Google Drive Create a folder of any name in the drive to save the project Create a new notebook via Right click > More > Colaboratory To rename the notebook, just click on the file name present at the top of the notebook. Setting up GPU in Colab In Colab, you will get 12 hours of execution time but the session will be disconnected if you are idle for more than 60 minutes. It means that for every 12 hours Disk, RAM, CPU Cache and the Data that is on our allocated virtual machine will get erased. To enable GPU hardware accelerator, just go to Runtime -> Change runtime type -> Hardware accelerator -> GPU Pytorch — Tensors Numpy based operations are not optimized to utilize GPUs to accelerate its numerical computations. For modern deep neural networks, GPUs often provide speedups of . So, unfortunately, numpy won’t be enough for modern deep learning. This where Pytorch introduces the concept of . A Pytorch Tensor is conceptually identical to an n-dimensional numpy array. Unlike the numpy, PyTorch Tensors can utilize GPUs to accelerate their numeric computations 50x or greater Tensor Let’s see how you can create a Pytorch Tensor. First, we will import the required libraries. Remember that torch, numpy and matplotlib are pre-installed in Colab’s virtual machine. torch numpy matplotlib.pyplot plt import import import as The default tensor type in PyTorch is a float tensor defined as . We can create tensors by using the inbuilt functions present inside the package. torch.FloatTensor torch ## creating a tensor rows and columns consisting ones >> x = torch.ones( , ) >> print(x) tensor([[ , ], [ , ], [ , ]]) ## creating a tensor rows and columns consisting zeros >> x = torch.zeros( , ) >> print(x) tensor([[ , ], [ , ], [ , ]]) of 3 2 of 3 2 1. 1. 1. 1. 1. 1. of 3 2 of 3 2 0. 0. 0. 0. 0. 0. Creating a tensor by random initialization To increase the reproducibility, we often set the random seed to a specific value first. >> torch.manual_seed( ) #generating tensor randomly >> x = torch.rand( , ) >> print(x) #generating tensor randomly normal distribution >> x = torch.randn( , ) >> print(x) 2 3 2 from 3 3 Simple Tensor Operations Slicing of Tensors You can slice PyTorch tensors the same way you slice ndarrays #create a tensor >> x = torch.tensor([[ , ], [ , ], [ , ]]) >> print(x[:, ]) # Every row, only the last column >> print(x[ , :]) # Every column first row >> y = x[ , ] # take the element first row and first column and create a another tensor >> print(y) 1 2 3 4 5 6 1 0 in 1 1 in Reshape Tensor Reshape a Tensor to a different shape >> x = torch.tensor([[ , ], [ , ], [ , ]]) #( rows and columns) >> y = x.view( , ) #reshaping to rows and columns 1 2 3 4 5 6 3 2 2 3 2 3 Use of to reshape the tensors -1 indicates that the shape will be inferred from previous dimensions. In the below code snippet will result in a tensor of shape 6x1 because we have fixed the size of rows to be 6, Pytorch will now infer the best possible dimension for the column such that it will be able to accommodate all the values present in the tensor. -1 x.view(6,-1) >> x = torch.tensor([[ , ], [ , ], [ , ]]) #( rows and columns) >> y = x.view( , ) #y shape will be x1 1 2 3 4 5 6 3 2 6 -1 6 Mathematical Operations #Create two tensors >> x = torch.ones([ , ]) >> y = torch.ones([ , ]) #adding two tensors >> z = x + y #method >> z = torch.add(x,y) #method #subtracting two tensors >> z = x - y #method >> torch.sub(x,y) #method 3 2 3 2 1 2 1 2 In-place Operations In Pytorch all operations on the tensor that operate in-place on it will have an postfix. For example, is the out-of-place version, and is the in-place version. _ add add_ >> y.add_(x) #tensor y added x and result will be stored y with in Pytorch to Numpy Bridge Converting a Pytorch tensor to a numpy ndarray is very useful sometimes. By using . on a tensor, we can easily convert tensor to ndarray. numpy() >> x = torch.linspace( , , steps = ) #creating a tensor using linspace >> x_np = x.numpy() #convert tensor to numpy >> print(type(x), type(x_np)) < 0 1 5 ' . '> < ' . '> class torch Tensor class numpy ndarray To convert numpy ndarray to pytorch tensor, we can use to convert ndarray to tensor. .from_numpy() >> a = np.random.randn( ) #generate a random numpy array >> a_pt = torch.from_numpy(a) #convert numpy array to a tensor >> print(type(a), type(a_pt)) < 5 ' . '> < ' . '> class numpy ndarray class torch Tensor During the conversion, Pytorch tensor and numpy ndarray will share their underlying memory locations and changing one will change the other. CUDA Support To check how many CUDA supported GPU’s are connected to the machine, you can use the code snippet below. If you are executing the code in Colab you will get 1, that means that the Colab virtual machine is connected to one GPU. is used to set up and run CUDA operations. It keeps track of the currently selected GPU. >> print(torch.cuda.device_count()) 1 If you want to get the name of the GPU Card connected to the machine: >> print(torch.cuda.get_device_name( )) Tesla T4 0 The important thing to note is that we can reference this CUDA supported GPU card to a variable and use this variable for any Pytorch Operations. All CUDA tensors you allocate will be created on that device. The selected GPU device can be changed with a context manager. torch.cuda.device #Assign cuda GPU located at location to a variable >> cuda0 = torch.device( ) #Performing the addition on GPU >> a = torch.ones( , , device=cuda0) #creating a tensor on GPU >> b = torch.ones( , , device=cuda0) #creating a tensor on GPU >> c = a + b >> print(c) tensor([[ , ], [ , ], [ , ]], device= ) '0' 'cuda:0' 3 2 'a' 3 2 'b' 2. 2. 2. 2. 2. 2. 'cuda:0' As you can see from the above code snippet the tensors are created on GPU and any operation you do on these tensors will be done on GPU. If you want to move the result to CPU you just have to do .cpu() #moving the result to cpu >> c = c.cpu() >> print(c) tensor([[ , ], [ , ], [ , ]]) 2. 2. 2. 2. 2. 2. Automatic Differentiation In this section, we will discuss the important package called automatic differentiation or autograd in Pytorch. The autograd package gives us the ability to perform automatic differentiation or automatic gradient computation for all operations on tensors. It is a define-by-run framework, which means that your back-propagation is defined by how your code is run. Let’s see how to perform automatic differentiation by using a simple example. First, we create a tensor with parameter set to True because we want to track all the operations performing on that tensor. requires_grad #create a tensor requires_grad = True >> x = torch.ones([ , ], requires_grad = True) >> print(x) tensor([[ , ], [ , ], [ , ]], requires_grad=True) with 3 2 1. 1. 1. 1. 1. 1. Perform a simple tensor addition operation. >> y = x + #tensor addition >> print(y) #check the result tensor([[ , ], [ , ], [ , ]], grad_fn= 5 6. 6. 6. 6. 6. 6. ) < > AddBackward0 Because y was created as a result of an operation on x, so it has a grad_fn. Perform more operations on y and create a new tensor z. >> z = y*y + >> print(z) tensor([[ , ], [ , ], [ , ]], grad_fn= <SumBackward0>) 1 37. 37. 37. 37. 37. 37. ) >> t = torch.sum(z) #adding all the values in z >> print(t) tensor(222., grad_fn= < > AddBackward0 Back-Propagation To perform back-propagation, you can just call t.backward() >> t.backward() #peform backpropagation but pytorch will not print any output. Print gradients . d(t)/dx >> print(x.grad) tensor([[ , ], [ , ], [ , ]]) 12. 12. 12. 12. 12. 12. will give you the partial derivative of with respect to . If you are able to figure out how we got a tensor with all the values equal to 12, then you have understood the automatic differentiation. If not don't worry just follow along, when we execute we are calculating the partial derivate of t with respect to . Remember that t is a function of , which in turn is a function of . x.grad t x t.backward() x z x d(t)/dx = 2y * 1 at x = 1 and y = 6, where y = x + 5 The important point to note is that the value of the derivative is calculated at the point where we initialized the tensor x. Since we initialized x at a value equal to one, we get an output tensor with all the values equal to 12. The entire code discussed in the article is present in the Kaggle Kernel. Feel free to fork it or download it. . The best part is that you can directly run the code in Kaggle Kernel once you fork it, don’t need to worry about installing the packages GettingStartedWithPytorch_GPU If Colab is your jam, click to execute the code directly and get your hands dirty. here Niranjankumar-c/DeepLearning-PadhAI Conclusion In this post, we briefly looked at the Pytorch & Google Colab and we also saw how to enable GPU hardware accelerator in Colab. Then we have seen how to create tensors in Pytorch and perform some basic operations on those tensors by utilizing CUDA supported GPU. After that, we discussed the Pytorch package which gives us the ability to perform automatic gradient computation on tensors by taking a simple example. If you have any issues or doubts while implementing the above code, feel free to ask them in the comment section below or send me a message in citing this article. autograd LinkedIn Learn More If you want to learn more about Data Science, Machine Learning. Check out the and by Abhishek and Pukhraj from . One of the good points about these courses is that they teach in both Python and R, so it’s your choice. Machine Learning Basics Advanced Machine Learning Starttechacademy Recommended Reading Deep Learning Best Practices: Activation Functions & Weight Initialization Methods — Part 1 Demystifying Different Variants of Gradient Descent Optimization Algorithm In my next post, we will discuss how to implement the feedforward neural network using Pytorch (nn.Functional, nn.Parameters). So make sure you follow me on medium to get notified as soon as it drops. Until then Peace :) NK. Author Bio is Retail Risk Analyst Intern at HSBC Analytics division. He is passionate about Deep learning and Artificial Intelligence. He is one of the top writers at in . You can find all of Niranjan’s blog . You can connect with Niranjan on , and to stay up to date with his latest blog posts. Niranjan Kumar Medium Artificial Intelligence here LinkedIn Twitter GitHub . I am looking for opportunities either full-time or freelance projects, in the field of Machine Learning and Deep Learning. If there are any relevant opportunities, feel free to drop me a message on LinkedIn or you can reach me through email as well. I would love to discuss Originally published at https://www.marktechpost.com on June 9, 2019.
![What Are Convolution Neural Networks? [ELI5]](https://hackernoon.imgix.net/images/69s32rn.jpg?auto=format&fit=max&w=3840)
