In this story i would go through how to begin a working on deep learning without the need to have a powerful computer with the best gpu , and without the need of having to rent a virtual machine , I would go through how to have a free processing on a GPU , and connect it to a free storage , how to directly add files to your online storage without the need to download then upload , and how to unzip file for free online . I am going through how i am beginning my deep learning project using google colab that allows you to start working directly on a free Tesla K80 GPU using Keras, Tensorflow and PyTorch, and how i connect it to google drive for my data hosting , I would also share some techniques i have used to automatically download data to google drive without needing to first download them , and then uploading them to google drive , then I would go through how you can unzip your files online , without the need to unzip them in python code . free Tesla K80 GPU Keras Tensorflow PyTorch So to sum all things up i would be talking Google Colab Connecting to Google Drive Downloading data directly to google drive without need to download then upload (from an external link , another google drive) unzip files online Google Colab Connecting to Google Drive Downloading data directly to google drive without need to download then upload (from an external link , another google drive) Downloading data directly to google drive without need to download then upload (from an external link , another google drive unzip files online so lets start Google Colab google colab is a free to use Jupyter notebook , that allows you to use free Tesla K80 GPU it also gives you a total of 12 GB of ram , and you can use it up to 12 hours in row . free Tesla K80 GPU You can either begin a new google colab or upload a github ipyn notebook begin a new google colab or upload a github ipyn notebook so lets begin with how you can create a google colab 1- Begin a new google colab 1- Begin a new google colab Begin a new google colab 1-a) go to https://colab.research.google.com https://colab.research.google.com 1-b) select Google Drive Tab (to save your new google colab to google drive) 2-c) select New Python 3 Notebook (you can also select python 2 notebook) New Python 3 Notebook a blank notebook would be created to your google drive , it would look like this You can change the runtime of your notebook from selecting the runtime button in the top menu , to change which python version you are using choose a hardware accelerator from ( GPU , TPU ) change which python version you are using choose a hardware accelerator from ( GPU , TPU ) 2- upload a github Jupyter notebook to google colab 2- upload a github Jupyter notebook to google colab upload a github Jupyter notebook to google colab you can automatically upload an already made Jupyter notebook from github automatically to colab , this truly helps in testing code on the fly , so lets try it out 2-a) this is a Jupyter notebook that i have created for text summurization Jupyter notebook 2-b) lets also go to https://colab.research.google.com , but this time we would select github tab https://colab.research.google.com 3-c) then we just paste the Jupyter notebook link , and click upload Jupyter notebook You can even read more in this tutorial tutorial As you have seen , its rather easy to start a colab project , but for any real world data project , it would require a truly huge datasets , there are many ways to do this , but to truly achieve the goal of having truly huge datasets , you would need the power of google drive . So lets connect our google drive to our google colab notebook. Connect to Google Drive Connect to Google Drive in the newly created notebook , add a new code cell then paste this code in it #https://stackoverflow.com/questions/52385655/unable-to-locate-package-google-drive-ocamlfuse-suddenly-stopped-working https://stackoverflow.com/questions/52385655/unable-to-locate-package-google-drive-ocamlfuse-suddenly-stopped-working !apt-get install -y -qq software-properties-common python-software-properties module-init-tools!wget https://launchpad.net/\~alessandro-strada/+archive/ubuntu/google-drive-ocamlfuse-beta/+build/15331130/+files/google-drive-ocamlfuse_0.7.0-0ubuntu1_amd64.deb!dpkg -i google-drive-ocamlfuse_0.7.0-0ubuntu1_amd64.deb!apt-get install -f!apt-get -y install -qq fusefrom google.colab import authauth.authenticate_user()from oauth2client.client import GoogleCredentialscreds = GoogleCredentials.get_application_default()import getpass!google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URLvcode = getpass.getpass()!echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} https://launchpad.net/\~alessandro-strada/+archive/ubuntu/google-drive-ocamlfuse-beta/+build/15331130/+files/google-drive-ocamlfuse_0.7.0-0ubuntu1_amd64.deb !mkdir -p drive!google-drive-ocamlfuse drive this would connect to your drive , and create a folder that your notebook can access your google drive from It would ask you for access to your drive , just click on the link , and copy the access token , it would ask this twice after writing this code , you run the code by clicking on the cell (shift enter) or by clicking the play button on the top of your code cell then you can simply access any file by its path in form of path = "drive/test.txt" by this you can load truly huge file , as your google drive can hold up to 115GB and google colab offers a good ram of 12 GB so this would open the way for truly great data apps But a new problem would arise , as most of the data that one would use in his data projects would be in the gigabytes , so to work with them in your google colab you would need to download them , then to upload them , this could really take much time , as most people have really slow upload speeds , which would make working on a multi gigabyte datasets quite an issue , so that what the next section discovers . I have managed to use some services that allow you to remote download links directly to your google drive without need to download them , and then uploading again to your drive . Also there is another service that allows you to have a copy of a file (that is hosted on another google drive ) directly to your google drive , so Lets begin Downloading data directly to google drive without need to download then upload Downloading data directly to google drive without need to download then upload primarily you have 2 scenarios , either the file that you need is hosted on a website , that you can download the dataset through a link the needed file is hosted on another google drive hosted on a website , that you can download the dataset through a link the needed file is hosted on another google drive lets discover the first scenario 1-a) Download from external link 1-a) Download from external link Here we would use multcloud , multcloud is a service that contains a free plan that allows you to Manage your cloud services , one of its services that we would use is its ability to download files directly to your google drive , multcloud You just paste your link , and wait for the link to be downloaded directly to your google drive , using he free plan you would be able to transfer 50GB/month After signing up Fist Add Google Drive to multcloud then authorize your account after sign in , and after registering your google drive , click on your google drive then click on upload then click on upload url then paste your link then you can see your download progress by click on the top button , then url tasks then a menu would appear , where you can track your progress , or even add urls from here 1-b) Download from another google drive 1-b) Download from another google drive sometimes there is a case where your files actually are on another google drive , and you just try to have a copy of them to your google drive I have tried using multcloud in this case , using the direct google link to download the files , but with no success , so i came across Copy, URL to Google Drive , which enables you to easily copy files between different google drives Copy, URL to Google Drive paste your link , name it , then save to google drive after authenticating , you just click save to google drive But most of the datasets are found in a zip format , they would need to be unzipped to be able to deal with them in your project . Some people unzip their files inside google colab using python code , but i find that there are much more productive ways , a way that don’t require writing code and maintaining the code for different zip formats , one of them is using cloudconvert Unzip Files online for free cloudconvert is an app that is available to use with google drive , that allows you to unizp your files for free , it can unzip a huge range of zip formats , so lets begin Fist lets connect more apps in your google drive search for cloudconvert , then hit enter , then add the app then lets open the zip through cloudconvert then you select the format to be extract , then you hit start converting (the checkbox of save to drive must be checked) then your zip would be saved to your drive , this is a much convent way than writing code to unzip through the above sections , we walked through how you can build a data project without need of a strong device , without need of renting one , or without having to have a huge expensive online storage , or without the need to download and then upload , and we presented a way of unziping files online I truly hope this was beneficial to you , I am looking forward to hear your comments , and please tell me if it was beneficial to you .
