A little backstory, I ran into an interesting problem today I needed to get a 40GB Virtual Machine Image to a storage account in Azure so i could create a Virtual Machine from it. Traditionally, I would download the image to my computer then upload to a storage account in Azure.  😞 Unfortunately I had poor bandwidth, this would've taken me almost the whole day. 💡 to the rescue. Here's what I thought would be easily achieved Azure Cloud Shell download file directly to Azure cloud shell upload directly to Azure Blob Storage from command line Here's what I did not know Azure Cloud Shell by default has very limited storage capacity Azure Cloud Shell creates a file share with a 6GB quota for each user session This file share contains a 5GB linux image which is the OS The file share storage is mounted to a directory called clouddrive AzCopy has a directory in the home directory .azcopy the directory is used as a cache directory that needs storage equal to what you are trying to copy ( ) .azcopy I dont know why, saw a few pointers though - https://github.com/Azure/azure-storage-fuse/issues/130 So to achieve my assumptions Step 1: Increase quota on File Share A default Storage Account exists that contains this File Share, you would need to locate this Storage Account and increase the File Share quota for your session. Step 2: Download the file to directory clouddrive navigate to clouddrive directory clouddrive cd download file here wget https://website-to-get-file Now the file downloads directly to the file share and you can view the file from the file share in the storage account. Step 3: provide storage for AzCopy caching As i specified earlier AzCopy has a directory named in the home directory for caching. and because of the storage limitation you cannot copy large files, even if you manage to get it to the directory. .azcopy clouddrive the error you would get would look like in the image below The workaround was simple, move the entire to the directory. .azcopy clouddrive create a symbolic link for in the home directory, pointing to the in the directory. .azcopy .azcopy clouddrive ~
cp -r .azcopy clouddrive
rm -rf .azcopy
ln -s clouddrive/.azcopy .azcopy cd Now AzCopy has enough storage to perform the copy operation. PS:  The file downloaded is stored to a file share, so we can move to Azure Blob storage in one of two ways. treat file as a local file and upload to blob container. get access url to file from file share and use this to upload file to blob container. 1. 2. you can read more about the Azure Cloud Shell here - https://docs.microsoft.com/en-us/azure/cloud-shell/overview The End.

Microsoft

Too Long; Didn't Read

Developers: Build Less. Deliver More. [Learn how]	

Azure Hack: Use AzCopy on Azure Cloud Shell to Upload Large Files to Storage Accounts

Too Long; Didn't Read

Company Mentioned

Uche Nebed

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

Azure Hack: Use AzCopy on Azure Cloud Shell to Upload Large Files to Storage Accounts

Too Long; Didn't Read

Company Mentioned

Uche Nebed

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES