Migrating WeMove.co from Azure to Ubuntu VM on Digital Ocean

Back in July 2016, when I started writing the code of what will become WeMove.co, I chose to use ASP.NET Core because C# is my go-to static language (I blogged about how I came upon that choice here: http://ezeokoyecelestine.blogspot.com.ng/2016/07/microsofts-confusing-dilemma-with.html). ASP.NET Core was new and very edgy at the time, I was using it at version 1.0 and I faced some really interesting issues. As I alluded in the post, Microsoft did sort out the issues and yesterday, I successfully migrated our website from Azure to Ubutu VM on Digital Ocean.

Why did we migrate?

Long story short: To cut down expenses.

On January 2nd, WeMove Technologies moved into our new office at Bode Thomas Surulere — a thriving business unit in Lagos. Running a business in Lagos, it’s imperative that we cut cost as much as possible. So it’s important I think medium term on how we can fix costs. One of the major strategies for me was either joining the Microsoft BizSpark program or getting rid of Azure, as the cost had started piling.

I’d applied for BizSpark since November and having not gotten any response, I decided to start shopping for an alternative cloud service. Azure’s PaaS is my go-to hosting service because of the ease of setting-up and deployment, directly from my BitBucket repo. Letting go meant I had to set-up a build pipeline while moving.

Even though the primary driver is reducing the cost of business, a close number 2 reason for migration was reduction of error in the future.

• We’ll start hiring soon and I need developers to just write and push code to repo, without breaking anything. I had to battle with the site being down on Christmas day because of a migration wrongly applied to production, while I was trying out a few stuff. I don’t ever want that mistake repeated.

Christmas day woes 😭

• I needed concerns separated. Currently, WeMove code is a C# class library (built in .NET Core) which contains all the business logic and an ASP.NET Core webapp for vehicle hire. When I was coding alone, I did a VS project reference of the class library inside the webapp. This meant that the codes resided within the same solution and a developer cannot focus on working on just the class library, without having all the code on their PC. This increases the likelihood of modifying something mistakenly and pushing to the repo. Hence increasing the likelihood of an error. I don’t want that shit happening.

How did we migrate?

A rough sketch of our build process (made with MS Paint)

Setting up the Code Repo

For all past projects and for WeMove.co, I’d preferred hosting code at BitBucket. Primarily because I get a lot of free private repository and my development team hasn’t exceeded 5. However, as we make plans for new hires, I decided that we need to upgrade our plan on BitBucket.

Then I discovered GitLab.

I went to GitLab to checkout the free CI/CD, but ended up using it because of the unlimited number of team members available (Sorry BitBucket, I still ❤️ you). Ended up not using their CI/CD, instead I rolled out my own build server.

Setting up the Build and Nuget Server

For the build server, we got a Windows Server box at Interswitch Cloud (https://cloud.interswitch.com). I chose Jenkins for the build server because that’s what I have the most experience with and trust.

It took me 1 week (of switching between coding and running the business), 71 builds & copying a folder from my development PC to the server, to successfully get Jenkins to deploy my nuget server as I want it. The major issue was getting Jenkins to make MSBuild to work with Web Deploy (the Nuget server is an ASP.NET 4.5 project). I’m not even sure I remember all the steps involved, so I apologise for not doing a step-by-step guide.

After doing this successfully, it was a lot easier to set-up scripts for the Vehicle Hire ASP.NET Core app.

1. Pull code from the repo after a commit.
2. To build and deploy in Jenkins, run the following Windows Batch Commands:

dotnet restore

dotnet publish -c Release -r ubuntu.16.04-x64 -o “%WORKSPACE%\deploy\path”

jar -cMf ZipPackage.zip %WORKSPACE%\deploy\path\*

curl http://<ubuntu-server-ip>:<hidden-port>/deploy-handler.php?tasktoperform=deploy_wemove --upload-file ZipPackage.zip

Notice that on line 2, I told _dotnet_ to publish for ubuntu, using the -r ubuntu.16.04-x64 option. Yes, I used the JDK “jar” utility to create a zip file on the third line. Did you see what I did there? Also, the curl command on line 4 only works after you download curl for windows from https://dl.uxnr.de/build/curl/curl_winssl_msys2_mingw64_stc/curl-7.57.0/curl-7.57.0.zip

Now I need to go to the Ubuntu server and set it up to accept the ZipPackage.zip. I decided to do this using PHP. That meant writing a little PHP script, which accepts the request and unzip the zip file to a specified /var/www/ path, depending on the action specified for ?tasktoperform=<action>.

Setting up the Ubuntu web server to host ASP.NET Core

For the web server, we acquired a VM from Digital Ocean. It took a little over 10 minutes from registration to the point of setting up the SSH keys. I was super impressed, to be honest.

I created a PHP script which receives the zip from the build server. The port it runs on is only open to the build server’s IP address, configured using the ufw firewall utility. It takes the file and unzips it into the appropriate directory. It also copies the appropriate appsettings.json file for the ASP.NET Core. I’ll explain why below under Gotchas.

The following links contain in-depth, step-by-step guides on how to set-up ASP.NET Core for Ubuntu and configure it to work with Nginx. Then installing as a service, so it starts up with the server. So I won’t bore you by repeating them just follow the links:

Host ASP.NET Core on Linux with Nginx_By Sourabh Shirhatti This guide explains setting up a production-ready ASP.NET Core environment on an Ubuntu 16.04…_docs.microsoft.com

Deploying and hosting ASP.NET Core applications on Ubuntu Linux_If you've read my other blog posts or seen some of my projects on my GitHub profile, you'll see that I'm a real Linux…_blog.bobbyallen.me

— —

Also, I went ahead to set-up Let’s Encrypt as the SSL provider for WeMove. The link below tells you how to do that:

How To Secure Nginx with Let's Encrypt on Ubuntu 16.04 | DigitalOcean_In this tutorial, we will show you how to use Let's Encrypt to obtain a free SSL certificate and use it with Nginx on…_www.digitalocean.com

Gotchas

These are the things that could kill your time if you decide on following these set-up to get ASP.NET Core running on your Ubuntu server.

• ASP.NET Core refusing to pickup environment variables — This was a major problem for me, as I couldn’t figure out how to tell ASP.NET Core to use the database connection I specified in the environment variables. It was totally ignoring them and only worked with the appsettings.json that’s within the application’s folder. As a workaround, I had to create a copy of that appsettings.json (which had all the correct settings), at a different location. Then I copied it into the application folder. This executed as part of the build process, done by the PHP script.
• Cloudflare looping HTTPS redirects — Part of the process of setting up Let’s Encrypt SSL was telling the web server to redirect all HTTP requests to HTTPS. If you configure your DNS/domain proxy on Cloudflare, the HTTPS is stripped and every request is sent to your server as HTTP. This will cause an infinite redirect loop on your server. To fix it, go to your Cloudflare control panel, under crypto > SSL, select “Full (strict)” (as shown below). This will force Cloudflare to send only SSL requests to your server.

Cloudflare settings for default SSL/strict

• Permissions on /var/www/app-dir directory, and other directories your web user uses — Don’t forget to set the appropriate permissions on all the directories to www-data user and group. This was a major headache for me. Make it write restricted to the public by doing chmod -R 775.

So how did this save cost for us in the medium/long run?

You might be wondering that with this complex set-up, how is it that we’re saving more money than just pushing to BitBucket and letting Azure PaaS build it into an App Service. The answer is below:

Not only can it listen to multiple ports, it can also serve multiple (sub-) domains. That means if it’s not being maxed-out or broken in any way, I ain’t setting-up another server to serve WeMove sub-domains.

For each ASP.NET Core application we deploy to Azure, we need to set-up a new App Service. To get SSL, it has to be a tier (or two) above entry payment tier. Average monthly use for us per App Service per tier is between $25 and $30. Other services like SQL Azure raises the cost way higher, especially when you have multiple databases for staging and production.

A Digital Ocean droplet, running Nginx lets us utilise the same server for all our web apps, at a $20 charge. The Nginx lets us reverse proxy to each ASP.NET Kestrel server, depending on the request that comes in.

Our database is still in Azure and I plan to keep it that way for a long time.

Thanks for taking out time to read this.

We’re launching a new service next week, lookout for it.

If you want to hire any vehicle for anything in Lagos, Nigeria, please use our service https://wemove.co. Follow us everywhere on social media: @WeMoveCo

WeMove.co (@WeMoveCo) | Twitter_The latest Tweets from WeMove.co (@WeMoveCo). Need a vehicle in Lagos? Book online. Or call 0808-651-4565. Or email…_twitter.com

WeMove.co (@wemoveco) * Instagram photos and videos_77 Followers, 110 Following, 18 Posts - See Instagram photos and videos from WeMove.co (@wemoveco)_www.instagram.com

WeMove.co_WeMove.co. 177 likes · 11 talking about this. Hire that vehicle you need for moving you, your guests or your goods…_www.facebook.com