Dear Hacker Noon community,
We’d like to apologize for our site going down from Sunday 12:01 a.m. to Monday 9:05 a.m. PST. It is now live. Contributor trust, reader trust, brand equity — we burnt some. It sucks. I’d like to share what happened, and let you know we’ve been working hard — with others — so this won’t happen again. Hope you can continue to spend some time reading with us.
The down time was caused by not updating the SSL certificate. Not updating an SSL certificate is such a stupid f*cking reason for a site going down.
If you build with a Medium publication, you can’t implement your own SSL certificate. Medium said it “covers setup, an SSL certificate, and ongoing support for domains.” This miscommunication of who, how, and when a SSL certificate would be renewed for a global top 10K site and U.S. top 5 K site was regrettable, and I hope we can use this situation to work better together.
After changing diapers at 4 a.m. and 6 a.m. on Sunday, I opened my phone at 8:50 a.m. on Sunday to a barrage of tweets, DMs, texts, and emails saying our SSL certificate had expired. This is what I love about the Hacker Noon community — they don’t tweet about a site being down, they tweet about why a site’s down and what needs to be done to fix it. Berik Dossayev from Medium support had emailed me at 1:34 a.m. saying, “Looks like our automatic renewal failed for HackerNoon.com.” Why would it fail? I replied at 8:57 a.m. And played the waiting game… I also DM’d support, reached out to my other contacts at Medium. Reached out in so many ways, I didn’t realize Medium consolidated the ticket, and replied to me on a chain I wasn’t following. Wrote this post. Ate lunch with family. Tried to forget about the cost of working with Medium. Then I found out my uncle died. RIP. I ignored this problem for awhile. Spent some more time outside. Easter Sunday continued. Grandma offered to take the morning shift changing diapers with our newborn. It’s the little things in life, like sleep. At 8:29am Monday, I received further clarification from Medium support team — that the auto renewal for SSL had failed. We implemented new CNAME information. Moving forward all AMI publications are setup for renewal of SSL certificates.
Hacker Noon averages 65,000 daily readers. Going down for 33 hours has damaged trust with 89,000+ readers, and many contributors. Our contributors are the foundation of this community, our readers are the ones who scaled this community, and by going down we compromised their trust today. Again, it sucks.
This business has a platform dependency risk. This event reminds me of the cost of relying on others. That’s the internet today. Login with Facebook. Signup with Google. All of tech has platform dependency issues. Mine is with Medium, and we will continue to build with Medium, but in these last 33 hours definitely make me ask the question of what we can do as people on the internet that use platforms to grow and scale our businesses. Are we more efficient collectively sharing resources as a part of a platform compared to on our own? I still think so. I hope you do too. Keep reading with us, and if you’d like to blow off some steam about how stupid this whole down time was, DM me, and I’ll be happy to hop on the phone.
I choose to build on Medium because when I saw it, I knew it was the best blogging platform on the planet. Lets keep working toward that standard. Failure in tech is a part of tech. I see this as an opportunity to strengthen our relationship with the platform.