Thursday, October 16, 2014

My journey to the public cloud

In spite of having architected, built and managed some of the largest public cloud infrastructures in Africa, I have until very recently run a private cloud exposed to the Internet for my personal needs.


I have two FreeBSD-based servers (one in South Africa, one in Germany), which for at least 10 years ran a very consistent configuration;

I kept my personal data in Dropbox since its inception, and about 18 months ago moved to Google Drive because of the more compelling pricing model, the pay-per-month option and the ability to search my data. I use a combination of Crashplan to keep a redundant copy in the cloud, and BitTorrent Sync to keep local files available (certain hosts are primary of certain data sets). LogMeIn Pro does a good job of allowing me to remotely manage several computer hosts. I kept my phone contacts in the corporate Exchange cluster, because in spite of all the terrible software they write, Microsoft has done a damn good job of licensing ActiveSync and making it fail-proof,  idiot proof and the de facto standard on every mobile platform out there.
This configuration served my data and personal needs, several vanity domains and similar for several friends and family members for over a decade. The only aspect that changed was upgrades for security purposes and a move from US-based hosting to Germany-based hosting (for cost and privacy reasons).

These servers acted as primary and secondary name servers, primary and backup MX and geo-distributed you to the closest web server for my family photo collection. And until about 6 months ago, I played down the progress in public cloud offerings and was confident in my own abilities as an ex-sysadmin.


Then Heartbleed hit. And then my FreeBSD version was EOL’d. And I had hardware failures, which caused me to stay awake at night, away from my family and friends and ruined an otherwise decent overseas trip. And while the lure of 18 degrees in a data centre used to thrill me, it no longer does. As does the reputation-damaging spectre of being hacked.


I decided to change all of this.


Over a 3 week period, while I’ve kept the Crashplan / Google Drive / BitTorrent Sync configuration to keep my “personal” computing requirements sane, I’ve migrated this configuration to;


Along the way, I tried a mail migration tool called (from Shuttlecloud), that did a great job on small mailboxes, but failed miserably on large ones (2-10 GB). And, they sneakily move you from a direct billing relationship with Google to a reseller model where they bill you and get commission from your business - not cool. I ended up using a mail migration tool called Yippiemove, that while expensive, certainly did end up moving users’ data.


I now have my phone contacts in Google Contacts, and am using a tool called Scrubly to clean it up, populating LinkedIn, Facebook and Twitter data into the contact stream and profile photos. iOS and Android devices use Google Sync (which is licensed ActiveSync) to replicate contact data, Google Drive is my data repo and I’ll be able to shut down my servers from active use by the end of the month.


In this scenario, on-boarding a new compute platform, tablet, phone or similar is a 2 minute exercise - and I have access to all my data.


Yes, this is costing me some money - but nowhere near the hosting costs I would have incurred if I wasn’t working for an ISP. You’re looking at $5 per user month for Google Apps, $30 annually for hosting 10 DNS zones, $60 annually for Zenfolio, $30 annually for the contact cleanup utility and I’m springing an additional $10 per month for 1Tb of storage. Crashplan is about $6 per month for unlimited storage and backup of 10 computers.


Ultimately, the headache of keeping these systems in sync / up to date and secure is now not my problem. For me, that’s money well spent.


PS. The biggest headache was merging my and my users’ Google accounts (Plus, Voice, Drive, Chrome Sync, etc) from being unmanaged consumer-level Google accounts to managed Google Apps accounts. This migration is still too damn hard.