Tuesday, 14 June 2016

5 Reasons Data Migration is Challenging

Data migration is hard. Granted, it's not something that most people think about, and very few have actually tried it. If you haven't, you'll just have to trust us on this one; it's hard. Also, if you're a software developer who hasn't tried it, it's probably much harder than you think. Despite this fact, there are many naysayers out there who will tell you it's a breeze. They glibly point out that system A has an API and system B has an API, so there you go--easy peasy, lemon squeezy (as my daughter would say). If only it were that simple. So why is data migration complicated, difficult, and riddled with booby traps? This post outlines five of the main reasons.

migrate to Google Apps and Google Drive

Before getting to the list, let's first establish that this post is about the technical challenges. User adoption can actually be the highest hurdle to a successful migration, but that's not in the scope of this post. In terms of the challenges to creating migration software, here are some of the thorniest issues we face.

5. People think it's easy so it always takes more time and effort than expected.

As mentioned above, there is a general misunderstanding about data migration out there that it should be easy. This leads to many problems. One of the most common is trying to build a custom migration solution for a project. I've heard about numerous failed migration projects because the company involved quite often has to seek an off the shelf migration solution after their custom solution fails. There simply is no way to argue with the ROI benefit of licensing an existing product rather than developing your own custom migration solution.

4. Data volume and performance are challenges--especially when dealing with the cloud.

Even under ideal performance conditions (e.g., running both the source and target on-premises), performance is an issue when dealing with large volumes of data. The problem is that an organization can literally spend years putting data into a content management system and then ask for that data to be migrated to a new system over the weekend. First of all, getting all the data out of a system is not the same as getting each piece out. When you try to get all the data, you (understandably) can run into all sorts of bottlenecks and intentional throttling. Even if the system has an API that allows one item to be copied out, asking that API to copy millions of things in one go can quickly go pear-shaped (more about that when we get to API issues).

3. There are a lot of file storage and content management systems out there and every company uses more than one.

The reality of today's IT infrastructure is that practically every company uses some sort of on-premises file shares in addition to a content management system, and cloud-based file storage. Taking the time to write migration software for each of these systems is just not feasible. This means migration Independent Software Vendors (ISVs) have to choose which ones they feel will be the best business case. Choosing which platforms to support can make or break a small ISV business.

2. No one wants you to migrate away from their platform so no API is ever built to easily move data out of a system.

Sadly, no API seems to ever get finished. This isn't because developers won't want to finish things; it's most commonly the reality of dealing with limited resources. However, it's also true that there are times when companies make the conscious decision to not provide API read access to everything in their system. By doing so, they enhance the 'stickiness' of their application. In other words, it's harder for customers to jump ship and migrate to another platform. That's one API issue, but it's not the only way. Even if the API is provided, it's built for customer use cases, not the migration use case. So those of us who try and use it for migration, find that it's either way too slow for a migration, or it simply does not work because it's incomplete or throws too many errors.

1. Data migration is often not apples to apples.

When you migrate from one system to another, there are times when it isn't clear where something should go; and sometimes there just isn't a right answer. You can choose pretty much any two content management systems (or file storage systems), and you'll find data in one that doesn't have a home in the other. The reason for this is clear; every company wants to add value to their software. As each system adds features to try and one-up one another, they create more and more islands of data. For example, as I wrote in a previous post, the Google Drive storage system is a graph. If you try and replicate that graph in a traditional file system, you'll quickly run into a wall. It's just not possible; they are built on different (and mutually exclusive architectures).

Data migration may not be easy, but therein lies the fun. Everyone loves a good challenge and those of us here at AppBridge are no exception. If it was easy, we'd probably move onto to some other project.

Note: AppBridge will be presenting a session on Google Apps migration at the Google Atmosphere conference in Tokyo tomorrow.

Here's the info about our session:
Bring your file servers into the cloud! Covering everything from security to data migration

Until now, replacing file servers by going to the cloud has presented numerous challenges to organizations, such as security concerns and how to migrate critical data. In this session, we'll explain in detail how to use Google Drive as a file server in the cloud, including how to migrate the data from existing file servers on premises. Observe how data migration occurs in real-time by viewing a demonstration of AppBridge's data migration solution.

No comments:

Post a Comment