Part 1 - Migrating a SharePoint 2007 farm to SharePoint 2013

How Do We Get There?

Our task was pretty straightforward - migrate an entire SharePoint 2007 farm for 300 users, with 3 Site Collections totaling 60 sites and 1.5 TB of data with minimal downtime.  Except when you consider that ensconced in those 60 sites were over 200 SQL Server Reporting Services (SSRS) Reports, many affected SQL Server Integration Services (SSIS) packages, thousands of lines of custom event receiver and webpart code, multiple 3rd-party add-ins (both commercial and CodePlex), InfoPath forms, and many workflows it sounded more like a nightmare.
Except that when you get right down to it, it really wasn't all that bad.


The key is to be organized and methodical - but more on that in a later article.


Our first challenge was to figure out How the Heck Do We Get There?  How could we conceivably get all that content from one farm to another in a minimal amount of time?  Just trying to imagine how we would do that was daunting.  Even just considering the amount of data to move and the speed of our network, we thought we were in a losing position from the get-go.

What Methodology Do We Use?

We did decide on a few things early on, which could be considered success criteria:

• We wanted to keep the site address the same when we went live - we wanted to break as few links, and as few user bookmarks, as possible
• We wanted to minimize confusion - we knew this would be a traumatic experience for our users just with the look-and-feel difference alone, so we wanted to reduce the pain to our users as much as we could
• We wanted to make improvements where we could - SharePoint search was pretty much universally despised by our users, there was a lot of unused 'junk' out there, and if we're spending all this money we should get something more out of it than just a different look
• We wanted to minimize downtime during the migration - without knowing how we would get there, we didn't know how to quantify 'minimal,' but we knew we had to strive for the least amount of downtime we could
• We wanted everything to work - kind of goes without saying, but with all the custom reports and code we had, we had to ensure our users would be reassured

With that decided, we could finally start brainstorming (remember, no idea is the wrong one when it comes to brainstorming - at least that's what they say).  Some of the ideas we considered:

Let's migrate all the sites at one time

"Let's just set that off to the side for later, shall we?"  We know that was a non-starter as soon as the suggestion was put forth.  The of amount of data to move vs. the speed of our network was enough of an obstacle we didn't need any other arguments toward setting this one aside.

Let's migrate one site at a time

Recall I said we had 60 sites, but only 3 site collections. Two of those site collections only had one site each, meaning our main site collection had all the rest.  Our main site collection was the star of the show - each department in our project had their own site, and many had more than one.  Each site had its own set of owners, but everyone on the project had some sort of access to every site in the site collection.  Our main site collection was huge - nearly 1 TB.


Let me step back for just one minute.  I'm sure those of you reading this who are SharePoint admins are raising your eyebrows right about now.  I am a SharePoint admin by circumstance - I didn't design our farm; I just inherited it.  Despite the fact we blew past the recommend site collection size years ago, the SharePoint farm's performance was still really quite good.


With that out of the way, let's return to the topic at hand.  This had a number of concerns that were readily apparent.  Confusion would reign  - users would have to go to two different websites to get their content (with completely different look-and-feel), broken links would abound, we'd be forever fixing links, and we'd be confused trying to keep track of it all.  We would pretty much fail all of our success criteria.

Let's do some reorganization and move the important functionality first

While this would meet the 'make improvements' success criteria, it has all the downfalls of the 'migrate one site at a time method,' while creating even more confusion amongst our users.

Let's pay someone else to do it

After a probe into this solution a year prior to this effort left a bad taste in our mouth, we decided no one would give our site more love than us, so we wanted to do it.

Well, I'm out...

There were some other suggestions that were just variations on the methods already considered,   but about this time we realized something (which may have already become apparent to other SharePoint admins reading this) - we didn't even know what 'migrate' really meant!  Obviously, we knew the general definition of the word, but didn't know how we could even migrate the data!  Since we already knew Microsoft doesn't support upgrading from SharePoint 2007 to SharePoint 2013, we had absolutely no idea what mechanism we could use to actually move the content.  We decided it would probably be wise to figure out what tool(s) we could use - as the tool we chose would probably drive our methodology.  Turns out we were right.

ShareGate Saves The Day

Let's spare you the pain of our migration tool selection process, and skip right to the final result.  We weren't halfway through the ShareGate demo when we figured out a migration methodology that would meet all of our success criteria.


If any of you out there are considering a SharePoint migration, or just have a need to move content between farms or sites, stop looking at all the other tools out there and look at this one.  I'm not getting paid to promote this small Canadian company, but I cannot praise them enough.  We had vendors calling us, "hey, I hear you are looking to migrate to SharePoint 2013; we'd like to help."  They would try to tell us how they were better than the other vendors we were considering, but when we told them we had chosen ShareGate, they hung up and never called back.  ShareGate is reasonably priced, does its job extremely well, and has great support.  But two main things really sold us on it:

• Pricing is not based on the size of the content you are moving (as most solutions seemed to be)
• It has use after migration is complete

ShareGate has some very nice SharePoint management tools, allowing you to manage your sites, lists, and users from within the ShareGate GUI.  It has the ability to create and save reports based on your own criteria that allow you to manage your farm with much greater ease than the SharePoint GUI alone.  All that was icing on the cake, compared to the functionality we really needed.


ShareGate allows you to migrate ANYTHING in your SharePoint site to another SharePoint site and works with 2007, 2010, and 2013 (the only limitation is the destination site can't be SharePoint 2007).  You can migrate at various levels - every list, library and page in a site (and anything below that site); just a library; just a workflow on a library; all SharePoint groups; or just one SharePoint group.  It supports user, group, content type, and column mapping - so your source and destination lists don't have to match exactly (which , in some cases, was a key in meeting the success criteria of making improvements - i.e. some list columns were replaced with site columns with new names that better reflected their meaning).  And the absolutely glorious feature we didn't even know was there until after we bought it - it's fully PowerShell capable.  That feature became one of the key factors in our success.  Another key factor is ShareGate's ability to migrate changes only  - when migrating content using Sharegate, you have to tell it what to do when it encounters the same record in the source and the destination - overwrite it, skip it, or update it with what's changed.  What constitutes 'same record' can be defined by the operator (with Document Libraries, it's always the document name - but with List Items, since IDs are not retained upon migration, you have to create a pseudo-relationship between source and destination records).


There are three main drawbacks to ShareGate (to be fair, these same drawbacks exist in any tool used in a migration):

• Item IDs are not retained upon migration - so an items' ID at the destination will likely not match the ID at the source.  In many cases, this is a non-issue.  In some cases, this is a deal-breaker.  A workaround that worked really well for us - adding a field called ID_Migrated at the destination allowed us to link the source and destination records.
• Person Fields at the destination can only contain users who still exist in Active Directory - just like IDs, sometimes this is an issue, sometimes it's not.  ShareGate does give you the ability to substitute a default user if a user can't be found, but if you do need to retain Person data, creating a text field at the destination to hold the person data will allow you to retain the data (ShareGate gives you the ability to send a single column to multiple destination columns).
• Workflow states cannot be migrated - ShareGate migrates workflow definitions just fine (Nintex workflows too), but it can't migrate the state of a running workflow, so all your workflows should be stopped prior to migration (it doesn't hurt anything, however, to migrate a record that has a workflow running against it).  Note the ability to migrate workflows makes ShareGate very useful to move workflows from a Development environment to a Production environment.

So What Methodology Did We Use?

Once we had a tool in hand, we knew what we could do (and not do).  Ultimately, we decided on a course of action that was never considered in our brainstorming session - we could migrate one list at a time to what we called a mirror, while keeping the same site hierarchy - and once the mirror was complete, we could copy it to Production.  The mirror became an exact copy of our SharePoint 2007 site, but in a SharePoint 2013 farm (which we called DEV).  The content in the mirror was updated twice a week with a single script.  Once we were ready to go live, we simply backed up the DEV instance of SharePoint and restored it to our Production instance of SharePoint.  It took us nine months to complete the mirror.  Once all sites, lists, and libraries existed in the mirror, the mirror could be updated in two and half hours.  The process of backing up DEV and copying to PROD could be accomplished in one hour.  This meant that the definition of 'minimal downtime' for us became 'one business day'.

Conclusion

At the outset, none of us would have believed it if someone said we could migrate 60 SharePoint sites carrying 1.5 TB of data in just six hours.  But using the mirror methodology, we could (and did) achieve all of our success criteria in migrating from SharePoint 2007 to SharePoint 2013, with ShareGate being a critical and integral part of that success.


But wait - there's more!  The next article in this series discusses the next challenge we faced - how do we know what we have to ensure it all get's migrated?

< Migrating a SharePoint 2007 farm to SharePoint 2013 in a Single Weekend | Part 2 - What Have We Got? >

My Task - Migrating a SharePoint 2007 farm to SharePoint 2013 in a Single Weekend

A little over a year ago, my boss called me into her office and asked me to be the Project Manager to migrate our SharePoint 2007 farm to SharePoint 2013.  I asked her, "you mean you want me to DO the migration?"  She just smiled at me with her "thanks for understanding what I'm not saying" smile, so I slunk back to my desk wondering how the heck I was going to migrate an entire SharePoint 2007 farm for 300 users, with 3 Site Collections totaling 60 sites and 1.5 TB of data with minimal downtime.
We utilize SQL Server Reporting Services (over 200 reports), SQL Server Integration Services, custom web parts and event receivers, and 3rd party add-ins (both commercial and CodePlex).  I'm a SQL Server Admin, MCSE, and .NET developer - aside from using SharePoint, I knew very little about how a SharePoint farm worked on any version.  I knew PowerShell existed, but had never used it.
What follows is a multi-part series on how we went about the migration process - high-level planning and preparation, strategy, tools, and processes that went into what was ultimately a very successful migration.  I'm not going to focus on technical aspects such as farm architecture, setting up farm services and service associations, service account configuration, features and things of that nature - articles on those topics have been done to death.  I wish I had kept better track of some of the blogs and sites that were instrumental to our success in order to give them credit, but I only saved a few that helped tremendously along the way:

Thanks to the following, who have no idea who I am, but who also have no idea how much they helped us:
For teaching me that using the Farm Configuration Wizard is not for DBAs who like to control their database names: Mike Robbins' Blog.


For helping me not go crazy after the Farm Configuration Wizard was used on our DEV site: Multiple TechNet Articles.


For a great series of articles regarding installation of Workflow Manager for SharePoint 2013 in a multi-server farm using SSL: Spence Harbar by way of Andrew Connell.


For a helpful article regarding the uninstallation of Workflow Manager for SharePoint 2013 - since it took a lot of trial and error in DEV getting things right in the first place: TechNet Again.


For a supremely helpful series of articles on scaling Enterprise Search in SharePoint 2013: SteveMann's Path


For providing the answer to the question "Why can't I search my external content types?!": Sivarajan's Blog.


For providing multiple answers to the single question, "How do I customize my search results?"
SharePoint IT Pro Blog, Abel Solutions, SharePoint 24x7, and Heather Solomon (I wish I could find it, but I swear the first article I read regarding SharePoint Search that was also the most helpful initially, came from her).

Part 1 - How Do We Get There?