Enterprise Wide PST Import - Script Requirements

This is a Part 1 in a series of posts about my experience tackling the migration of PST files.
The first post in the series is here.
The next post in this series is here.


In order to import all the PST files from the user's home drive, we needed some guidelines. And lets not forget about managers. They always want to feel they are adding in value ;)

Users need to know whats going on.
These guys are very wary of anyone touching their PST files. "This is very important data and I may need it someday." So we wanted to be upfront and completely honest.
They get an 'Initial Report' showing how many files were found on their Home share and the state of those files. All files are imported with these exceptions:

• A file of 256K in size is just an empty shell and don't even waste the time trying to import them.
• A file that has not been touched in 2 years is considered dormant and is skipped as well
• If a file has the word "backup" or back up", we assume its a backup 

Next, once the import is complete, they get a 'Final Report' show what files were processed and the result of each. As you might expect, some files are corrupt and can't be imported. The use is also give a very concise list of instructions to disconnect the existing PST files from Outlook.

After 6 days from the Final Report, we send the 1st of the nudge reports. We beg them to disconnect those PST files. We send another after 13 days and after 24 days. All the while we are getting copies of these emails so we know who these people are and can open tickets so they can be assisted.

Even with all these notices, we noticed that people don't have time to read them and they generally ignored them. At least until the GPO set in.

After 12 days, users are added to a GPO that disables PST growth.
We did want to scare people by taking away their PST files. We even say in the 1st email to them, 'We Don't Delete Anything." -- but if we give them an Archive Mailbox we can't let them continue adding messages to PST files. So we compromised and allow them to open an review older PST files, but we do not allow the to add any new messages there. We tell the users they can keep their PST files forever if they want. But we all know that will be changed in some future meeting when The Powers That Be say to kill them all.


Reclaim Space on Home Shares.
Since this all started with file server farms filling up, there was also the dire need of removing the PST files from these home shares.
Certainly a dilemma since we're planning on not scaring the users with the "D" word (delete).
So it was decided to put the users PST on their local drive. This solves the need to clean off Home Shares and the need to not delete the files for the users warm and fuzzy feeling.



Introduction: The Beginings
Part 1: Script Requirements
Part 2: Add-PSTImportQueue
Part 3: Process-PSTImportQueue
Part 4: Some Tools we added
Part 5: Set-PSTImportQueue
Part 6: About PST Capture
Part 7: More PST Import Tools
Part 8: Using RoboCopy
Part 9: Morning Status Report
Part 10: Using BITS Transfer
Part 11: Get the script / Set up
Part 12: The Functions

Enterprise Wide PST Import - The Beginnings

When Exchange 2010 SP1 came out, the  Email Team decided the secondary mailbox, or Archive mailbox, was not really archiving, but it was something we could use.

The Problems:
Our server farm housing our Home Shares were always full and the Server Team were constantly struggling to get 1G back here and 1G back there. Then after all their hard work, before you knew it, the file servers were full again.
The Server Team added 6T of space in the last 2 years. It was beginning to look like they could never keep pace. And, of course, we were reminded over and over that Microsoft does not support PST files on a share in this way.
During that same time, we were testing a product that could do PST file ingestion, but it required something to be installed on the users PC and our delivery system was not working so well. (Let's not even start talking about the red tape!) So for about a year we stayed stuck in this quagmire.
At one point, The Powers That Be came to the Email Team and wanted us "To import these PST files into our archiving software 'right away' and how long would it take to complete?"
If we were going to decide how long it would take to ingest all PST files, we'd be needing some information to go on.  I wrote a quick report in Powershell taking all the people in AD with mailboxes (enabled and disabled) then scanned their home directory for PST files.
To our surprise there was 12T used by PST's. Over 21,000 created by just a little over 3000 people. There was one person that had over 500 PST's. Wow! How could one person have that much time to create that many?
We scratched our heads on how to get these imported with the software we already paid for which doesn't work so good. And also this software that didn't work so good depended on an EXE being delivered to the PC and that process didn't work so good either. So I told The Powers That Be ingesting all PST files will take 2 years. I guessed.

Along comes 2010 SP1 and Archive Mailbox
Since we had been fighting the evil demon of PST files for about a year, without anyone really doing anything or getting anywhere, The Email Team started making plans to import all these PST files into Archive mailboxes and bypass our Archiving Software completely. We were thinking it was time for new Archiving Software and no sense spending time on software we would not be using. So using the Archiving Software for PST ingestion went out the window.

We could use PowerShell to import directly into our Archive Mailbox without touching the PC at all.

We did not want the current mailbox databases to get bloated with new data, so we decided to create new mailbox databases and excluded them from automatic mailbox creation. We added 500G drives to each of our 8 DAG member server and set up mailbox databases. We ended up with 32 drives, equaling 16T.  We also wanted a fail over copy of each database so we really only had 8T. We hoped with de-duplication and a lot of PST files being "backup of backups" we could fit the 12T into the 8T. We found a pilot group and started Importing PST files. We had about 45 users imported when ...

Disaster Happens
Luckily, is wasn't anything to do with Exchange. It was the Home Shares. Turned out the push of Office 2010, many open PST files, and probably no telling what else, used up all the resources on the server farm housing the Home Shares and it went down. There were terrified people running up and down the halls but mainly everyone wanted to know how to get to their PST files! "You have to fix this and NOW!"
You think I'm joking. We had to say we are not on the Server Team, and we just have to wait. More than likely nothing will be damaged and you'll be just fine.

Those 40 people we had already migrated, were very happy their PST (now folders in the Archive Mailbox) were safe and sound.

So how do we convince all the other Users to switch over to Archive Mailbox?
We use the Home Share Disaster to scare the hell out of them!

I started working on the scripts, we were doing our pilot group one by one manually using a snippet here and there. But now we're talking about potentially 100 users at a time. We had to come up with a better script.


This is the intoduction to a series of posts I've made about my experiences using Powershell to import PST files.

Part 1: Script Requirements
Part 2: Add-PSTImportQueue
Part 3: Process-PSTImportQueue
Part 4: Some Tools we added
Part 5: Set-PSTImportQueue
Part 6: About PST Capture
Part 7: More PST Import Tools
Part 8: Using RoboCopy
Part 9: Morning Status Report
Part 10: Using BITS Transfer
Part 11: Get the script / Set up
Part 12: The Functions

About Me and this Blog

My name is Dan Thompson. I am one of several Email Administrators for a large retail chain.
During our work day, we solve many problems small and large - at least to us anyway. I want to journal some of those problems and solutions here. Mainly because I am the worst at documenting my work and I want a place to jot down an idea to work toward, a snippet of PowerShell code I can reuse later, or document some process before I forget all the reasons I wanted to do something some certain way.

But what drives me most is trying to "Find the solution to that problem we had 1 month ago. How did we fix that again?"-- or -- "Where was that link we found that had that thing we loved?"
Yes, I know about SharePoint.  I wanted something outside work.

Our environment is MS Exchange 2010 SP1 and Exchange 2007 co-existence. We have about 65 servers in various DAGS on Dell hardware blades.

These posts will mainly be PowerShell related and how I can do my work faster and better using PowerShell with Microsoft Exchange.