As previously discussed we have a huge amount of data in the current database, we know we cant take it all – and wouldn’t want to anyway – so we have to decide what to keep and also have a few issues to consider
- Primary Keys – the auto-generated GUID’s have been used to associate records in CRM together and to integrated systems, we heed to find out if we can retain them or find some way to handle them changing
- Email Attachments – these make up the majority of the data in raw bits, we need to define a cut off time for emails and consider if we even need the attachments
- Users – oh yes, all the users are having new accounts created in Office 365 so their names and email addresses will be the same but they will be different accounts, another issue with linking records together to resolve
There is a lot of information about the methods we could use to migrate the data from csv files through third party tools like scribe to had crafted batch scripts.
We have a good long look at all of these and decided in this case to use a combination of csv files and hand crafted importer scripts written in c#. We have made this choice largely because of the users preference for cleansing the data in Excel rather than directly in CRM and because for one off jobs like this c# tends to be quicker then SSIS.
Dark Matter Diary 