Migration of attachments is always a pain for data migration specialists. But choosing the right approach can make it a lot more simple. Typically, there are various questions you need to answer before migrating the attachments:
1. How does the client provide attachments?
2. How many attachments and in wht timeframe do you want to load?
3. What is the size of attachments?
4. Which objects are suitable for the attachments?
How does the client provide the attachments
The pain starts at the very first step. Here, you need to decide whether you are going to export attachments from the client’s legacy system or if the client is going to provide you with an attachment folder. The easier way is to opt for the latter as the client understands their legacy system better and knows what exactly needs to be migrated.
If the size of the entire set of attachments is up to 50 GB and the file count is in millions, it is preferable to use a remote connection. The client may provide you attachments in other servers like Unix Box. If so, you need to copy them using WINSCP to your remote server (maybe an EC2 instance) as it provides you with more space. Additionally, you can leave attachment loads running even when you are offline since the attachments will take time to run. However, for larger file sizes (50-200 GB), you can increase your remote space to accommodate all your files and perform migrations effectively.
Once you copy those attachments to your remote machine, you can use CMD to create a CSV file of the complete attachments listed in that folder. That file will work as a key source file for the attachments. If the attachment names are in different languages — or have some special characters like $,@,%,#, etc. — you can use the CHCP 65001 cmd command to preserve the language changes while creating the .csv file. In the .csv file, you will have all the attachment names under that folder and the name of the .csv file which you just created. So make sure to remove your .csv name from the attachment list and save.
Once the file is ready, you can either import it to the database or create additional columns in your CSV files to generate other required columns like ParentId, path and others.
In most scenarios, when the client provides attachments, the attachment has all the required information of the parent ID. If not, you should request it as you need to know which attachment can go to which parent record in Salesforce.
How many attachments and in what timeframe you want to load
Depending on the count of attachments you need to manage, the load timeline for production is 200,000 per day or approximately 50,000 per day for sandbox (may vary with new Salesforce releases). However, if you need to increase this limit, you will need to contact Salesforce. Salesforce will open a window for 2-3 days in sandbox, where you can load 200,000 attachments per day. The same process will follow for increasing production limit.
What is the size of your attachments
The size of the attachment is the key for attachment migration as it drives the batch size to load attachments. Typically, you need to keep the batch size limit <25MB and <30MB for content version, attachment using Soap API, and <2GB from UI for content version. So, you need to check the attachment size and arrange it in a way that each batch should not exceed the maximum limit. Be watchful of the batch size limits as Salesforce keeps updating it.
Which objects are suitable for the attachments
It is also critical to decide which object should be chosen to use for attachment migration. Here are the key pointers that will help you decide-
- Attachments - Use this object when the file size is <25MB. It will get attached to its parent object.
- Content Version and Feed item - This is getting more popular among Salesforce users as the maximum size of any file that we can attach using UI is 2GB and 38MB using SOAP API. Additionally, once the content gets loaded, you can quickly attach/detach it to its related record ID using feed items without re-inserting/deleting the attachment physically from Salesforce.
- Knowledge article - For Salesforce knowledge articles, you have a 5MB limit for attachments. Knowledge articles are preferred when you have inline images, rich text etc. as there is a lot of space in articles to store rich text description.
Additionally, you can go for notes, files, documents, or new enhanced notes, which are straightforward. But for handling complex attachments, the aforementioned objects will help you meet your requirements more closely.
These approaches will enable you to load your attachments successfully into Salesforce depending on your data and requirements. For large data sets (>100K), Content Version and Feed items are the best ways to handle attachments.
Reach out to chat with us about the most meaningful Salesforce solutions our experts can provide to transform your business. Read more on the Appirio Hub about how Salesforce triggers the next-gen IoT experience.