By Ravi Somisetty
Mulesoft is a name that came into the limelight when Salesforce announced they were investing in their platform. Mulesoft provides an integration platform for connecting any application, data source or API, whether in the cloud or on-premises. Mulesoft deserves the attention it’s getting, since it is a useful tool that’s continuing to evolve with new features.
I recently had a chance to work with Mulesoft to integrate flat files with Salesforce for one of our customers. The experience was pleasantly eye-opening.
I used Mulesoft 3.4 version. The files that were to be integrated with Salesforce varied in sizes from 2 MB to 165 MB. To integrate these files, you can use a file connector to read the files and the Salesforce cloud connector would then be used to connect to Salesforce. It is that simple. But with the size of these files, you can use the Bulk API to connect as an alternative. As with any middleware tool, the source is read and then is mapped to the target using a mapper. For this, Mulesoft provides a very good mapping tool appropriately called “Data Mapper.” Using Data Mapper to map the source and target for the data is an easy task.
As for how the Data Mapper works, it caches all the data in the source file and automatically tries to map it. It then pushes the data to Salesforce. This works pretty well for small sized files (up to 3 MB), but it fails to do its job when it comes to bigger size files. To resolve this, one of the options that we came up with is to feed the Data Mapper with smaller sized files by breaking down the larger ones.
We wrote custom code to break each file down and feed the Data Mapper the smaller chunks of the data. Mulesoft supports Java APIs, so we had a java component that breaks the file into configurable sized chunks. This solved the issue.
Here is a screenshot that shows what the Mulesoft flow looks like:
Recently Mulesoft came up with their next version called, Anypoint Studio. This version has the batch job capabilities needed. With this version users can configure the batch size and achieve the handling big size files.