topic Connector SDK CSV Chunking On Size in Workato Pros Discussion Board https://systematic.workato.com/t5/workato-pros-discussion-board/connector-sdk-csv-chunking-on-size/m-p/8185#M3433 <P>We are currently working on a dynamic data migration framework and are developing a custom connector that processes CSV files and sends them to Salesforce without needing a predefined schema.</P><P>The Bulk APis of Salesforce has a limit of about 100MB per batch, we plan to chunk the CSV in custom Connector and then submit multiple batches- similar to how the OOB connector does. &nbsp;</P><P>Any help or pointers to achieve the same will be much appreciated.&nbsp;<BR /><BR /></P> Mon, 11 Nov 2024 19:05:01 GMT ssrikantam 2024-11-11T19:05:01Z Connector SDK CSV Chunking On Size https://systematic.workato.com/t5/workato-pros-discussion-board/connector-sdk-csv-chunking-on-size/m-p/8185#M3433 <P>We are currently working on a dynamic data migration framework and are developing a custom connector that processes CSV files and sends them to Salesforce without needing a predefined schema.</P><P>The Bulk APis of Salesforce has a limit of about 100MB per batch, we plan to chunk the CSV in custom Connector and then submit multiple batches- similar to how the OOB connector does. &nbsp;</P><P>Any help or pointers to achieve the same will be much appreciated.&nbsp;<BR /><BR /></P> Mon, 11 Nov 2024 19:05:01 GMT https://systematic.workato.com/t5/workato-pros-discussion-board/connector-sdk-csv-chunking-on-size/m-p/8185#M3433 ssrikantam 2024-11-11T19:05:01Z Re: Connector SDK CSV Chunking On Size https://systematic.workato.com/t5/workato-pros-discussion-board/connector-sdk-csv-chunking-on-size/m-p/8341#M3487 <P>Hi Srikant,<BR />You can refer to the following article, not sure if it would help.<BR /><A href="https://systematic.workato.com/t5/workato-pros-discussion-board/chunk-large-amounts-of-data-including-files/td-p/8102" target="_blank">https://systematic.workato.com/t5/workato-pros-discussion-board/chunk-large-amounts-of-data-including-files/td-p/8102</A><BR />Regards,<BR />Prudvi</P> Wed, 18 Dec 2024 10:01:06 GMT https://systematic.workato.com/t5/workato-pros-discussion-board/connector-sdk-csv-chunking-on-size/m-p/8341#M3487 Prudvi 2024-12-18T10:01:06Z