topic Re: Data Transformation Using Workato JS Connector in Workato Pros Discussion Board

Data Transformation Using Workato JS Connector

Patel0786 — Thu, 17 Oct 2024 05:52:24 GMT

i created a recipe for data transformation using JS snippet by workato, in this recipe i have a group of JSON object and i'm doing data transformation like converting firstName, lastName etc to - (based on requirement) using JS code.

(in this recipe i put the complete json data in logger)

Recipe:- https://app.workato.com/recipes/54842488?st=22358e2f16126f8b0aa51e349ae0de7633781b47a307c56fdd951d59...

now i want to use the csv data in place of direct data passing in logger message, how to do i don't have an idea.

i tried by parsing the csv data action but it's throwing error.

Re: Data Transformation Using Workato JS Connector

javier-riesco — Thu, 17 Oct 2024 10:37:09 GMT

Hey @Patel0786 , how about a simpler approach WITHOUT the JS code? check this example:

https://app.workato.com/recipes/54939691?st=82110955554ef33874944e291d2ff20788bb9f4c9cac02831996aa649332f5e5

Re: Data Transformation Using Workato JS Connector

Patel0786 — Thu, 17 Oct 2024 11:44:04 GMT

hey @javier-riesco thaks for your solution

- your approach is also correct but i want to Use Js code block to transform the data uisng JS to minimise the task count for data manipulation because if we use for each loop for iterate the record then it will consume one task per row , and in my case there is limitation on task consumption limit by using the JS code we can use the for loop and iteration also possible with less task count.

Re: Data Transformation Using Workato JS Connector

mppowe — Thu, 17 Oct 2024 13:28:22 GMT

The transformations seem simple enough, so I think you could take the input JSON, use the Parse JSON action to convert it to the datapill structure, then use the Parse CSV action where you feed in the JSON list and use the .to_csv function. Then you give it the schema for the CSV and you can do you transformations in that step, minimizing both complexity and task usage.

I didn't create a new recipe of my own to do these exact steps, so there may be a gap somewhere but in general I think this path is probably worth exploring for your solution. Another possibility is to take the JSON and convert it to a Collection, then do you bulk transformations there using SQL statements, then converting that to CSV.

Re: Data Transformation Using Workato JS Connector

javier-riesco — Thu, 17 Oct 2024 14:39:27 GMT

Hi @Patel0786 the proposed method will use just 2 tasks, no matter how many elements are in the JSON list. There’s no need for For Each or loops to generate the CSV file.

Re: Data Transformation Using Workato JS Connector

shivakumara — Fri, 18 Oct 2024 05:17:32 GMT

Hi @Patel0786 ,

I am just curious, whether you just wanted convert incoming payload to CSV contents?

Thanks and regards,
Shivakumara Avadhani