setting up recipe for looping through items with individual api authorizations

[rm135]

Workato newbie here, I'm wondering how to migrate an existing coded workflow to workato. 

I loop through a data structure of buildings each with their own api key, and each building has 1 or more tills, each with their own api key for a different cloud service 

e.g.

data = [

    {

        "name": 'property1',

        "ID": 'someGUID',

        "propertyapikey":keyring.get_password('property1', 'property1key'),

        "tills": [

            "till1": keyring.get_password('till1', 'till1key'),

            "till2": keyring.get_password('till2', 'till2key')

        ]

    },

    {

        "name": 'property2',

        "ID": 'someGUID',

        "propertyapikey":keyring.get_password('property2', 'property2key'),

        "tills": [

            "till1": keyring.get_password('till1', 'till1key')

        ]

    }

]

I'm wondering the best way to approach this in workato with storing api keys somewhere, plus my data structure of things to loop through. The lookup table or project settings seem to be simple key value pairs. I could begin my recipe with this data structure defined in the output of a python script, but then all my keys are plain text. 

[gary1 · Accepted solution]

This is all possible to do, but the complexity of your recipes depends on how securely you want to manage the API keys. Task efficiency is also important, but it's not worth discussing until security requirements are understood.

Here are Workato's best practices for sensitive data. In my opinion, following these can often overcomplicate recipe creation, especially in your situation where you need to hot swap API keys: https://docs.workato.com/recipes/recipe-security.html#handling-sensitive-data

The easiest way to store a key in an environment or project property, but this is also least secure because the job logs do not obfuscate the value unless you manually turn on job masking (which IMO vastly limits troubleshooting). If security is paramount, then this approach should not be used. If security is flexible, then this might be the way to go (but still not recommended).

The same security disadvantage applies to any other key storage method (properties, LUTs, etc.) except storing the key as part of the HTTP connection (this is the recommended method). However, this is where things can get more complicated because swapping out the HTTP connection requires some additional framework. 

Assuming you need to securely store your keys, here's how I would approach building the workflow using two recipes and a lookup table:

• Assume you need to call 10 properties with 10 tills each -- 100 unique key combinations
• To make a secure API call using the HTTP connector (following best practices), you need to create an HTTP action connection for every unique key combination
• You then need to make a lookup table mapping your HTTP connections by ID to each property/till combination
• Next you make a function recipe that makes your API call using an HTTP action. In the recipe, you need to configure the settings to allow "Runtime user connections" on the HTTP action. This will allow you to swap out the HTTP connection value when you call this function from the next recipe
• Lastly, you make an outer/controller recipe to kick off your process. You start by getting the LUT of HTTP connections, then looping through each entry, calling your API function and swapping out the connection using the HTTP connection ID lookup

I hope this helps. You said you're a newbie, so let me know if some (or all) of this doesn't make sense! : )

[rm135]

great! runtime user connections on the http actions is a great tip. I'll try that.  

What is the best way to store the initial data structure of [{properties, propertyID, httpActionID, [{tillid,tillHttpActionID..]}...] ? 

It's not a flat table data structure given the variable amount of till objects per property. I'm thinking I could create two tables of properties and tills and use the workato sql collection to do a sql join? any other ways?

[gary1]

There's no way to "securely" store the data structure as is, so I would convert it to a flat table. Keep in mind I have no idea what other input your API call needs, so I'm focusing just on the basic orchestration and key management.

Here's my take on the lookup table:

lut_key	property_name	property_guid	http_connection_id
property1_till1	property1	111-222-333	1
property1_till2	property1	111-222-333	2
property2_till1	property2	222-333-444	3

The keys will get securely stored in the connections, and each connection will have an ID (created by Workato). When iterating through your list of property-tills, you'll have to look up the HTTP connection ID and pass that into the recipe making the API call.

Here's what I don't know that probably has a big impact on the approach:

• How do you know which properties/tills to call? Do you always call all of them, or only call some of them?
• What other data needs to be passed to the API, and how is that structured? Does it include something that could be used to deterministically map to the lut_key in the above, or otherwise tell you "this data is for this property and till"?

[rm135]

Luckily I always call all properties/tills and the authentication is the only thing that changes between API calls. 

thanks this has been super helpful