Systematic Community

spencerstone · ‎04-03-2025

Hi all,

My process is that I have a list of data, I extract the relevant data from the list, add the data to the payload, and then POST that data to a particular endpoint. The key consideration is that when a key-value pair has a null value, that key-value pair should be omitted from the payload. Every system I have seen has a consideration for this, but what I understood to be the solution here is not working. I employed the "SKIP" function in an "If value is present, pass value, if not then SKIP" logic (Figure 1). That said, when the value is not present, the key-value pair is still being included with "null" (Figure 2).

My impression is that this should not be happening. I have had to create separate outbound schemas for every potential combination of fields that are null vs populated, which is insane. What other methods can I employ to achieve the same goal?

Figure 1

Figure 2

Note 1: I am using the Dayforce SDK

Note 2: I have already attempted to use a common data model schema and the the .COMPACT function, however it had unintended consequences for integers in the payload that has made it untenable as a solution.

#compact #skip #automation #Dayforce

gary1 · ‎04-03-2025

Skip is just another way of providing nil, but it doesn't remove the key.

Only compact removes the key, but the problem with compact is it's not recursive.

If you use compact on an array, it's only going to remove nil items from the array. It's not going to skim through the contents of each item in the array and remove keys with nil values.

My preferred way of solving this is with a Ruby action. Add your array or object as the input, and clean it up using the below code. The "data" object is an example of an array of objects with various null keys to purge.

data = [
  {"one": "a", "two": "b", "three": null},
  {"one": "a", "two": null, "three": null},
  {"one": null, "two": "b", "three": null},
  {"one": null, "two": [1,2,3], "three": {"four": "hello", "five": null}}
]

def deep_compact(obj)
  case obj
  when Hash
    obj.transform_values { |v| deep_compact(v) }.compact
  when Array
    obj.map { |v| deep_compact(v) }
  else
    obj
  end
end

cleaned_data = deep_compact(data)

{
  cleaned_data: cleaned_data
}

The end result:

{
    "output": {
        "cleaned_data": [
            {
                "one": "a",
                "two": "b"
            },
            {
                "one": "a"
            },
            {
                "two": "b"
            },
            {
                "two": [
                    1,
                    2,
                    3
                ],
                "three": {
                    "four": "hello"
                }
            }
        ],
        "log": ""
    }
}

View solution in original post

spencerstone · 2 weeks ago

@shivakumara @Prudvi @gary1

If you are interested in a no-code solution to omitting nested null key-value pairs from a JSON payload, please add signal to the below ideas:

https://ideas.workato.com/app/#/case/415544?cpid=931134c4-fa0c-41ba-815c-c31e90cf9eaa

https://ideas.workato.com/app/#/case/504059?cpid=23919909-12f4-4112-8d25-efba83464960

FYI the only solution our consultant team has found that has worked is the Deep Compact Ruby formula that Gary shared, and a JavaScript code snippet that one of my teammates wrote. There needs to be a better solution here!

To access the ideas, you may need to log out of Workato, go to this link https://docs.workato.com/en/contact-us.html#for-requesting-enhancements:~:text=US%20Data%20Center,op... then log in again! Weird process but it worked for me

gary1 · 2 weeks ago

Thanks for sharing. I voted for it.

While I'm here...

This may have zero relevance for you, but would you mind voting on this request for me? It has languished in the queue for a couple years and it would be a huge improvement for the work I do.

https://ideas.workato.com/app/#/case/288061?cpid=9bc8cfd7-90d4-4bd0-a768-986da12e1afe&section=my_pro...

@shivakumara @Prudvi @mppowe : )

mppowe · 2 weeks ago

Done, and while we're here, what about this one. The recipe diff always flags every Recipe Function call I make, which makes the diff cumbersome and less useful. Does it do that for anyone else? It says I'm the only one who wants it, but... I wonder...

https://ideas.workato.com/app/#/case/476366?currentProductId=caf1efdf-1d7e-4b7b-aeb9-28e4b0f4b25c&cp...

Systematic Community

How to omit empty key-value pairs from Payload