How to omit empty key-value pairs from Payload

[spencerstone]

Hi all, 

My process is that I have a list of data, I extract the relevant data from the list, add the data to the payload, and then POST that data to a particular endpoint. The key consideration is that when a key-value pair has a null value, that key-value pair should be omitted from the payload. Every system I have seen has a consideration for this, but what I understood to be the solution here is not working. I employed the "SKIP" function in an "If value is present, pass value, if not then SKIP" logic (Figure 1). That said, when the value is not present, the key-value pair is still being included with "null" (Figure 2).

My impression is that this should not be happening. I have had to create separate outbound schemas for every potential combination of fields that are null vs populated, which is insane. What other methods can I employ to achieve the same goal?

Figure 1 

 

Figure 2 

 

Note 1: I am using the Dayforce SDK

Note 2: I have already attempted to use a common data model schema and the the .COMPACT function, however it had unintended consequences for integers in the payload that has made it untenable as a solution. 

#compact #skip #automation #Dayforce

[gary1]

Skip is just another way of providing nil, but it doesn't remove the key.

Only compact removes the key, but the problem with compact is it's not recursive.

If you use compact on an array, it's only going to remove nil items from the array. It's not going to skim through the contents of each item in the array and remove keys with nil values.

My preferred way of solving this is with a Ruby action. Add your array or object as the input, and clean it up using the below code. The "data" object is an example of an array of objects with various null keys to purge.

data = [
  {"one": "a", "two": "b", "three": null},
  {"one": "a", "two": null, "three": null},
  {"one": null, "two": "b", "three": null},
  {"one": null, "two": [1,2,3], "three": {"four": "hello", "five": null}}
]

def deep_compact(obj)
  case obj
  when Hash
    obj.transform_values { |v| deep_compact(v) }.compact
  when Array
    obj.map { |v| deep_compact(v) }
  else
    obj
  end
end

cleaned_data = deep_compact(data)

{
  cleaned_data: cleaned_data
}

 The end result:

{
    "output": {
        "cleaned_data": [
            {
                "one": "a",
                "two": "b"
            },
            {
                "one": "a"
            },
            {
                "two": "b"
            },
            {
                "two": [
                    1,
                    2,
                    3
                ],
                "three": {
                    "four": "hello"
                }
            }
        ],
        "log": ""
    }
}

[Prudvi]

Hi @spencerstone,
Did you try the optional field value on the CDM.
Regards,
Prudvi

[shivakumara]

Hi @spencerstone ,
Here is my solution using formulae:
1. Apply variable.blank?? skip : variable
2. Response - [{}] -->  (apply difference operator) 
Please let me know if you need further clarifications, Happy to help

Thanks and Regards,
Shivakumara K A