Enhancement? Data Masking at a Recipe Level and/or Connector Level

[mroldanvega]

It would be great if we could set up the data masking flag at a recipe leval, or at a connector level. In recipes with lots of actions, it is difficult to keep track and manage which actions were masked and which were not.

Anyone has any suggestions for this?

[steven-marissen]

I assume it would also come in handy to have the main recipe provide context to underlaying (possibly shared) callable recipes whether to process that request confidentially or not depending on which main recipe it gets called from.

[deven-maru]

hi Steven Marissens good question. Our approach has been to keep the settings at per recipe level for all recipe types (trigger based or callable). Since the callable recipe is also a recipe on its own, the recipe level settings are available there as well to provide maximum flexibility to the users. It becomes important in cases where the callable recipe is a shared recipe where the context could be different depending on the calling recipe. Since it's a recipe level setting it will apply to all jobs of the callable recipe regardless of which recipe called it.

[tridivesh]

Manuel Roldan-Vega Steven Marissens Deven Maru :

Since turning on data masking turns off logging, you lose the access to the data for troubleshooting. If the primary purpose of data masking is to keep data protected/private from "unauthorized eyes", would a special permission to view logs suffice? Whether it be masking recipes, connections or steps in a recipe, it still does require a good amount of rigor/discipline to turn on masking.

It would be simpler to block users/roles from viewing log data, the same way network tracing access works today?

[mroldanvega]

Tridivesh Sarangi - The only problem i see is that it would 'hide' all the logs, and not logs specific to a connection/application. So it is an all or nothing.

Also, will the role only prevent the user from seeing the data, but would still allow to see the execution table (job id, time, etc.), or would it hide the complete job tab in the recipe?

cc. Deven Maru Steven Marissens

[mroldanvega]

Tridivesh Sarangi - one other consideration here is that with data masking you are not storing sensitive data in the logs, but with the permissions approach you are (just hiding it). I think this has can have implications from a compliance perspective. Yet again, for compliance we could set the masking at a step level.