Idea to support - Pausing recipes

[mppowe]

If this Use Case applies to you too, consider supporting this Idea

I have a recipe using the Webhook trigger. It receives messages from an outside system, then the recipe writes that data to a target system. When the target system goes down for maintenance, I want the messages to queue up until the maintenance is complete. If I stop the recipe, then the messages are lost. If I keep the recipe running, then each job will fail.
If there were a "pause" status, then the recipe could not fail but the inbound messages retained. Then when "resumed" once the target system is back online, the recipe would process the queued messages.  This pause/resume functionality would be ideal.

[dcornwell]

That's an interesting idea....it could indeed be handy to be able to pause or switch off a recipe without switching off the webhook trigger.

As a workaround, I would look into:

1. Using a Monitor for Errors block with retries over a sufficient duration to cover the outage.
2. Or, using a Lookup Table (if large enough for the volume) or other storage to keep the webhook recipe on and write the events to that, and then have a second recipe (the one that talks to the external system) which reads the events from the table.  That way you could switch it off and the webhook events would still queue up.

Regards, David

[steven-marissen · Accepted solution]

We do this already for some integrations as follows:

During our Salesforce release we don't want notifications that come from an external system to be processed until we say all is ok.

So what we've built is a webhook endpoint to receive ALL notification types from that external system and all that recipe does is triage them based on their type before publishing the notification to it's respective pub/sub topic.

So in short:  External system --> webhook endpoint --> recipe --> Pub/sub topic.

Then we have a second recipe per topic that consumes messages from it in real-time.

During the release of Salesforce, we stop all those recipes, therefor halting the processing. The external system can continue sending us notifications and they will continue to be stored in the pub/sub topics.

Once the release is done, we turn on the recipes again and they resume consumption from their topics.

Hope this helps.

Kr

Steven

[mppowe]

Thanks Steven.  Yea this situation is a good argument for pub/sub.  Though out of curiosity, what "triaging" does your webhook endpoint do, rather than just going straight to the publish topic?

[steven-marissen]

In my example chronology does not matter, throughput does a lot more.

So my webhook recipe puts all paid orders in topic1, refunds in topic2, quote request in another, ....

Therefor each topic can be consumed at maximum speed by a dedicated recipe and leaves room for concurrency increases.

Long story short, keep in mind that you want to make sure that you can keep emptying your topic quicker than filling it and if chronology is important or not. 🙂