Debug Information when a job failed

[lcury]

A feature request that would be tremendously helpful is if Workato would have the Debug logs (raw API requests/responses) available right away when a job is failed. Only adding that information when a job is repeated is just not practical at all when troubleshooting issues and I'll give 2 real life examples where it can be a real pain...

• Lately we've been experiencing sporadic failures with Workato out of the box connector for Salesforce where it would only show the error "Timed out connecting to server". But we know it was not a real timeout proble, because workato has 3 minutes default timeout but the API call failed right away (you could figure this out by the job start time and the job duration). It seems more of a "503 Service Unavailable" where the request is rejected right away, but there's no telling. And in these cases, repeating the job would simply work and we would never have access to the raw logs for the API requests/responses. When working with a partner like Salesforce Support, they can only do so much to troubleshoot on their end if we can't share the raw logs.
• On a more generic note where you're trying to troubleshoot an action (API call) failure but even though the problem would persist, there are times we can't simply repeat the job with the failure on a chain of jobs. When we design more complex integration scenarios, we make use of the Function Recipes and you would have like the parent job that's your trigger then a function recipe call (the first child job) that would act like a dispatch layer, which could potencially call another function (the second child job in the chain) that would eventually call one or multiple actions (third job in the chain). In such design patterns, it's not realistic or safe to rerun just a child job because Workato wouldn't continue running everything up the chain from the failure point. Instead it runs from the first step of that child job and ends right there. This can become a potential problem and cause data loss or inconsistencies. Then you'd think, oh, I can just rerun the parent job from scratch, as it's the right way to do it, right? Correct, but all the chain of children jobs will be new jobs and therefore won't log the actual debug information in it again, only in case these child jobs are individually repeated!

I don't know if other customers experience similar struggle and frustration, but please echo here in case you do in case you see this change bringing real benefits to you and your team as well.

[deven-maru]

Hi Lucas Cury thanks so much for the detailed feedback. Really appreciate you taking the time. You have raised some valid concerns. I would like to explore if we can do something along the lines you've suggested. The debug information on job repeat was done by design. Mainly to avoid extra data load and processing for normal running jobs. Do you have any information on what's the percentage of failed jobs out of total jobs? Can you please create a support ticket which will help track this request?

Thanks

Deven

[lcury]

I'm sorry I missed your response, Deven Maru. When you ask me the % of jobs failing I believe that's the wrong perspective that leads to architectural designs that aren't always helpful.

That 0.1% can cause you a lot of pain and a lot of wasted hours, and possibly if that one critical job that's supposed to run once daily fails and you don't have tools to support you troubleshoot the problem further can affect your business severily.

I can understand what excessive logging can impact performance at scale, but we'd need to have somewhat control to enable that if needed (e.g. ability to enable an account setting to keep debug logs for the last hour or to always keep debug logs for failures.

Would you like me to raise a ticket so you can take the ownership and have internal discussions regarding this request internally? If that's the intent, I'd definitelly raise one for you.

[patrick-steil]

Agree!  And the fact that a Log Message is a "billable" step adds to the pain... 🙂

It is annoying anytime workato cuts off the data output... maybe they could just hold on to it for a certain period of time before scaling it back?

[lcury]

Like, at least let us have some control over it, right?

We understand that performance can be degraded but, if we could for example temporarily enable a Debug flag that would capture and keep all debug information for the last hour or always keep the debug only for failed jobs, that would be REALLY helpful!