topic Re: Method to check for invalid URLs in Workato Pros Discussion Board

Method to check for invalid URLs

bedelman0731 — Thu, 27 Feb 2025 16:00:02 GMT

Is there a method by which a URL can be checked within a recipe to see if the URL is valid?

I have a recipe that, among other things, is constructing a URL that should be a reference to an image based on one of the ID of a data element. The issue arises in that the image has not always been uploaded and the URL that should point to the image returns a 404 (Page Not Found).

The output of the recipe is a CSV file where each row contains what "should" be the URL to the image, but (in some cases) the image has not been uploaded and the URL that it would have then throws a 404.

I did find a App Script for Google Sheets that will check the URLs - but this requires importing the CSV into Google Sheets, adding in the App Script, adding a column that then uses the App Script function to return the status of the URL. All of these are additional steps that I would prefer to avoid and merely do the URL checking within the recipe itself.

Is there a way to implement such a URL checker into a recipe in Workato?

Re: Method to check for invalid URLs

Prudvi — Thu, 27 Feb 2025 16:35:33 GMT

Hi @bedelman0731,
You can try using a execute python code action which will take URL as input and StatusCode as output.
Python code will make a request to the URL and captures the StatusCode
Based on the value of StatusCode you filter the URL from being added to the CSV.
Regards,
Prudvi

Re: Method to check for invalid URLs

gary1 — Thu, 27 Feb 2025 16:51:03 GMT

There's no need for Python. You can do this using the HTTP action. Set the HTTP action to treat non-2xx status codes as success and log the status code after the call. (Edit: Alternatively you can wrap the HTTP action in an error handler to catch non-2xx codes.)

If you want save tasks, you could use Python or JS to loop through the URLs and return an array of results. If you're checking thousands of URLs this could save quite a few tasks.

Re: Method to check for invalid URLs

bedelman0731 — Fri, 28 Feb 2025 14:43:26 GMT

I think I will use a python code action to do this. While I am not proficient in writing Python code from scratch, I used ChatGPT to help me out to come up with the following function that appears to work. The input field is "input_data" and the output field is "Response"

In this usage, I only care about results that do not yield a 200 response

import requests def get_status_code(input_data): url = input_data.get("input_data", "").strip() # Extract URL from input data if not url: return {"status_code": None} # Ensure response is always a dictionary try: response = requests.get(url, allow_redirects=False) # Similar to 'followRedirects': false return {"status_code": response.status_code} # Return as dictionary for Workato mapping except requests.exceptions.RequestException: return {"status_code": None} # Handle request errors safely # Main function that Workato calls def main(input_data): return get_status_code(input_data) # Ensure output is always a dictionary

Re: Method to check for invalid URLs

gary1 — Thu, 27 Feb 2025 18:34:46 GMT

I'm a big proponent of using basic Workato actions for basic operations. This is completely possible using native low-code actions, which are arguably easier to understand, implement, and maintain -- no ChatGPT required. As someone who is using a low-code automation platform and doesn't know python, I'm kind of flabbergasted by your approach, but to each his own I guess.

Re: Method to check for invalid URLs

bedelman0731 — Thu, 27 Feb 2025 20:25:37 GMT

I get that completely - but I did not really see a way in which this could be done using Workato actions

Re: Method to check for invalid URLs

bedelman0731 — Fri, 28 Feb 2025 14:44:43 GMT

I agree on using Workato actions whenever possible, but I didn't really find a good way to check a URL by doing so. What did you have in mind as a way to do this using basic Workato actions?

Re: Method to check for invalid URLs

gary1 — Fri, 28 Feb 2025 16:58:22 GMT

It's a two step process.

Use the HTTP action to call the URL
Log the status code from the response

Here's a closer look.

Two actions:

Set up the HTTP action like the below. A GET request to a URL (you'll use a data pill since you'll be in a loop). Set the response type to "Text" so it doesn't try to parse the response (if it were set to JSON, it would crash).

Very important, set the below to "Yes". Usually when you receive a 404 the recipe will throw an error, but this will prevent the error and your recipe will continue.

In this step, simply log the status code data pill from the output of the HTTP action. This could be in a list variable or really anywhere/anything else.

In the HTTP action output, a 200/success will look like this:

And a 404 will look like this: