Method to check for invalid URLs

[bedelman0731]

Is there a method by which a URL can be checked within a recipe to see if the URL is valid?

I have a recipe that, among other things, is constructing a URL that should be a reference to an image based on one of the ID of a data element. The issue arises in that the image has not always been uploaded and the URL that should point to the image returns a 404 (Page Not Found).

The output of the recipe is a CSV file where each row contains what "should" be the URL to the image, but (in some cases) the image has not been uploaded and the URL that it would have then throws a 404.

I did find a App Script for Google Sheets that will check the URLs - but this requires importing the CSV into Google Sheets, adding in the App Script, adding a column that then uses the App Script function to return the status of the URL. All of these are additional steps that I would prefer to avoid and merely do the URL checking within the recipe itself.

Is there a way to implement such a URL checker into a recipe in Workato?

[Prudvi]

Hi @bedelman0731,
You can try using a execute python code action which will take URL as input and StatusCode as output. 
Python code will make a request to the URL and captures the StatusCode
Based on the value of StatusCode you filter the URL from being added to the CSV.
Regards,
Prudvi

[gary1]

There's no need for Python. You can do this using the HTTP action. Set the HTTP action to treat non-2xx status codes as success and log the status code after the call. (Edit: Alternatively you can wrap the HTTP action in an error handler to catch non-2xx codes.)

If you want save tasks, you could use Python or JS to loop through the URLs and return an array of results. If you're checking thousands of URLs this could save quite a few tasks. 

[bedelman0731]

I think I will use a python code action to do this. While I am not proficient in writing Python code from scratch, I used ChatGPT to help me out to come up with the following function that appears to work. The input field is "input_data" and the output field is "Response"

In this usage, I only care about results that do not yield a 200 response

def get_status_code(input_data):
    url = input_data.get("input_data", "").strip()  # Extract URL from input data
    
    if not url:
        return None  # Return None if no URL is provided

    try:
        response = requests.get(url, allow_redirects=False)  # Similar to 'followRedirects': false
        return response.status_code  # Only return the status code
    except requests.exceptions.RequestException:
        return None  # Return None if there's an error

# Main function that Workato calls
def main(input_data):
    return get_status_code(input_data)  # Only return the status code

[gary1]

I'm a big proponent of using basic Workato actions for basic operations. This is completely possible using native low-code actions, which are arguably easier to understand, implement, and maintain -- no ChatGPT required. As someone who is using a low-code automation platform and doesn't know python, I'm kind of flabbergasted by your approach, but to each his own I guess.