Encoding Error in CSV Parsing

VaradS — Tue, 03 Sep 2024 04:11:53 GMT

I am parsing a CSV to send out email reminders.

I have a scheduled job which emails me the CSV every day via Metabase. I set the encoding to UTF-8, but got error: Error parsing CSV file: Invalid byte sequence in UTF-8 in line 1

If I change the encoding to US-ASCII, it is giving error:"\x89" on US-ASCII

If I download the CSV on my Mac and manually run the recipe, it works fine. Has anyone faced this issue before?

Re: Encoding Error in CSV Parsing

timwolfe — Sun, 17 Nov 2024 03:39:58 GMT

Your CSV likely contains non-UTF-8 characters or a Byte Order Mark (BOM). To fix:

1) Check the file's actual encoding (e.g., file -I yourfile.csv on Mac).
2) Clean it using a script like this in Python:

import codecs
with codecs.open('input.csv', 'r', 'utf-8-sig') as source_file:
with codecs.open('output.csv', 'w', 'utf-8') as target_file:
target_file.write(source_file.read())

3) Ensure Metabase outputs clean UTF-8 without BOM.
4) Compare the "manual" and "automated" CSV versions for differences.

topic Re: Encoding Error in CSV Parsing in Community Chat

Encoding Error in CSV Parsing

Re: Encoding Error in CSV Parsing