In this post we are going to cover ON_ERROR in Snowflake, which is the option that decides what COPY INTO does when the data in the file is wrong.
The default is that nothing loads. One bad row takes the whole statement down, and that includes the good files sitting in the same statement. Every other setting is a decision about how much bad data you will accept quietly.
We will start with the default and work through the settings one at a time, and along the way we will get the rejected rows back out verbatim. Every query below was run on a real Snowflake account, and every result you see is the output it gave back.
Episode 127 of the Snowflake SQL: Zero to Hero series on TechBrothersIT.
- Let's set up two files, one clean and one broken
- Let's look at the broken file line by line
- Let's run the simplest COPY INTO across both files
- Is ABORT_STATEMENT really the default?
- Now let's try ON_ERROR = CONTINUE
- So what actually survived?
- Can we get the rejected rows back?
- Now let's skip the whole file instead
- Let's put a number on how many errors we will take
- Can we ask what would go wrong without loading anything?
- What happens when we ask for a few rows instead?
- One last thing: both option values are checked by name
- And let's put everything back
Let's set up two files, one clean and one broken
We need a stage, a format, a table, and two files. One of the files is clean. The other has two problems in it, on purpose, and they are two different kinds of problem.
-- Sample setup for this post.
USE DATABASE DEMO_DB;
USE SCHEMA PUBLIC;
CREATE OR REPLACE STAGE STAGE_T127;
CREATE OR REPLACE FILE FORMAT CSV_T127
TYPE = CSV
SKIP_HEADER = 1
FIELD_OPTIONALLY_ENCLOSED_BY = '"';
CREATE OR REPLACE TABLE TARGET_T127 (
SALE_ID INT,
CITY VARCHAR(20),
AMOUNT NUMBER(10,2)
);
-- One clean file and one with two problems in it, both with a header row.
-- Point these at a folder of your own.
PUT 'file://C:/your/folder/t127_good.csv' @STAGE_T127/in/ AUTO_COMPRESS = FALSE;
PUT 'file://C:/your/folder/t127_bad.csv' @STAGE_T127/in/ AUTO_COMPRESS = FALSE;
-- A second format whose field delimiter never occurs in the file,
-- so we can read the raw lines instead of a parsed grid.
CREATE OR REPLACE FILE FORMAT RAW_T127
TYPE = CSV
FIELD_DELIMITER = '~';Let's check both files are on the stage.
LIST @STAGE_T127/in/;| name | size | md5 | last_modified |
|---|---|---|---|
| stage_t127/in/t127_bad.csv | 112 | df5e99ab8c7c6d82635674365ed90fd0 | Thu, 1 Oct 2026 14:22:00 GMT |
| stage_t127/in/t127_good.csv | 80 | b2482de277162b89324a3a08e965652f | Thu, 1 Oct 2026 14:21:59 GMT |
Both there. The broken one is the bigger of the two, because it has an extra field on one of its lines.
Let's look at the broken file line by line
To see what is wrong with a file you have to see the raw lines, not a parsed grid. A parsed grid has already made the decision we are arguing about.
So we read it through RAW_T127, the format whose field delimiter is a tilde. A tilde never occurs in this file, so every line comes back whole in $1.
SELECT METADATA$FILE_ROW_NUMBER AS LINE_NUMBER, $1 AS THE_WHOLE_LINE
FROM @STAGE_T127/in/ (FILE_FORMAT => 'RAW_T127', PATTERN => '.*t127_bad.*')
ORDER BY LINE_NUMBER;| LINE_NUMBER | THE_WHOLE_LINE |
|---|---|
| 1 | sale_id,city,amount |
| 2 | 4,London,310.00 |
| 3 | 5,Austin,not_a_number |
| 4 | 6,Dublin,55.00,extra_column |
| 5 | 7,Oslo,88.25 |
Now we can see it. The header line, then London, then 5,Austin,not_a_number, where the amount is a word. Then 6,Dublin,55.00,extra_column, which has a fourth field. Then Oslo.
Two different kinds of wrong. One is a value that will not convert, and one is a line with too many fields. Remember that, because Snowflake tells them apart for us later.
Let's run the simplest COPY INTO across both files
No ON_ERROR anywhere. Just a load, over the whole prefix, so both files are in the one statement.
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127');100016 (22000): Field delimiter ',' found while expecting record delimiter '\n' File 'in/t127_bad.csv', line 4, character 15 Row 3, column "TARGET_T127"["AMOUNT":3] If you would like to continue loading when an error is encountered, use other values such as 'SKIP_FILE' or 'CONTINUE' for the ON_ERROR option. For more information on loading options, please run 'info loading_data' in a SQL client.
Field delimiter ',' found while expecting record delimiter, at line four of the bad file. That is the line with the extra field on it.
Now the part that matters. Let's count the table.
SELECT COUNT(*) AS ROWS_IN_THE_TABLE FROM TARGET_T127;| ROWS_IN_THE_TABLE |
|---|
| 0 |
Zero. Not zero from the bad file. Zero altogether. The clean file was in that statement too and none of it went in.
So the default is ABORT_STATEMENT, and it means the statement, not the file.
Is ABORT_STATEMENT really the default?
Rather than tell you it is, let's spell it out and compare what comes back.
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127')
ON_ERROR = ABORT_STATEMENT;100016 (22000): Field delimiter ',' found while expecting record delimiter '\n' File 'in/t127_bad.csv', line 4, character 15 Row 3, column "TARGET_T127"["AMOUNT":3] If you would like to continue loading when an error is encountered, use other values such as 'SKIP_FILE' or 'CONTINUE' for the ON_ERROR option. For more information on loading options, please run 'info loading_data' in a SQL client.
The identical message, line and character included. That is the proof.
Now let's try ON_ERROR = CONTINUE
CONTINUE says: skip the rows you cannot read and load the rest.
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127')
ON_ERROR = CONTINUE
FORCE = TRUE;| file | status | rows_parsed | rows_loaded | error_limit | errors_seen | first_error | first_error_line | first_error_character | first_error_column_name |
|---|---|---|---|---|---|---|---|---|---|
| stage_t127/in/t127_good.csv | LOADED | 3 | 3 | 3 | 0 | NULL | NULL | NULL | NULL |
| stage_t127/in/t127_bad.csv | PARTIALLY_LOADED | 4 | 2 | 4 | 2 | Numeric value 'not_a_number' is not recognized | 3 | 10 | "TARGET_T127"["AMOUNT":3] |
And it succeeded. Two rows in the report, and one of them has a status we have not seen before.
The clean file is LOADED. Three parsed, three loaded, no errors. The bad one is PARTIALLY_LOADED. Four parsed, two loaded, two errors seen, and the first of those errors is spelled out in the grid.
That grid is wide, so let's pull the columns that matter out of it with a RESULT_SCAN.
SELECT "file" AS THE_FILE,
"status" AS THE_STATUS,
"rows_parsed" AS ROWS_PARSED,
"rows_loaded" AS ROWS_LOADED,
"errors_seen" AS ERRORS_SEEN,
"first_error" AS FIRST_ERROR
FROM TABLE(RESULT_SCAN(LAST_QUERY_ID()))
ORDER BY THE_FILE;| THE_FILE | THE_STATUS | ROWS_PARSED | ROWS_LOADED | ERRORS_SEEN | FIRST_ERROR |
|---|---|---|---|---|---|
| stage_t127/in/t127_bad.csv | PARTIALLY_LOADED | 4 | 2 | 2 | Numeric value 'not_a_number' is not recognized |
| stage_t127/in/t127_good.csv | LOADED | 3 | 3 | 0 | NULL |
File, status, rows parsed, rows loaded, errors seen and the first error. Those six are the ones to read on any load.
Have a look at error_limit in the wide grid above. We did not set a limit anywhere in that statement, and the two files came back with different values for it. Each one is that file's own rows_parsed. error_limit only reports the number you asked for when you actually asked for one, which we do further down.
So what actually survived?
This is the question the report does not answer. Let's look at the table.
SELECT SALE_ID AS SALE_ID, CITY AS CITY, AMOUNT AS AMOUNT
FROM TARGET_T127
ORDER BY SALE_ID;| SALE_ID | CITY | AMOUNT |
|---|---|---|
| 1 | Toronto | 100.00 |
| 2 | Boston | 250.50 |
| 3 | Zurich | 75.25 |
| 4 | London | 310.00 |
| 7 | Oslo | 88.25 |
Five rows. The three from the clean file, and then London and Oslo out of the bad one. Austin and Dublin are not there.
SELECT COUNT(*) AS ROWS_IN_THE_TABLE,
SUM(AMOUNT) AS TOTAL_AMOUNT
FROM TARGET_T127;| ROWS_IN_THE_TABLE | TOTAL_AMOUNT |
|---|---|
| 5 | 824.00 |
Five rows and 824.00.
Nothing warned us. The statement reported success, the row count looks perfectly plausible, and two rows of real data are gone. If this load ran on a schedule and you only looked at whether it succeeded, you would never know.
Which is why the next grid is the one that matters.
Can we get the rejected rows back?
Yes. The rows that did not make it are not lost. VALIDATE hands them back, and it takes a job id, where '_last' means the most recent load into that table.
SELECT * FROM TABLE(VALIDATE(TARGET_T127, JOB_ID => '_last'));| ERROR | FILE | LINE | CHARACTER | BYTE_OFFSET | CATEGORY | CODE | SQL_STATE | COLUMN_NAME | ROW_NUMBER | ROW_START_LINE | REJECTED_RECORD |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Field delimiter ',' found while expecting record delimiter '\n' | in/t127_bad.csv | 4 | 15 | 75 | parsing | 100016 | 22000 | "TARGET_T127"["AMOUNT":3] | 3 | 4 | 6,Dublin,55.00,extra_column |
| Numeric value 'not_a_number' is not recognized | in/t127_bad.csv | 3 | 10 | 47 | conversion | 100038 | 22018 | "TARGET_T127"["AMOUNT":3] | 2 | 3 | 5,Austin,not_a_number |
Twelve columns. Two rows, one for each bad line.
REJECTED_RECORD is the one you want. It holds the raw line from the file, exactly as it was. 6,Dublin,55.00,extra_column and 5,Austin,not_a_number, whole. So you can fix them, load them somewhere else, or just look at them.
And CATEGORY tells the two kinds of wrong apart. parsing for the line with the extra field, conversion for the word where a number should be. With a different CODE for each: 100016 and 100038.
Now let's skip the whole file instead
CONTINUE made the decision row by row. SKIP_FILE makes it file by file. Let's start from an empty table.
TRUNCATE TABLE TARGET_T127;| status |
|---|
| Statement executed successfully. |
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127')
ON_ERROR = SKIP_FILE
FORCE = TRUE;| file | status | rows_parsed | rows_loaded | error_limit | errors_seen | first_error | first_error_line | first_error_character | first_error_column_name |
|---|---|---|---|---|---|---|---|---|---|
| stage_t127/in/t127_good.csv | LOADED | 3 | 3 | 1 | 0 | NULL | NULL | NULL | NULL |
| stage_t127/in/t127_bad.csv | LOAD_FAILED | 4 | 0 | 1 | 2 | Numeric value 'not_a_number' is not recognized | 3 | 10 | "TARGET_T127"["AMOUNT":3] |
A third status. The clean file is LOADED. The bad one is LOAD_FAILED, with four parsed, nothing loaded and two errors.
SELECT SALE_ID AS SALE_ID, CITY AS CITY, AMOUNT AS AMOUNT
FROM TARGET_T127
ORDER BY SALE_ID;| SALE_ID | CITY | AMOUNT |
|---|---|---|
| 1 | Toronto | 100.00 |
| 2 | Boston | 250.50 |
| 3 | Zurich | 75.25 |
Three rows, all from the clean file. So SKIP_FILE does what it says, and the good file still gets through. That is the difference from the default.
Let's put a number on how many errors we will take
SKIP_FILE_n says: fail the file once it has this many errors in it. The bad file has two. Let's allow three.
TRUNCATE TABLE TARGET_T127;| status |
|---|
| Statement executed successfully. |
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127')
ON_ERROR = SKIP_FILE_3
FORCE = TRUE;| file | status | rows_parsed | rows_loaded | error_limit | errors_seen | first_error | first_error_line | first_error_character | first_error_column_name |
|---|---|---|---|---|---|---|---|---|---|
| stage_t127/in/t127_good.csv | LOADED | 3 | 3 | 3 | 0 | NULL | NULL | NULL | NULL |
| stage_t127/in/t127_bad.csv | PARTIALLY_LOADED | 4 | 2 | 3 | 2 | Numeric value 'not_a_number' is not recognized | 3 | 10 | "TARGET_T127"["AMOUNT":3] |
PARTIALLY_LOADED, and look at error_limit now. It is the number we asked for.
SELECT COUNT(*) AS ROWS_IN_THE_TABLE FROM TARGET_T127;| ROWS_IN_THE_TABLE |
|---|
| 5 |
Five rows, the same as CONTINUE gave us.
Now a limit the file cannot meet
TRUNCATE TABLE TARGET_T127;| status |
|---|
| Statement executed successfully. |
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127')
ON_ERROR = SKIP_FILE_1
FORCE = TRUE;| file | status | rows_parsed | rows_loaded | error_limit | errors_seen | first_error | first_error_line | first_error_character | first_error_column_name |
|---|---|---|---|---|---|---|---|---|---|
| stage_t127/in/t127_good.csv | LOADED | 3 | 3 | 1 | 0 | NULL | NULL | NULL | NULL |
| stage_t127/in/t127_bad.csv | LOAD_FAILED | 4 | 0 | 1 | 2 | Numeric value 'not_a_number' is not recognized | 3 | 10 | "TARGET_T127"["AMOUNT":3] |
LOAD_FAILED for the bad file, with the limit reported as one and errors seen as two.
SELECT COUNT(*) AS ROWS_IN_THE_TABLE FROM TARGET_T127;| ROWS_IN_THE_TABLE |
|---|
| 3 |
Three rows. Only the clean file again.
And the percentage version
SKIP_FILE_n% works out the limit from the rows in the file. Our bad file parses four rows, so half of it is two, and the file has two errors. Right on the line.
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127')
ON_ERROR = 'SKIP_FILE_50%'
FORCE = TRUE;| file | status | rows_parsed | rows_loaded | error_limit | errors_seen | first_error | first_error_line | first_error_character | first_error_column_name |
|---|---|---|---|---|---|---|---|---|---|
| stage_t127/in/t127_good.csv | LOADED | 3 | 3 | 1 | 0 | NULL | NULL | NULL | NULL |
| stage_t127/in/t127_bad.csv | LOAD_FAILED | 4 | 0 | 2 | 2 | Numeric value 'not_a_number' is not recognized | 3 | 10 | "TARGET_T127"["AMOUNT":3] |
error_limit two, errors_seen two, and LOAD_FAILED.
So the file fails when errors_seen reaches error_limit, not when it goes past it. If you want to tolerate two bad rows, you have to allow three.
Can we ask what would go wrong without loading anything?
VALIDATION_MODE is the option for that. RETURN_ERRORS says: tell me what is wrong and load nothing.
TRUNCATE TABLE TARGET_T127;| status |
|---|
| Statement executed successfully. |
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127')
VALIDATION_MODE = RETURN_ERRORS;| ERROR | FILE | LINE | CHARACTER | BYTE_OFFSET | CATEGORY | CODE | SQL_STATE | COLUMN_NAME | ROW_NUMBER | ROW_START_LINE | REJECTED_RECORD |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Field delimiter ',' found while expecting record delimiter '\n' | in/t127_bad.csv | 4 | 15 | 75 | parsing | 100016 | 22000 | "TARGET_T127"["AMOUNT":3] | 3 | 4 | 6,Dublin,55.00,extra_column |
| Numeric value 'not_a_number' is not recognized | in/t127_bad.csv | 3 | 10 | 47 | conversion | 100038 | 22018 | "TARGET_T127"["AMOUNT":3] | 2 | 3 | 5,Austin,not_a_number |
The same twelve column grid VALIDATE gave us, with the same two rows, the same categories and the same rejected records.
But this time nothing was loaded, and that is the claim worth checking rather than believing.
SELECT COUNT(*) AS ROWS_IN_THE_TABLE FROM TARGET_T127;| ROWS_IN_THE_TABLE |
|---|
| 0 |
Zero rows. So it really is a dry run.
RETURN_ALL_ERRORS is the same idea without a cap on how many it reports.
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127')
VALIDATION_MODE = RETURN_ALL_ERRORS;| ERROR | FILE | LINE | CHARACTER | BYTE_OFFSET | CATEGORY | CODE | SQL_STATE | COLUMN_NAME | ROW_NUMBER | ROW_START_LINE | REJECTED_RECORD |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Field delimiter ',' found while expecting record delimiter '\n' | in/t127_bad.csv | 4 | 15 | 75 | parsing | 100016 | 22000 | "TARGET_T127"["AMOUNT":3] | 3 | 4 | 6,Dublin,55.00,extra_column |
| Numeric value 'not_a_number' is not recognized | in/t127_bad.csv | 3 | 10 | 47 | conversion | 100038 | 22018 | "TARGET_T127"["AMOUNT":3] | 2 | 3 | 5,Austin,not_a_number |
The same two rows, because two is all there were in this file.
What happens when we ask for a few rows instead?
There is a third form, RETURN_n_ROWS, and I assumed it would show me the rows it was about to load. Let's ask for five of them.
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127')
VALIDATION_MODE = RETURN_5_ROWS;100016 (22000): Field delimiter ',' found while expecting record delimiter '\n' File 'in/t127_bad.csv', line 4, character 15 Row 3, column "TARGET_T127"["AMOUNT":3]
It did not show me anything. 100016, the same parsing error the default load gave us, and it aborted.
So RETURN_n_ROWS is not a safe dry run. It reads the file the way a real load does and it hits the bad line like any other load would. I got that one wrong, and it is on the page as it came back.
And VALIDATION_MODE on a transforming COPY is not supported at all.
COPY INTO TARGET_T127 (SALE_ID, CITY, AMOUNT)
FROM (
SELECT $1::INT, UPPER($2::VARCHAR), $3::NUMBER(10,2)
FROM @STAGE_T127/in/ (FILE_FORMAT => 'CSV_T127')
)
VALIDATION_MODE = RETURN_ERRORS;002302 (0A000): SQL Compilation error: Expression 'SALE_ID' not supported within a VALIDATE expression.
Expression 'SALE_ID' not supported within a VALIDATE expression. So the dry run and the transforming load are two things you cannot have at once.
One last thing: both option values are checked by name
A misspelled option value could easily be ignored in silence, which would be the worst possible behaviour here. Let's check both of them.
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127')
ON_ERROR = KEEP_CALM_AND_CARRY_ON
FORCE = TRUE;001008 (22023): SQL compilation error: invalid value [KEEP_CALM_AND_CARRY_ON] for parameter 'ON_ERROR'
COPY INTO TARGET_T127
FROM @STAGE_T127/in/
FILE_FORMAT = (FORMAT_NAME = 'CSV_T127')
VALIDATION_MODE = RETURN_EVERYTHING;001008 (22023): SQL compilation error: invalid value [RETURN_EVERYTHING] for parameter 'VALIDATION_MODE'
Both refused by name, both 001008. Good.
And VALIDATE on a table that never had a load into it.
SELECT * FROM TABLE(VALIDATE(CUSTOMERS, JOB_ID => '_last'));002018 (22023): SQL compilation error: Invalid argument [The specified Job UUID does not load into the specified table] for table function. Table function argument is required to be a constant.
The specified Job UUID does not load into the specified table. So '_last' is the last load into that table, and there has not been one.
And let's put everything back
Everything here was made by the script at the top, so everything in it can go.
-- Put the schema back the way we found it.
TRUNCATE TABLE TARGET_T127;
REMOVE @STAGE_T127/;
DROP STAGE IF EXISTS STAGE_T127;
DROP FILE FORMAT IF EXISTS RAW_T127;
DROP FILE FORMAT IF EXISTS CSV_T127;
DROP TABLE IF EXISTS TARGET_T127;LIST @STAGE_T127/;| name | size | md5 | last_modified |
|---|
SELECT CURRENT_DATABASE() AS THE_DATABASE,
CURRENT_SCHEMA() AS THE_SCHEMA,
(SELECT COUNT(*) FROM INFORMATION_SCHEMA.TABLES
WHERE RIGHT(TABLE_NAME, 5) = '_T127') AS PROBE_TABLES_LEFT,
(SELECT COUNT(*) FROM INFORMATION_SCHEMA.STAGES
WHERE STAGE_NAME LIKE '%_T127%') AS PROBE_STAGES_LEFT,
(SELECT COUNT(*) FROM INFORMATION_SCHEMA.FILE_FORMATS
WHERE RIGHT(FILE_FORMAT_NAME, 5) = '_T127') AS PROBE_FORMATS_LEFT,
(SELECT COUNT(*) FROM CUSTOMERS) AS DEMO_DB_IS_READABLE;| THE_DATABASE | THE_SCHEMA | PROBE_TABLES_LEFT | PROBE_STAGES_LEFT | PROBE_FORMATS_LEFT | DEMO_DB_IS_READABLE |
|---|---|---|---|---|---|
| DEMO_DB | PUBLIC | 0 | 0 | 0 | 10 |
No probe tables, no probe stages, no probe formats, and the database still reads.
Key points
- The default is
ON_ERROR = ABORT_STATEMENT, and it means the statement. One bad row in one file left the clean file in the same statement with zero rows loaded. - Read a file's raw lines by giving the format a
FIELD_DELIMITERthat never occurs in it. - A value that will not convert is a
conversionerror. A line with too many fields is aparsingerror. ON_ERROR = CONTINUEgives a third status,PARTIALLY_LOADED, and the statement reports success.- On this load that meant five rows and
824.00in the table, with two rows of real data silently missing. VALIDATE(table, JOB_ID => '_last')returns twelve columns, includingREJECTED_RECORD, which is the raw line exactly as it was in the file.CATEGORYandCODEtell the two kinds of bad row apart:100016and100038.SKIP_FILEfails the file and lets the clean one through.SKIP_FILE_nandSKIP_FILE_n%seterror_limit, and the file fails whenerrors_seenreaches it, not when it passes it.VALIDATION_MODE = RETURN_ERRORSreally does load nothing. We checked the table afterwards.RETURN_n_ROWSis not a safe dry run. It hit the bad line and aborted with100016.VALIDATION_MODEis not supported on a transformingCOPY, and both option values are checked by name.
Episode 128 — COPY INTO a location. Unloading, which is the same statement pointed the other way. We will look at the prefix it writes into, the file names it chooses for you, and the compression nobody asked for.
Every query and every result grid on this page was captured from a live Snowflake account (version 10.35.101) running the demo script for this episode. The sales data is made up for teaching, and the two broken lines in t127_bad.csv are broken deliberately. The two PUT statements take a path on your own machine, so the folder is written as C:/your/folder and their captured grids are not shown. Cells reading NULL in the COPY report are real nulls. Nothing on this page is timed and no performance claims are made anywhere in this series. The option behaviour follows the Snowflake COPY INTO <table> documentation.



No comments:
Post a Comment
Note: Only a member of this blog may post a comment.