Parquet vs Avro vs ORC vs CSV: Choosing a Data File Format

Parquet vs Avro comes down to how you read the data: Parquet is a columnar format built for analytical queries that scan a few columns across many rows, while Avro is a row-based format built for writing and reading whole records, which makes it a natural fit for streaming and data exchange. ORC is a columnar format similar to Parquet with deep roots in the Hive ecosystem, and CSV is a plain-text row format that is universal but slow and loosely typed. The rest of this guide explains why, and how to pick one for each layer of a pipeline.

Row-based vs columnar storage: what is the difference?

Everything else follows from this one design choice. Imagine a table with columns order_id, customer_id, amount, and created_at.

A row-based format stores each record together:

Row layout (CSV, Avro)
[1001, C7, 25.00, 2026-09-01] [1002, C3, 14.50, 2026-09-01] [1003, C7, 99.90, 2026-09-02]

Columnar layout (Parquet, ORC)
order_id:    [1001, 1002, 1003]
customer_id: [C7, C3, C7]
amount:      [25.00, 14.50, 99.90]
created_at:  [2026-09-01, 2026-09-01, 2026-09-02]

Row layouts are efficient when you write or read an entire record at a time, such as appending events from a producer or processing messages one by one. Columnar layouts are efficient when a query touches only some columns. SELECT SUM(amount) FROM orders reads just the amount column and skips the rest, which on a wide table can mean reading a small fraction of the bytes.

Columnar storage also compresses better. Values in one column share a type and often repeat, so encodings like dictionary and run-length encoding shrink them dramatically before a general-purpose codec such as Snappy or Zstandard runs.

Parquet vs Avro vs ORC vs CSV at a glance

What is predicate pushdown and why does it matter?

Parquet and ORC split files into chunks (row groups in Parquet, stripes in ORC) and store statistics such as the minimum and maximum value of each column in each chunk. When a query filters WHERE created_at >= '2026-09-01', the reader checks those statistics and skips any chunk whose max created_at is earlier. This is predicate pushdown: the filter is applied at the storage layer so irrelevant data is never decompressed.

Pushdown works best when the data is sorted or clustered on the filter column, so each chunk covers a narrow range. If values are scattered randomly, every chunk's min and max span nearly the whole range and nothing gets skipped. That is why file layout and data partitioning for faster queries go hand in hand with file format choice.

CSV and Avro cannot do this. A reader has to parse every record to evaluate the filter.

How does schema evolution work in each format?

Schemas change: new fields appear, optional fields get added, types widen. How each format handles that is often the deciding factor.

• CSV has no schema. A new column in the middle of a file silently shifts every value to the wrong field for any reader that assumes positions. Type information is lost, so 00123 may become 123.
• Avro is designed around evolution. Every file carries its writer schema, and a reader supplies its own reader schema. Avro resolves the two by field name: fields missing from the data get their declared default, and fields the reader does not know are ignored. Pair it with a schema registry and compatibility rules, and producers and consumers can upgrade independently.
• Parquet and ORC store the schema in the file footer. Adding a nullable column is safe; older files simply return null for it. Renames, drops, and type changes depend on the query engine and whether columns are resolved by name or by position. Table formats such as Apache Iceberg and Delta Lake add a metadata layer that tracks columns by ID, which makes renames and drops safe on top of Parquet.

Here is a small Avro schema with a field added later with a default, which lets old records be read by new consumers:

{
  "type": "record",
  "name": "Order",
  "fields": [
    {"name": "order_id", "type": "string"},
    {"name": "amount_cents", "type": "long"},
    {"name": "coupon_code", "type": ["null", "string"], "default": null}
  ]
}

When to use Avro

Avro fits wherever data moves one record at a time or crosses team boundaries:

• Event streams and message queues, where producers serialize individual records. Avro is widely used with Kafka and a schema registry.
• Landing zones for raw ingestion, where you want a compact binary format that preserves the full record and the writer's schema.
• Service-to-service data exchange, where independent evolution of producer and consumer matters.

Avro is a poor choice for analytical queries over large tables, because every query reads every column.

When to use Parquet

Parquet is the default for analytical storage in most modern stacks. Spark, Trino, DuckDB, pandas, and the major cloud warehouses read it natively, and open table formats use it as their primary data file format. Use it for curated tables in a data lake or lakehouse, for feature datasets in machine learning, and for any large dataset queried by column.

It is less suited to single-record writes. Writers buffer rows and emit a file per batch, so writing tiny Parquet files continuously from a stream creates the small files problem. Buffer and write larger files, or compact them later.

When to use ORC

ORC offers the same core benefits as Parquet: columnar storage, lightweight indexes, strong compression, and predicate pushdown. It grew up alongside Hive and has features like optional bloom filters per column and built-in support for Hive ACID tables. If your platform is Hive-centric, ORC is a solid choice. Outside that ecosystem, Parquet usually has broader tool support, which is why it tends to win by default.

When is CSV still the right answer?

CSV remains useful for small exports, handing data to spreadsheets and non-technical users, and quick debugging. For anything that is stored long-term, queried repeatedly, or large, convert it at the ingestion boundary. Common CSV pitfalls include embedded commas and newlines, inconsistent quoting, ambiguous null representations, locale-specific number formats, and character encoding issues.

A practical layered approach

Many pipelines use more than one format, each where it is strongest:

1. Ingest events as Avro through a stream, or accept CSV and JSON from external partners.
2. Land raw data as-is for replay, often Avro or the original files.
3. Transform into Parquet, partitioned and sorted on common filters, typically managed by a table format. See ETL vs ELT for where this conversion usually sits.
4. Serve analytics from Parquet-backed tables, and export CSV only at the edges when a human asks for it.

Key takeaways

• The core difference in Parquet vs Avro is columnar vs row-based layout, which maps to analytics vs record-at-a-time workloads.
• Columnar formats (Parquet, ORC) enable column pruning, predicate pushdown, and far better compression.
• Avro has the strongest built-in schema evolution and is the common choice for streaming and messaging.
• ORC is comparable to Parquet technically; Parquet generally has wider ecosystem support.
• CSV is fine at the edges for humans, but should be converted at ingestion for anything stored or queried at scale.

Frequently asked questions

Is Parquet faster than Avro?

For analytical queries that read a subset of columns, Parquet is typically much faster because it skips unneeded columns and chunks. For writing or reading full records one at a time, Avro is usually the better fit. The right answer depends on the access pattern, not the format alone.

Can Parquet be used for streaming?

Yes, but not record by record. Streaming systems usually buffer events and write Parquet files in micro-batches. Writing very small files frequently hurts query performance, so plan for larger batches or periodic compaction.

Should I choose Parquet or ORC?

Both are mature columnar formats with similar capabilities. Choose ORC if you are heavily invested in Hive and its ACID tables; otherwise Parquet is the more common default because of broader support across engines, libraries, and table formats.

Does converting CSV to Parquet reduce storage?

Usually by a large margin, because Parquet stores typed binary values and applies column-level encoding and compression. The exact savings vary with the data, so measure on a representative sample of your own files.