Parquet File Viewer: Validate and Inspect Parquet Files
Parquet files are binary — you can't open them in a text editor and see the schema. This tool validates the Parquet magic bytes (PAR1) to confirm your file is well-formed, then generates Python + pyarrow code to inspect the schema, row groups, column statistics, and data. Useful for debugging data pipeline outputs and verifying ETL results.
The PAR1 magic bytes appear at byte 0 and the last 4 bytes. The footer is a Thrift-encoded FileMetaData struct containing schema, row group locations, and column statistics.
Understanding the Parquet File Format
A Parquet file is organized as: (1) 4-byte magic header PAR1, (2) one or more row groups each containing column chunks, (3) a Thrift-encoded footer with the full schema and row group metadata, (4) 4 bytes encoding the footer length, (5) 4-byte magic footer PAR1. Query engines read the footer first (from the end of the file) to get the schema and row group locations, then seek to only the column chunks they need.
What Row Group Metadata Contains
Each row group's metadata includes: row count, total byte size, and per-column statistics including min value, max value, null count, and distinct count. Query engines use these statistics for predicate pushdown: if you filter WHERE year = 2024 and a row group's year column has max = 2023, the engine skips that row group entirely without reading it. This is why sorting your data before writing to Parquet can dramatically speed up filtered reads.
Common Parquet Issues
File not opening in Spark/DuckDB: verify magic bytes with this tool. Wrong data types: inspect schema with pyarrow and regenerate with explicit dtype mapping. Poor compression: check what codec was used (metadata shows it); switch to Zstd. Slow queries on filtered data: if min/max statistics are showing wide ranges per row group, your data is not sorted — sort by the most common filter column before writing. Schema mismatch across files: use pyarrow.dataset to read multiple files and detect schema conflicts.
What This Tool Checks
- PAR1 magic byte at file start
- PAR1 magic byte at file end
- File size (reported)
- Generated Python code: data view (head, dtypes, nulls)
- Generated Python code: metadata (row groups, schema, compression)
Frequently Asked Questions
Why can't I see the Parquet contents directly in the browser?
Parquet files are binary columnar files with Thrift-encoded metadata and dictionary-encoded, run-length-compressed column chunks. Decoding them requires a full Parquet library (pyarrow, fastparquet). No stable WebAssembly decoder exists yet. This tool validates the file's magic bytes and generates Python code to inspect it locally with pyarrow.
What does the magic byte check tell me?
Every valid Parquet file starts and ends with the 4-byte magic number PAR1 (bytes 0x50 0x41 0x52 0x31). If either magic byte sequence is wrong, the file is corrupted, truncated, or not a Parquet file. A passing magic byte check confirms the file was written by a valid Parquet encoder.
What is a Parquet row group?
A Parquet file is divided into horizontal slices called row groups, each containing a configurable number of rows (typically 128MB of data per row group). Each row group contains one column chunk per column. Row group statistics (min/max per column) are stored in the file metadata, allowing query engines to skip row groups that don't contain matching data.
How do I inspect a Parquet file's schema without loading the data?
Use pyarrow: import pyarrow.parquet as pq; pf = pq.ParquetFile('file.parquet'); print(pf.schema_arrow). This reads only the footer metadata (a few KB at most) without loading any row data. The schema shows all column names, types, and nested structures.
What compression does Parquet use?
Parquet supports per-column compression: Snappy (fast, moderate compression), Gzip (slow, best compression), Zstd (fast + good compression, recommended for new files), LZO, LZ4, and BROTLI. Most files use Snappy (the default in pandas) or Zstd. The compression codec is stored in the column chunk metadata — pyarrow reports it in the row group metadata.