Apache Iceberg Time Travel

Introduction to Apache Iceberg Time Travel

In traditional data engineering, modifying data in a data lake is a destructive process. When you overwrite a Parquet file or drop a Hive partition, the historical state of that data is permanently lost unless you have manually engineered complex, costly backup pipelines. If an erroneous ETL job corrupts a production table, restoring the data involves digging through object storage snapshots and enduring hours of downtime.

Apache Iceberg completely revolutionizes this paradigm through its immutable metadata architecture, enabling a powerful feature known as Time Travel.

Time travel allows data engineers, data scientists, and analysts to query the historical state of a table exactly as it existed at a specific point in time or at a specific snapshot ID. Because Iceberg treats data modification as a series of atomic, immutable commits, moving backward in time is not a costly data recovery operation—it is simply a matter of reading a different metadata file.

In this guide, we will explore the mechanics of Iceberg snapshots, how Time Travel is executed across various compute engines, and the profound architectural benefits it brings to the modern data lakehouse.

The Architecture of Time Travel: Snapshots

To understand Time Travel, we must first understand the concept of a Snapshot.

In Apache Iceberg, data files (Parquet, ORC, Avro) are immutable. Once written to object storage, they are never modified. When a user executes a DELETE, UPDATE, or INSERT statement, Iceberg does not alter the existing files. Instead, it writes new data files (and/or delete files) and generates a new Metadata Tree.

This new metadata tree culminates in a Snapshot. A snapshot is a comprehensive map of the table representing its complete, consistent state at the exact millisecond the transaction was committed.

The top-level table metadata file (v1.metadata.json, v2.metadata.json, etc.) maintains an array of all historical snapshots:

• snapshot_id: A unique 64-bit integer identifying the commit.
• timestamp_ms: The exact time the snapshot was created.
• manifest_list: A pointer to the Avro file that tracks all data files valid for this snapshot.

Because the old data files are never deleted by the transaction itself (they are simply no longer referenced by the current snapshot), the historical data remains perfectly intact in object storage. Time Travel simply tells the query engine: “Instead of reading the current snapshot, read the snapshot associated with this specific timestamp or ID.”

Executing Time Travel Queries

Apache Iceberg’s time travel capabilities are exposed seamlessly through standard SQL extensions across modern compute engines like Dremio, Apache Spark, and Trino.

Time Travel by Timestamp

The most common use case is querying a table as it existed at a specific date and time. This is invaluable for debugging (“What did this table look like right before the pipeline failed at 2:00 AM?”) or regulatory compliance.

In Apache Spark:

SELECT * FROM my_catalog.db.orders 
TIMESTAMP AS OF '2026-05-14 02:00:00';

Alternatively, using the Spark DataFrame API:

df = spark.read \
    .option("as-of-timestamp", "1715666400000") \
    .format("iceberg") \
    .load("my_catalog.db.orders")

In Dremio / Trino:

SELECT * FROM my_catalog.db.orders 
FOR SYSTEM_TIME AS OF TIMESTAMP '2026-05-14 02:00:00';

When this query is submitted, the engine parses the metadata.json file, scans the snapshots array, and selects the snapshot whose timestamp_ms is closest to, but not exceeding, the requested timestamp. It then reads the manifest list for that snapshot, ignoring any data files added or deleted in subsequent commits.

Time Travel by Snapshot ID

Sometimes, you need absolute precision. If you are auditing a specific ETL run, you can query exactly based on the Snapshot ID. You can easily find historical snapshot IDs by querying the Iceberg system tables:

-- View the history of the table
SELECT * FROM my_catalog.db.orders.history;

Once you have the ID (e.g., 1098234791234), you can query it directly:

In Apache Spark:

SELECT * FROM my_catalog.db.orders 
VERSION AS OF 1098234791234;

df = spark.read \
    .option("snapshot-id", 1098234791234) \
    .format("iceberg") \
    .load("my_catalog.db.orders")

In Dremio / Trino:

SELECT * FROM my_catalog.db.orders 
FOR SYSTEM_VERSION AS OF 1098234791234;

Real-World Use Cases for Time Travel

The ability to instantly query historical states unlocks several powerful capabilities for data teams:

1. Instant Rollbacks and Disaster Recovery

If bad data is written to a production table, data engineers no longer need to execute complex restoration scripts. Because the previous state still exists, you can simply roll back the table pointer to the last known good snapshot.

CALL catalog.system.rollback_to_snapshot('db.orders', 1098234791234);

This is a metadata-only operation. It takes milliseconds, instantly restoring the table to its correct state without moving any actual data.

2. Machine Learning Reproducibility

Machine Learning models rely heavily on training data. If a data scientist trains a model on May 14th, and attempts to retrain it a month later, the underlying data may have changed, making it impossible to reproduce the original results. By saving the Snapshot ID used during the initial training run, data scientists can use Time Travel to guarantee they are training against the exact same dataset, pixel-for-pixel, regardless of how much the table has mutated since.

3. Auditing and Compliance

Financial and healthcare institutions often face strict regulatory requirements (e.g., GDPR, CCPA, SOX) requiring them to prove exactly what data was used to make a decision at a specific point in the past. Time travel allows auditors to query historical states natively, without requiring separate archive tables.

Managing Storage Costs: Expiring Snapshots

While keeping every historical snapshot is incredibly powerful, it introduces a trade-off: Storage Costs.

Because Iceberg does not delete old data files automatically, highly transactional tables (e.g., tables receiving streaming inserts every minute) will rapidly accumulate millions of “dead” files that are only referenced by historical snapshots.

To prevent object storage bills from skyrocketing, data teams must implement Snapshot Expiration.

Snapshot expiration is a maintenance procedure that tells Iceberg: “Delete any snapshots older than X days, and physically delete the data files that are no longer referenced by the surviving snapshots.”

CALL catalog.system.expire_snapshots(
  table => 'db.orders', 
  older_than => TIMESTAMP '2026-05-01 00:00:00'
);

Most organizations configure automatic snapshot expiration jobs to run daily, keeping a sliding window of Time Travel history (e.g., 7 or 30 days) while pruning older, unneeded data files to optimize cloud storage costs.

Conclusion

Apache Iceberg’s Time Travel is a fundamental shift in data engineering. By decoupling the logical state of a table from the physical files in storage, Iceberg provides safety, auditability, and reproducibility without the operational overhead of traditional data backups. Whether you are recovering from a bad pipeline run, ensuring ML model consistency, or performing temporal analytics, Time Travel is an indispensable tool in the modern lakehouse architect’s arsenal.