Skip to content
DP-700 Fabric Engineer Hub

Work with Delta Lake Tables in Microsoft Fabric

Work with Delta Lake Tables in Microsoft Fabric

Source

Summary

This module focuses on the inner mechanics of Delta Lake storage in Microsoft Fabric: transaction logging (_delta_log/), ACID compliance, time travel, schema enforcement/evolution, table optimization commands (OPTIMIZE, V-ORDER, VACUUM), and Liquid Clustering.

Core Architectural Mechanisms

1. Delta Transaction Log (_delta_log/)

  • Every transaction creates an atomic JSON commit log file (000000.json, 000001.json).
  • Every 10 commits, a checkpoint .checkpoint.parquet file is compiled to speed up log replay.
  • Enables single-source-of-truth concurrency, preventing dirty reads and conflicting writes.

2. Time Travel & Version History

-- View audit history of changes
DESCRIBE HISTORY fact_sales;


-- Query specific historical version
SELECT * FROM fact_sales VERSION AS OF 3;


-- Query historical timestamp
SELECT * FROM fact_sales TIMESTAMP AS OF '2026-08-01 10:00:00';


-- Restore to an earlier version
RESTORE TABLE fact_sales TO VERSION AS OF 2;

3. Maintenance & Performance Tuning Operations

OperationCommand SyntaxPurpose
Compaction (Bin-packing)OPTIMIZE fact_sales;Merges small parquet files into optimal ~1GB file sizes.
V-Order ApplicationOPTIMIZE fact_sales VORDER;Applies columnar sorting optimized for VertiPaq & Direct Lake.
Z-OrderingOPTIMIZE fact_sales ZORDER BY (CustomerID);Collocates related data across multidimensional space.
Garbage CollectionVACUUM fact_sales RETAIN 168 HOURS;Deletes historical files older than retention period (default 7 days).
Liquid ClusteringALTER TABLE fact_sales CLUSTER BY (Region, Date);Replaces static partitioning and Z-Order with flexible, auto-tuning clustering.

Exam Traps & Gotchas

[!CAUTION]

  • VACUUM vs Time Travel: Running VACUUM permanently removes physical parquet files beyond the retention threshold. Once vacuumed, time travel queries attempting to read those versions will fail with FileNotFoundException.
  • Zero-Duration Vacuum Danger: Overriding retention checks (SET spark.databricks.delta.vacuum.parallelDelete.enabled = true or spark.microsoft.delta.vacuum.relinquishCheck = true) can corrupt concurrent transactions if set below the retention safety limit.