Cheatsheet: Capacity Metrics, Monitoring & Troubleshooting
Cheatsheet: Capacity Metrics, Monitoring & Troubleshooting
Source
- Content type: Quick-Reference Cheatsheet
- Target Exam: Exam DP-700
- Date captured: 2026-08-15
- Last reviewed: 2026-08-15
1. Capacity Smoothing & Throttling Rules
┌────────────────────────────────────────────────────────┐│ Fabric Capacity Smoothing │├──────────────────────────┬─────────────────────────────┤│ Interactive Operations │ 5-minute moving average ││ Background Operations │ 24-hour moving average │└──────────────────────────┴─────────────────────────────┘
┌────────────────────────────────────────────────────────┐│ Throttling Escalation Order │├──────────────────────────┬─────────────────────────────┤│ 1. Interactive Delay │ > 10 min burndown debt ││ 2. Interactive Rejection │ > 60 min burndown debt ││ 3. Background Rejection │ > 24 hr persistent overload │└──────────────────────────┴─────────────────────────────┘2. Spark Job Diagnosis & Remediation
| Symptom | Probable Cause | Diagnostic Tool | Recommended Solution |
|---|---|---|---|
| 99 tasks finish in seconds, 1 task takes 15 mins | Data Skew | Spark UI Stage Timeline | Apply salt keys, or enable spark.sql.adaptive.skewJoin.enabled |
Spark job fails with OutOfMemoryError (OOM) | Memory Pressure / Disk Spill | Spark UI Executor Memory Graph | Increase node size, reduce shuffle partitions, avoid collecting massive DataFrames (df.collect()) |
| Massive small file buildup | Unmanaged frequent appends | OneLake file explorer | Run OPTIMIZE <table> VORDER; |
| Query fails after retention cleanup | VACUUM executed within time travel window | Error log (FileNotFoundException) | Increase VACUUM RETAIN <N> HOURS retention period |
3. Warehouse Query Tuning
- Check
queryinsights.exec_requests_historyfor long duration or high CPU time. - Avoid non-sargable functions in
WHEREclauses (e.g.WHERE YEAR(OrderDate) = 2026$\rightarrow$ useWHERE OrderDate >= '2026-01-01' AND OrderDate < '2027-01-01'). - Ensure auto-create statistics is active.