Name: Unlimited
Brand: Storytel
SKU: 7777
Availability: InStock

Trex Team

"Deequ Data Quality: Constraint‑Based Validation for Big Data Pipelines"

Data quality failures in big data systems rarely look like broken code—they look like “successful” jobs shipping quietly corrupted tables. This book is for experienced data engineers, platform engineers, and analytics/ML practitioners who need enforceable guarantees, not ad‑hoc SQL spot checks. It treats data quality as an engineering discipline: explicit contracts, measurable signals, and operational response patterns that keep pipelines trustworthy without freezing delivery.

You’ll learn Deequ’s core model—metrics plus assertions—and how it maps onto Spark execution, cost, and reproducibility. The book goes deep on authoring production-grade constraints (completeness, uniqueness, validity, ranges, patterns, proportions), composing checks with stable thresholds, and turning failures into actionable diagnostics. It then operationalizes validation via VerificationSuite, showing how to plan analyzer execution, interpret VerificationResult edge cases, and implement gating strategies such as fail-fast, quarantine, and partial publishes. Profiling and constraint suggestion are covered as accelerators—followed by governance and rollout workflows that keep rules maintainable as data and business semantics evolve.

A strong working knowledge of Spark and DataFrames is assumed. Coverage includes longitudinal quality via metrics repositories, regression detection, and alerting, plus advanced patterns for partitioned/incremental data, late arrivals, custom analyzers, and real-world version compatibility across

NobleTrex Press

Deequ Data Quality: Constraint‑Based Validation for Big Data Pipelines

Accedi con...

Accedi

Accedi con

Crea un account

Inserisci un codice

Storytel

Con Storytel Unlimited, ottieni ascolto e lettura illimitati ogni mese, con accesso a un vasto catalogo di audiolibri ed ebook.