Data Engineering on GCP (Part 5): Medallion Architecture with Dataform and dbt
Build a reliable BigQuery transformation layer with Medallion architecture, Dataform or dbt, incremental processing, deduplication, and practical data-quality controls.
Bookmark deep-dive architecture articles, guides, and tutorials to read anytime.
Explore ArticlesSynced across tabs • Zero tracking cookies
Build a reliable BigQuery transformation layer with Medallion architecture, Dataform or dbt, incremental processing, deduplication, and practical data-quality controls.
19 engineering guides & blueprints
Build a resilient GCP streaming pipeline with Pub/Sub, Apache Beam on Dataflow, raw-event landing, quarantine handling, event-time windows, and BigQuery Storage Write API sinks.

A practical guide to choosing between Pub/Sub BigQuery subscriptions, Dataflow, the BigQuery Storage Write API, and batch loads—without over-engineering the path.

A practical, end-to-end GCP data engineering lab covering Cloud Storage, BigLake external connections, partitioned and clustered BigQuery, materialized views, Time Travel, and authorized views.

Explore Google Cloud Gemini Enterprise Agent Platform Express Mode, compare API keys with ADC, and build a schema-constrained travel-intent parser in Python.

A step-by-step architectural guide to core Google Cloud data engineering storage and access primitives. From Cloud Storage to partitioned BigQuery, materialized views, and authorized views.

Configure production mTLS for Kubernetes and microservices: Java KeyStore vs TrustStore, cert-manager automation, PKIX path building troubleshooting, and recovery.

Master the TLS handshake packet-by-packet: 1-RTT vs 2-RTT flows, ECDHE key exchange, cipher suites, SNI routing, and practical OpenSSL debugging one-liners.

High-speed terminal shortcuts, imperative commands, JSONPath, Ingress, and troubleshooting one-liners to ace your Certified Kubernetes Administrator (CKA) exam.

A complete DevOps guide to TLS foundations: Private keys, CSR generation, Public vs Private CAs, trust stores, Diffie-Hellman, and OpenSSL CLI cheat sheets.

No engineering articles found for this topic.

Practitioner specializing in distributed backend systems, Kubernetes orchestration, and cloud infrastructure delivery. GCloud Cafe is an independent engineering journal documenting reproducible architectures, hands-on lab experiments, and real-world production engineering lessons.
Deep dives on cloud architecture, distributed systems, Kubernetes, and security. Written by practitioners, not content teams.
Curated architecture & DevOps insights. Zero spam, unsubscribe anytime.