External: Coursera Courses ↗ · Coursera

Open Course on External: Coursera

Free to audit · Opens on External: Coursera

Cloud Composer: Copying BigQuery Tables Across Different Locations

Coursera · Advanced ·🔄 Data Engineering ·3mo ago

Skills: Tool Use & Function Calling90%AI Workflow Automation80%

Key Takeaways

Copies BigQuery tables across different locations using Cloud Composer

Original Description

This is a self-paced lab that takes place in the Google Cloud console. In this advanced lab you will create and run an Apache Airflow workflow in Cloud Composer that exports tables from a BigQuery dataset located in Cloud Storage bucktes in the US to buckets in Europe, then import th0se tables to a BigQuery dataset in Europe.

Watch on External: Coursera ↗ (saves to browser)

Sign in to unlock AI tutor explanation · ⚡30

More on: Tool Use & Function Calling

View skill →

Adding a Phone Gateway to a Virtual Agent

Administering an AlloyDB Database

Cloud Storage: Qwik Start - CLI/SDK

Getting started with Firebase Cloud Firestore

Getting Started with Liquid to Customize the Looker User Experience

Getting Started with Neo4J Enterprise on Google Cloud

Related Reads

I Built My Second ETL Pipeline. This Time, I Started Thinking Like a Data Engineer

Learn how to build a production-ready ETL pipeline with Python, Docker, PostgreSQL, and Kestra by thinking like a data engineer

Towards Data Science

JuiceFS Sync for PB-Scale Data Transfers: Resumable Sync, Encryption, and Bandwidth Control

Learn how to efficiently transfer large volumes of data using JuiceFS Sync, which offers resumable sync, encryption, and bandwidth control, ideal for PB-scale data transfers.

How Airflow is using AI to make data engineering more resilient, not more complex

Airflow uses AI to make data engineering more resilient by detecting data drift, resuming failed pipelines, and fixing issues automatically, reducing complexity and improving reliability.

What Can We Do When Memory Becomes the New Bottleneck in Data Engineering?

Learn how to overcome memory bottlenecks in data engineering using Pandas chunking, Dask, and Polars, and why it matters for processing large datasets

Towards Data Science

A Moment Frozen in Time | Arnav Iyengar | TEDxJenks Youth