Build an AI Agent knowledge base using SQL (BigQuery + Gemini)

Google Cloud Tech · Beginner ·🔄 Data Engineering ·3mo ago
Skills: ML Pipelines60%

Key Takeaways

Builds an AI Agent knowledge base using SQL, BigQuery, and Gemini

Original Description

GCP credit → https://goo.gle/handson-ep2-lab1 Codelab & source code → https://goo.gle/scholar ML in BigQuery → https://goo.gle/3O5squw Did you know you can call a Gemini model directly from a SQL query in BigQuery? In this hands-on codelab, Ayo and Annie do exactly that, and use it to solve a real problem: converting messy, unstructured text into clean, structured data at scale. This is Episode 1 of our multi-part series where we build a fully functional, data-aware AI agent on Google Cloud. 🛠️ *What we cover:* * Loading raw text files from Cloud Storage as BigQuery external tables * Using BQML.GENERATE_TEXT to send prompts to Gemini inside SQL * Parsing and structuring LLM output using JSON functions in BigQuery * Building a clean, queryable dataset ready for downstream AI pipelines This pattern is incredibly powerful for any team sitting on a mountain of unstructured documents, and wanting to make them queryable without a heavy ETL pipeline. Chapters: 0:00 - Intro 1:44 - Claim GCP credit 2:40 - Data project overview 4:31 - Project set up 15:00 - ELT extraction loading transform intro 18:09 - Loading data 26:24 - BigQuery external table 33:52 [BQML] ML Generate In BigQuery Watch more Hand on AI → https://goo.gle/HowToWithGemini 🔔 Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech #Gemini #GoogleCloud Speakers: Ayo Adedeji, Annie Wang Products Mentioned: Gemini, BigQuery
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Related Reads

📰
I Built My Second ETL Pipeline. This Time, I Started Thinking Like a Data Engineer
Learn how to build a production-ready ETL pipeline with Python, Docker, PostgreSQL, and Kestra by thinking like a data engineer
Towards Data Science
📰
JuiceFS Sync for PB-Scale Data Transfers: Resumable Sync, Encryption, and Bandwidth Control
Learn how to efficiently transfer large volumes of data using JuiceFS Sync, which offers resumable sync, encryption, and bandwidth control, ideal for PB-scale data transfers.
Dev.to AI
📰
How Airflow is using AI to make data engineering more resilient, not more complex
Airflow uses AI to make data engineering more resilient by detecting data drift, resuming failed pipelines, and fixing issues automatically, reducing complexity and improving reliability.
Medium · AI
📰
What Can We Do When Memory Becomes the New Bottleneck in Data Engineering?
Learn how to overcome memory bottlenecks in data engineering using Pandas chunking, Dask, and Polars, and why it matters for processing large datasets
Towards Data Science

Chapters (8)

Intro
1:44 Claim GCP credit
2:40 Data project overview
4:31 Project set up
15:00 ELT extraction loading transform intro
18:09 Loading data
26:24 BigQuery external table
33:52 [BQML] ML Generate In BigQuery
Up next
A Moment Frozen in Time | Arnav Iyengar | TEDxJenks Youth
TEDx Talks
Watch →