✕ Clear all filters
27 articles
▶ Videos →

📰 Dev.to · Ethan Walker

27 articles · Updated every 3 hours · View all reads

All Articles 185,344Blog Posts 168,177Tech Tutorials 49,634Research Papers 36,197News 22,822 ⚡ AI Lessons
Our eval gate runs 22 minutes. The queue behind it hit three hours.
Dev.to · Ethan Walker ☁️ DevOps & Cloud ⚡ AI Lesson 1mo ago
Our eval gate runs 22 minutes. The queue behind it hit three hours.
Nobody complained about the gate. They complained about Tuesday. Our merge queue runs the full eval...
Our few-shot examples came from the eval set. The 0.94 was fiction.
Dev.to · Ethan Walker 📐 ML Fundamentals ⚡ AI Lesson 2mo ago
Our few-shot examples came from the eval set. The 0.94 was fiction.
TL;DR. Our ticket-routing eval scored 0.94 for five weeks. The number was manufactured. We had built...
Our few-shot examples came from the eval set. The 0.94 was fiction.
Dev.to · Ethan Walker 📐 ML Fundamentals ⚡ AI Lesson 2mo ago
Our few-shot examples came from the eval set. The 0.94 was fiction.
TL;DR. Our ticket-routing eval scored 0.94 for five weeks. The number was manufactured. We had built...
We gated CI on six open-source LLM eval frameworks. Only two survived the merge queue.
Dev.to · Ethan Walker 🧠 Large Language Models ⚡ AI Lesson 2mo ago
We gated CI on six open-source LLM eval frameworks. Only two survived the merge queue.
TL;DR. Most "top open-source LLM eval framework" roundups rank features. None of them ask the one...
The golden set stopped catching regressions the day traffic changed
Dev.to · Ethan Walker 📐 ML Fundamentals ⚡ AI Lesson 2mo ago
The golden set stopped catching regressions the day traffic changed
TL;DR. Our overall eval pass rate read 0.88 through a model change and looked stable. Sliced by...
Your LLM-as-judge disagrees with itself between runs
Dev.to · Ethan Walker 🧠 Large Language Models ⚡ AI Lesson 2mo ago
Your LLM-as-judge disagrees with itself between runs
Same outputs, same judge, two runs, two scores. The gate flickered red then green on a branch with...
LLM-as-judge disagrees with itself between runs
Dev.to · Ethan Walker 🧠 Large Language Models ⚡ AI Lesson 2mo ago
LLM-as-judge disagrees with itself between runs
The flap I had a faithfulness gate on merge: judge scores every case, the mean has to clear 0.80....
When an LLM answer is wrong, the trace is where you look. Some tools make that easy.
Dev.to · Ethan Walker 🧠 Large Language Models ⚡ AI Lesson 2mo ago
When an LLM answer is wrong, the trace is where you look. Some tools make that easy.
A user reports a hallucinated answer in prod. To fix it you need the full trace of that one request,...
our CI passed. Your agent isn't operator-ready.
Dev.to · Ethan Walker ☁️ DevOps & Cloud ⚡ AI Lesson 2mo ago
our CI passed. Your agent isn't operator-ready.
Your CI passed. Your agent isn't operator-ready. We shipped a document-extraction agent to...
The stale eval fixture that passed a broken model
Dev.to · Ethan Walker 🔧 Backend Engineering 2mo ago
The stale eval fixture that passed a broken model
The stale eval fixture that passed a broken model A regression shipped green last month....
91% pass rate. Gate green. Shipped. Worst regression we had all quarter.
Dev.to · Ethan Walker 📐 ML Fundamentals ⚡ AI Lesson 3mo ago
91% pass rate. Gate green. Shipped. Worst regression we had all quarter.
The gate was a fixed 90% threshold on an intent-classification eval. The change came in at 91%,...
Datadog dashboards for prompt regression: the panels we actually keep
Dev.to · Ethan Walker 🧠 Large Language Models ⚡ AI Lesson 3mo ago
Datadog dashboards for prompt regression: the panels we actually keep
We wired our LLM eval suite into Datadog over about four months. Most of the panels we built...
Promptfoo is a CI gate, not an eval framework. Treating it like one cost us $4,200
Dev.to · Ethan Walker 📰 AI News & Updates ⚡ AI Lesson 3mo ago
Promptfoo is a CI gate, not an eval framework. Treating it like one cost us $4,200
Last Monday I logged into our billing dashboard and saw a $4,200 LangSmith spike from the weekend....
Building Accessible Dialog Components with Melt UI in Svelte
Dev.to · Ethan Walker 🖌️ UI/UX Design ⚡ AI Lesson 8mo ago
Building Accessible Dialog Components with Melt UI in Svelte
Melt UI is a headless, accessible component library for Svelte that provides unstyled UI primitives...
Advanced Theme Customization and Dynamic Color Schemes with m3-svelte in Svelte
Dev.to · Ethan Walker 🖌️ UI/UX Design ⚡ AI Lesson 8mo ago
Advanced Theme Customization and Dynamic Color Schemes with m3-svelte in Svelte
m3-svelte is a powerful Material Design 3 component library for Svelte, providing a complete set of...
Building Advanced Component Scaffolding with lomer-ui CLI in Svelte
Dev.to · Ethan Walker 🖌️ UI/UX Design ⚡ AI Lesson 8mo ago
Building Advanced Component Scaffolding with lomer-ui CLI in Svelte
lomer-ui is a dead-simple CLI tool that instantly kickstarts your Svelte components, allowing...
Building Native Mobile Features with Capacitor and ionic-svelte in Svelte
Dev.to · Ethan Walker 🖌️ UI/UX Design ⚡ AI Lesson 8mo ago
Building Native Mobile Features with Capacitor and ionic-svelte in Svelte
ionic-svelte provides seamless integration between Ionic's mobile UI components and Svelte, enabling...
Building Advanced Data Tables with Server-Side Processing in flowbite-svelte and Svelte
Dev.to · Ethan Walker 🖌️ UI/UX Design ⚡ AI Lesson 8mo ago
Building Advanced Data Tables with Server-Side Processing in flowbite-svelte and Svelte
flowbite-svelte is a comprehensive UI component library built on Tailwind CSS that provides...
Building Advanced Modal Systems with State Management in daisyUI and Svelte
Dev.to · Ethan Walker 🌐 Frontend Engineering ⚡ AI Lesson 8mo ago
Building Advanced Modal Systems with State Management in daisyUI and Svelte
daisyUI is a popular Tailwind CSS component library that provides ready-to-use component classes for...
Building Forms with Validation in carbon-components-svelte and Svelte
Dev.to · Ethan Walker 🖌️ UI/UX Design ⚡ AI Lesson 8mo ago
Building Forms with Validation in carbon-components-svelte and Svelte
carbon-components-svelte is a Svelte component library that implements IBM's Carbon Design System,...