AI Engineering Architecture & User Feedback for LLMs | Chapter 10

Chapter 10: AI Engineering Architecture and User Feedback. So far, this book has covered a wide range of techniques to adapt foundation models to specific applications. This chapter will discuss how to bring these techniques together to build successful products. Given the wide range of AI engineering techniques and tools available, selecting the right ones can feel overwhelming. To simplify this process, this chapter takes a gradual approach. It starts with the simplest architecture for a foundation model application, highlights the challenges of that architecture, and gradually adds components to address them. We can spend eternally reasoning about how to build a successful application, but the only way to find out if an application actually achieves its goal is to put in front of users. User feedback has always been invaluable for guiding product development, but for AI applications, user feedback has an even more crucial role as a data source for improving models. The conversational interface makes it easier for users to give feedback, but harder for developers to extract signals. AI engineering architecture A full-fledged AI architecture can be complex. This section follows the process that a team might follow in production, starting with the simplest architecture and progressively adding more components. Despite the diversity of AI applications, they share many common components. The architecture proposed here has been validated at multiple companies to be general for a wide range of applications, but certain applications might deviate. In its simplest form, your application receives a query and sends it to the model. The model generates a response, which is returned to the user, as shown in figure 10.1. There is no context augmentation, no guardrails, and no optimization. The {underscore}model_api_box refers to both third-party APIs, e.g., OpenAI, Google, Anthropic, and self-hosted models. Building an inference server for self-hosted model This image shows a diagram of a model description automatically generated. Figure 10.1. From this simple architecture, you can add more components as needs arise. One. Enhance context input into a model by giving the model access. One, putting guardrails to protect your system and your users. One, add model router and gateway to support complex pipelines and add more security. One, optimize for latency and cost with caching. One, add complex logic and write actions to maximize your system. This chapter follows the progression I commonly see in production. However, everyone's needs are different. You should follow the order that makes the most sense for your application. Monitoring and observability. Step one, enhance context. The initial expansion of a platform usually involves adding mechanisms to allow the system to construct the relevant context needed by the model to answer each query. As discussed in chapter six, context can be constructed through various retrieval mechanisms, including text retrieval, image retrieval, and tabular data retrieval. Context can also be augmented using tools that allow the model to automatically gather information through APIs such as web search, news, weather, events, etc. >> Underscore context construction is like feature engineering for foundation models. Underscore it gives the model the necessary information to produce an output. Due to its central role in a system's output quality, context construction is almost universally supported by model API providers. For example, providers like OpenAI, Claude, and Gemini allow users to upload files and allow their models to use tools. However, just like models differ in their capabilities, these providers With context construction, the arc This image shows a diagram of a database description automatically generated. Figure 10.2, a platform architecture with context construction. Step two, putting guardrails. Guardrails help mitigate risks and protect you and your users. They should be placed whenever there are exposures to risks. In general, they can be categorized into guardrails around inputs and outputs. Input guardrails. Input guardrails typically protect against two types of risks, leaking private information to external APIs and executing bad prompts that compromise your system. Chapter 5 discusses many different ways attackers can exploit an application through prompt hacks and how to defend your application against them. While you can mitigate risks, they can never be fully eliminated due to the inherent nature of how models generate responses. Leaking private information to external APIs is a risk specific to using external model APIs when you need to send your data outside your organization. This might happen for many reasons, including the following. An employee copies the company's secret or user's private information into a prompt and sends it to a third-party API. {point} 1 An application developer puts the company's internal policies and data A tool retrieves private information from internal database and adds There's no airtight way to eliminate potential leaks when using third-party APIs. However, you can mitigate them with guardrails. You can use one of the many available tools that automatically detect sensitive data. What sensitive data to detect is specified by you. Common sensitive data classes are the following. Personal information, ID numbers, phone numbers, bank accounts. Human faces. Specific keywords and phrases associated with the company's intellectual property or privileged information. Figure 10-3, an example of masking and unmasking PII information using a reverse PII map to avoid sending it to external APIs. Output guardrails. A model can fail in many different ways. Output guardrails have two main functions. Catch output failures. Specify the policy to handle different failure modes. To catch outputs that fail to meet your standards, you need to understand what failed. Quality. Malformatted responses that don't follow the expected output format. For example, the application expects JSON and a model generates invalid JSON. >> Factually inconsistent responses hallucinated by the model. Generally bad responses. For example, you ask the model to write an essay, and that essay is just bad. Security. Toxic responses that contain racist content, sexual content, or illegal activities. Responses that contain private and sensitive information. Responses that trigger remote tool and code execution. Brand risk responses that mischaracterize your company or your competitors. Recall from chapter 5 that for security measurements, it's important to track not only the security failures, but also the false refusal rate. It's possible to have systems that are too secure, e.g. one that blocks even legitimate requests, interrupting user workloads, and causing user frustration. Many failures can be mitigated by a simple retry logic. AI models are probabilistic, which means that if you try a query again, you might get a different response. For example, if the response is empty, try again x times or until you get a non-empty response. Similarly, if the response is malformed, try again until the response is correctly formatted. This retry policy, however, can incur extra latency and cost. Each retry means another round of API calls. If the retry is carried out after failure, the user perceived latency will double. To reduce latency, you can make calls in parallel. For example, for each query, instead of waiting for the first query to fail before retrying, you send this query to the model twice at the same time, get back two responses, and pick the better one. This increases the number of redundant API calls, while keeping latency manageable. It's also common to fall back on humans for tricky requests. For example, you can transfer the queries that contain specific phrases to human operators. Some teams use a specialized model to decide when to transfer a conversation to humans. One team, for instance, transfers a conversation to human operators when their sentiment analysis model detects anger in users' messages. Another team transfers a conversation after a certain number of turns to prevent users from getting stuck in a loop. Guardrail implementation. Guardrails come with tradeoffs. One is the underscore reliability versus latency tradeoff underscore. While acknowledging the importance of guardrails, some teams told me that latency is more important. The teams decided not to implement guardrails because they can significantly increase the application's latency point three. Output guardrails might not work well in a stream completion mode. By default, the whole response is generated before being shown to the user, which can take a long time. In a stream completion mode, new tokens are streamed to the user as they are generated, reducing the time the user has to wait to see the response. The downside is that it's hard to evaluate partial responses. So, unsafe responses might be streamed to users before the system guardrails can determine that they should be blocked. How many guardrails you need to implement also depends on whether you self-host your models or use third-party APIs. While you can implement guardrails on top of both, third-party APIs can reduce the guardrails you need to implement since API providers typically provide many guardrails out of the box for you. At the same time, self-hosting means that you don't need to send requests externally, which reduces the need for many types of input guardrails. Given the many different places where an application might fail, guardrails can be implemented at many different levels. Model providers give their models guardrails to make their models better and more secure. However, model providers have to balance safety and flexibility. Restrictions might make a model safer, but can also make it less usable for specific use cases. Guardrails can also implemented by application developers. Many techniques are discussed in defenses against prompt attacks. Guardrail solutions that you can use out of the box include Meta's Purple Llama, Nvidia's NeMo Guardrails, Azure's PYRIT, Azure's AI content filters, the Perspective API, and OpenAI's content moderation API. Due to the overlap of risks in inputs and outputs, a guardrail solution will likely provide protection for both inputs and outputs. Some model gateways also provide guardrail functionalities, as discussed in the next section. >> With guardrails, the architecture looks like figure 10-4. I put scores under model API since scores are often AI-powered. Even if scores are typically smaller and faster than generative models. However, scores can also be placed in the output guardrails box. This image shows a diagram of a diagram description automatically generated. Figure 10-4. Application architecture with the addition of input and output guardrails. Step three, add model router and gateway. As applications grow to involve more models, routers and gateways emerge to help you manage the complexity and costs of serving multiple models. Router. Instead of using one model for all queries, you can have different solutions for different types of queries. This approach has several benefits. First, it allows specialized models, which can potentially perform better than a general-purpose model for specific queries. For example, you can have one model specialized in technical troubleshooting and another specialized in billing. Second, this can help you save costs. Instead of using one expensive model for all queries, you can route simpler queries to cheaper models. A router typically consists of {underscore} an intent classifier {underscore} that predicts what the user is trying to do. Based on the predicted intent, the query is routed to the appropriate solution. As an example, consider different intentions relevant to a customer support chatbot. If a user wants to reset the password, route them to the FAQ page about recovering the password. If the request is to correct a billing mistake, route it to a human operator. If the request is about troubleshooting a technical issue, route it to a chatbot specialized in troubleshooting. An intent classifier can prevent your system from engaging in out-of-scope conversations. If the query is deemed inappropriate, the chatbot can politely decline to respond using one of the stock responses without wasting an API call. For example, if If user asked who vote for in the upcoming election, a chatbot can respond with, "As a chatbot, I don't have the ability to vote. If you have questions about our products, I'd be happy to help." An intent classifier can help the system detect ambiguous queries and ask for clarification. For example, in response to the query "freezing", the system might ask, "Do you want to freeze your account, or are you talking about the weather?" Or simply ask, "I'm sorry. Can you elaborate?" Other routers can aid the model in deciding what to do next. For example, for an agent capable of multiple actions, a router can take the form of a {underscore}next_action_predictor{underscore}. Should the model use a code interpreter or search API next? For a model with a memory system, a router can predict which part of the memory hierarchy the model should pull information from. Imagine that a user attaches a document that mentions Melbourne to the current conversation. Later on, the user asks, "What's the cutest animal in Melbourne?" The model needs to decide whether to rely on the information in the attached document or to search the internet for this query. Intent classifiers and next action predictors can be implemented on top of foundation models. Many teams adapt smaller language models like GPT-2, BERT, and Llama 7B as their intent classifiers. Many teams opt to train even smaller classifiers from scratch. Routers should be fast and cheap so that they can use multiples of them without incurring significant extra latency and cost. When routing queries to models with varying context limits, the query's context might need to be adjusted accordingly. Consider a 1,000 token query that is slated for a model with a 4K context limit. The system then takes an action, e.g., a web search that brings back 8,000 token context. You can either truncate the query's context to fit the originally intended model or route the query to a model with a larger context limit. Because routing is usually done by models, I put routing inside the model API box in figure 10.5. Like scores, routers are typically smaller than models used for generation. Grouping routers together with other models makes models easier to manage. However, it's important to note that routing often happens {underscore} before {underscore} retrieval. For example, before retrieval, a router can help determine if a query is in scope and if yes, if it needs retrieval. Routing can happen after retrieval, too, such as determining if a query should be routed to a human operator. However, routing, retrieval, generation, scoring is a much more common AI application pattern. This image shows a diagram of a system description automatically generated. Figure 10.5, routing helps the system use the optimal solution for each query. Gateway. A model gateway is an intermediate layer that allows your organization to interface with different models in a unified and secure manner. The most basic functionality of a model gateway is to provide a unified interface to different models, including self-hosted models and models behind commercial APIs. A model gateway makes it easier to maintain your code. If a model API changes, you only need to update the gateway instead of updating all applications that depend on this API. Figure 10.6 shows a high-level visualization of a model gateway. >> This image shows a diagram of a model gateway description automatically generated. Figure 10-6. A model gateway provides a unified interface to work with different models. In its simplest form, a model gateway is a unified wrapper. The following code example gives you an idea of how a model gateway might be implemented. It's not meant to be functional, as it doesn't contain any error checking or optimization. Here is the code for this step. Focus on the structure first, then connect each line to the idea we just introduced. A model gateway provides {underscore} access control and cost management {underscore}. Instead of giving everyone who wants access to the Open AI API, your organizational tokens, which can be easily leaked, you give people access only to the model gateway, creating a centralized and controlled point of access. The gateway can also implement fine-grained access controls, specifying which user or application should have access to which model. Moreover, the gateway can monitor and limit the usage of API calls, preventing abuse, and managing costs effectively. A model gateway can also be used to implement fallback policies to overcome rate limits or API failures. The latter is unfortunately common. When the primary API is unavailable, the gateway can route requests to alternative models, retry after a short wait, or handle failures gracefully in other ways. This ensures that your application can operate smoothly without interruptions. Since requests and responses are already flowing through the gateway, it's a good place to implement other functionalities such as load balancing, logging, and analytics. Some gateways even provide caching and guardrails. Given that gateways are relatively straightforward to implement, there are many off-the-shelf gateways. Examples include Portkey's AI Gateway, ML Flow AI Gateway, Well Simple's LLM Gateway, TrueFoundry, Kong, and Cloudflare. >> In our architecture, the gateway now replaces the model API box, as shown in figure 10-7. This image shows a diagram of a data flow description automatically generated. Figure 10-7. The architecture with the added routing and gateway modules. Note: A similar abstraction layer, such as a tool gateway, can also be useful for accessing a wide range of tools. It's not discussed in this book, since it's not a common pattern as of this writing. Now we'll look at caching as a simple way to cut latency and cost. The main idea is to reuse work the system has already done instead of recomputing it every time. Caching is a classic software technique, and it transfers well to AI systems. Here we're focusing on system caching, while inference-side caching like KV cache and prompt cache is covered elsewhere. At a high level, there are two main kinds: exact caching and semantic caching. >> Let's start with a simpler case. Exact caching only reuses a result when the incoming request matches a cached request exactly. This is straightforward. If the same product summary is requested again, the system returns the stored answer immediately. If the request is new, it computes the summary once and saves it for next time. The same idea helps with embedding-based retrieval. If the query has already been searched before, the system can skip the vector search and return the cached retrieval result instead. Caching pays off most when a request triggers expensive work like retrieval, SQL execution, web search, or multi-step reasoning pipeline. The more work you avoid, the more useful the cache becomes. A cache can live in memory for speed, but limited capacity usually forces a more flexible design. In practice, teams may use Redis, PostgreSQL, or tiered storage, and any eviction policy like LRU, LFU, or FIFO to keep the cache healthy. Not every query should be cached. User-specific or time-sensitive often have low reuse value. So, teams usually avoid storing them. And some systems even use a classifier to predict whether a request is worth caching. There's an important safety issue here. A cache can accidentally reuse personalized information across users if the system treats a private query like a generic one. This is the dangerous failure mode. One user's answer gets cached, but that answer included account-specific information. If another user later gets the same cached response, the system has leaked data across users. Semantic caching goes one step further. Instead of requiring an exact match, it tries to reuse answers for queries that mean the same thing. For example, two differently worded questions about the capital of Vietnam can share the same cached answer. That can raise the hit rate and reduce cost, but the trade-off is that performance may suffer if the similarity judgment is wrong. Semantic caching depends on a reliable way to measure similarity. A common approach is to embed each query, compare it against cached embeddings, and use a similarity threshold to decide whether the cached result is close enough. First, represent the query as a vector. This gives the cache a numerical form of the question that can be compared efficiently. Use an embedding model to turn the incoming query into a vector. That vector becomes the basis for similarity comparison. Next, compare that vector with what is already in the cache. The goal is to find the closest stored query, not just any stored query. This is uses vector search to find the cached embedding with the highest similarity score, which we'll call X. That score tells us how close the new query is to something we have already answered. Finally, the similarity score decides the action. If the match is strong enough, reuse the answer. Otherwise, compute a fresh one and store it. If X is above the threshold, return the cached result. If not, process the query normally, then save both the embedding and the result for future reuse. Because this depends on embedding search, semantic caching usually requires a vector database to store and retrieve the cached query vectors efficiently. This approach is more fragile than exact caching because several parts can fail: embeddings, vector search, and the similarity metric itself. If the threshold is poorly tuned, the cache may return the wrong answer, which is a serious correctness risk. >> There's also overhead. A semantic cache can be slow and expensive because every lookup may require vector search, and that cost grows with the size of the cached embedding set. So, semantic caching is only worth it when many requests truly repeat in meaning. Before adding that complexity, it is important to weigh efficiency, cost, and the risk of returning the wrong response. With these caches added, the application architecture gets an extra layer for storing generated responses. In practice, model API providers often handle KV cache and prompt cache internally, while your system cache sits around the application flow. Notice the additional cache path in the diagram. The key idea is that responses can now be stored and reused, which reduces repeated work across the system. This figure closes the loop. Requests come in, the system checks for reusable results, and cached outputs can be returned instead of recomputed. That is the architectural payoff of caching. Now we move from simple sequential flows to agentic patterns. This section shows how loops, branching, and write actions can make an application much more capable and also much more complex. So far, each query has followed a straight path. Here, the key idea is that the system can inspect its own result, decide it needs more information, and send that result back through retrieval or another model call until the task is complete. Notice the feedback path in this diagram. The important point is that the model output is not always the end of the process. It can become input for another round, which is what makes the application agentic. This figure highlights the yellow return arrow. That arrow is the mechanism that lets the system reuse its own generated response and build more complex multi-step behavior. >> Agent outputs can also trigger real-world actions, like sending email, placing an order, or starting transfer. That power is useful, but it also raises the stakes. So, right access has to be tightly controlled and carefully designed. As these pieces accumulate, the architecture becomes harder to reason about and debug. The next topic is observability, which helps you understand where failures happen and how the system is behaving. Here the diagram shows the system expanding beyond read-only behavior. The main thing to notice is that the model can now drive actions that change external state, not just produce text. This figure represents an architecture that can perform write actions. It is more capable, but it also needs stronger safeguards because the model is now connected to operations that affect the outside world. Now we move from building the system to understanding how it behaves in a real world. This section is about making observability part of the design, so you can detect problems early and improve the product over time. Observability should not be something you bolt on after launch. As products get more complex, the need to see what is happening inside the system becomes much more important. This is a mature area in software engineering with many proven tools and best practices already available. So instead of reinventing everything, we focus on what is specific to foundation model applications and how to monitor those well. Monitoring serves the same broad purpose as evaluation, reduce risk and reveal opportunities. It helps catch failures, attacks, and drift, but it also shows where you can improve quality, lower cost, and stay accountable for system performance. A useful way to judge observability is with three operational metrics. These come from DevOps practice and they tell you how quickly you notice issues, how quickly you recover, and how often changes cause problems. First, let's define the metrics one by one so the names become easy to remember. MTTD means mean time to detection. If something goes wrong, this measures how long it takes before your system or team notices it. Once a problem is detected, the next question is how fast you can act on it. MTTR means mean time to response or recovery in practice. It measures the time from detection to resolution, which is a direct sign of how effective your incident handling is. The last metric tells you how safe your releases are. CFR means change failure rate, the percentage of deployments that need fixes or rollbacks. If you do not know this number, your platform probably needs better observability and safer release processes. >> A high CFR does not automatically mean monitoring is bad. It may mean the evaluation pipeline is not catching issues early enough. The key idea is that evaluation and monitoring should reinforce each other. What works in evaluation should continue to look good in production, and production issues should feed back into evaluation. In this section, we'll separate two ideas that are often used interchangeably, but are not the same. The key difference is whether you're only watching for problems, or whether your instrumentation is good enough to help explain them. Traditional monitoring starts from the outside. You look at outputs and alerts, then infer that something may be wrong inside the system. That works for detection, but it does not guarantee you'll understand the root cause. Observability goes a step further. It assumes your logs and metrics contain enough signal to reconstruct what happened without shipping new code, so debugging becomes faster and more reliable. >> Here, I'll use the words in a practical way. Monitoring means collecting system information, while observability means the full loop of instrumenting, tracking, and investigating what the system is doing. Let's start with metrics because that's what most people mean when they say monitoring. The important point is that metrics are only useful when they answer a question that matters to the system or the business. A metric is not valuable just because it is measurable. It should help you detect problems or find opportunities to improve the application, otherwise it is just noise on a dashboard. The right way to choose metrics is to start with failure modes. If hallucinations are a concern, track signals that reveal them. If cost is the concern, track token usage, cache behavior, and anything that drives spend. Because foundation models can fail in many open-ended ways. Metric design takes judgment. You need to think statistically, understand product, and often invent the right signal for your use case. The book has already covered many ways to measure model quality. So, here the goal is just to connect those ideas back to monitoring. We'll now review the most practical kinds of metrics to keep an eye on in production. The easiest failures to catch are format problems because they are obvious and cheap to verify. If you expect JSON, measure how often the output is invalid and whether those errors are easy to repair. For open-ended text, you usually need softer quality measures such as factual consistency, conciseness, creativity, or positivity. Many teams now use AI judges to score those outputs at scale. Safety monitoring should cover toxicity, private data, guardrail triggers, refusals, and unusual prompts. Those signals help you spot both harmful content and attack patterns before they become bigger problems. You can also infer quality from how users behave. When people stop a response early, ask many turns, or change how they prompt, those are often useful clues about system quality. These are the kinds of usage questions that turn into practical metrics. A simple count can reveal whether users are losing interest, getting what they need quickly, or abandoning the conversation. If users interrupt a generation, that could be a signal that the output is too slow, too long, or simply not useful. It is one of the most direct behavioral indicators of friction. Conversation length tells you something about engagement and task complexity. More turns can mean the system is helpful, but it can also mean the model is not resolving the user's request efficiently. Track this as a trend, not just a single number. A rising or falling average can show that users are changing how they work with the system. Input length can reveal whether users are asking more complex questions or learning to prompt more efficiently. It also connects directly to cost and latency. A change here may reflect user behavior, not model quality. That distinction matters because the metric can still move even when the underlying system has not changed. Output length is another useful signal. Some models are naturally more verbose, and some query types invite longer answers. So, compare the distribution over time and across use cases. This metric helps you detect verbosity changes and query mix changes. If outputs start getting longer, that can affect both user experience and your operating cost. Instead of only tracking averages, look at the full distribution. That makes it easier to see whether responses are becoming more repetitive or more diverse over time. Distribution changes can reveal drift that averages hide. This is a good example of why monitoring should look for patterns, not only single summary numbers. Length metrics matter for more than quality. Longer contexts and longer responses usually increase both latency and cost. So, these signals affect the user experience directly. Every stage in the pipeline needs its own measurements. In a RAG system, for example, retrieval quality is one set of metrics, while vector database storage and query time are another. Once you have several metrics, compare them to your North Star metrics, such as DAU, session duration, or subscriptions. That tells you which signals matter and which ones are just interesting, but not actionable. Latency deserves its own attention because it shapes the user experience immediately. The usual breakdown is time to first token, time per output token, and total response time. These are the core timing signals to watch. Each one tells you something slightly different about where the delay is happening. TTFT measures how quickly the system starts responding. It is often the number users feel first because it determines whether the interface feels alive or stuck. TPOT measures generation speed after the first token arrives. This helps you see whether the model is slow throughout the whole completion or only slow to start. This metric is especially useful when comparing models or deployments. A small change here can have a large effect on how long long answers take to finish. Total latency is the end-to-end delay from request to completed answer. It is the simplest measure of whether your system feels fast enough in practice. This is the number most users experience directly. It combines startup delay and generation time into one practical measure. Always break latency down by user or traffic segment. That helps you see how the system scales and whether some users are getting a worse experience than others. Cost monitoring should include query volume and token volume. It should also respect provider rate limits. If you exceed those limits, the result can be failed requests or service interruptions. You do not always need to inspect every request. Spot checks are faster and cheaper, while exhaustive checks give a complete view. So, the best system is often use both. Good metrics should be sliceable by user, release, prompt version, chain type, and time. That granularity is what lets you localize issues instead of only seeing a blended average. Metrics tell you that something changed, but logs and traces help you understand what happened. This next part is about moving from detection to investigation. Metrics are aggregated, so they show the shape of the system, but not the individual events. When you see a spike, logs help you check whether the same pattern appeared before and under what conditions. Think of logs as an append-only event history. Once you have the event record, you can reconstruct the sequence around a problem instead of guessing from summary data. First, the metric tells you there was an issue. The number alone is not enough to diagnose it. A metric can tell you that something went wrong 5 minutes ago, but it usually cannot tell you why. That is the gap logs are meant to fill. Next, you move to the logs from that same time window. You are looking for the concrete events that explain the metric change. The log trail often reveals the actual failure, a bad input, a tool error, a timeout, or an unexpected branch in the pipeline. That is what turns an alert into an explanation. Finally, connect the errors in the logs back to the metrics. That correlation tells you whether you have found a real root cause or just a symptom. This last step matters because logs alone can be noisy. By matching them to the metric spike, you can confirm that the error pattern really explains the observed behavior. For observability to work, both sides have to be timely. Metrics need to update quickly, and logs need to arrive soon enough that you can still investigate the incident while it's fresh. The safest logging rule is to record more than you think you need because future debugging questions are hard to predict. That includes configuration, sampling settings, prompt templates, and model endpoints. You also want the full execution record, user query, final prompt, output, intermediate steps, tool calls, and tool results. Tags and IDs are essential so you can connect one log line to the right request. >> Once you log everything, the volume grows quickly, which is why log analysis and anomaly detection are often automated. AI tools are especially useful when a stream is too large to read manually. Even if you cannot read every log, a daily manual review is still valuable. It helps you build intuition about real user behavior and improve both prompts and evaluation rules. Traces go one step beyond logs by linking related events into a single timeline. In an AI system, that means you can follow a request from a user query all the way to the final response, including retrieval, tool use, timing, and cost. The goal is to pinpoint exactly where failure happened. If the answer is wrong, a trace should help you see whether the issue came from pre-processing, retrieval, or generation. >> This image is meant to show a request trace in context. Notice how a trace lets you inspect the full path of a single interaction instead of only seeing an aggregate metric. This figure is the visual version of the trace idea we just discussed. It shows how one request can be broken into steps, making it easier to debug each stage separately. Now, let's move to drift detection. Once a system is in production, the inputs, prompts, and model behavior can all change over time. So, you need a way to notice those shifts early. The more components a system has, the more opportunities there are for something to change. In AI applications, drift can come from prompts, users, or the underlying model itself. A prompt can change without anyone meaning to alter the system's behavior. Even a small edit in a template can shift the model's responses. So, you want a simple way to detect that change. >> This is why prompt versioning and prompt checks matter. If the system prompt changes unexpectedly, you want to know immediately before the downstream behavior drifts. Users also adapt. Over time, they learn how to phrase requests to get better outputs, which can shift your metric trends even if the application code stays the same. That is why a slow drift in response length or prompt style is not always obvious from metrics alone. You often need investigation to explain why the pattern changed and whether it is good or bad. Another source of drift is the model behind an API. The interface may stay the same while the provider silently updates the model, and that can change performance in noticeable ways. That is why production monitoring has to watch for version shifts, not just code changes. A model update can improve one metric while harming another. So, you need ongoing detection rather than one-time evaluation. >> In this section, we look at how to coordinate multiple AI components into one working system. The key idea is not just having models, but making sure they receive the right inputs in the right order with the right checks along the way. As AI applications grow, they often combine several models, database lookups, and external tools. An orchestrator defines how those pieces interact, so the whole system behaves like a single pipeline instead of a collection of separate parts. The first step is to list what your system uses. They usually includes models, retrieval sources, and tools, plus anything you need for evaluation or monitoring. This is where you tell the orchestrator what exists in the system. A model gateway can simplify adding or swapping models. And you can also register tools for testing, monitoring, or other support tasks. Once the components are defined, you connect them into a sequence. That chaining is what turns isolated parts into a usable application flow. You can think of chaining as function composition. One step produces the input for the next. The orchestrator describes the journey from user query to completed answer. First, we start with a raw user query. This is the unprocessed input that begins the pipeline. The system may clean up the query, classify it, or extract useful structure before doing anything else. Next comes retrieval, where the system looks for the most relevant supporting data. At this stage, the orchestrator sends the processed query to the right data source, so the model has better context. Then the system prepares the final prompt by combining the query and the retrieved information. This step matters because the model expects input in a specific format. The orchestrator packages the pieces, so the model can use them correctly. >> Now the prepared prompt is ready for generation. Here the model produces the answer based on the assembled prompt. Good orchestration makes sure the model sees the right context, not just the raw question. After generation, the system checks the result before delivering it. Evaluation helps decide whether the answer is good enough, needs correction, or should be handled differently. The last step is the final handoff. If the response passes, it goes back to the user. If it fails quality checks, the orchestrator can route the query to a human operator instead. The orchestrator's job is to move data between steps and make sure each step receives what it expects. It should also surface failures, such as component errors or mismatched data formats, so problems are easier to catch. There's an important distinction here. An AI pipeline orchestrator is not the same as a general workflow tool. The two may overlap, but they are optimized for different kinds of work. Tools like Airflow or Metaflow are general workflow orchestrators. An AI pipeline orchestrator focuses more specifically on model calls, retrieval, tool use, and a data pass between them. When low latency is important, do independent steps in parallel whenever possible. For example, routing and PI removal can happen at the same time because neither depends on the other. Several tools support this style of application building, including LangChain, LlamaIndex, Flowise, LangFlow, and Haystack. Many RAG and agent frameworks also count as orchestration tools because they coordinate retrieval and tool use. It is often better to build the first version without an orchestrator. Extra abstraction can hide important details and make debugging harder, especially early in a project. >> Once your application grows, an orchestrator can become useful. At that point, it helps to evaluate a few practical qualities before choosing one. First, check whether the tool supports the components you already use and the ones you may adopt later. If something is missing, the question becomes how difficult it is to extend. No orchestrator supports everything, so compatibility matters. If you want to add a new model or database, you should know whether the integration is already available or easy to build. As systems get larger, you may need branching logic, parallel execution, and error handling. A capable orchestrator should handle those patterns without forcing you into awkward workarounds. These features matter because real AI systems are rarely linear. The orchestrator should help manage complexity instead of adding more of it. Finally, think about how easy the tool is to use and whether will stay fast as demand grows. A good orchestrator should be clear to learn, avoid hidden overhead, and scale with your team and traffic. Good documentation, strong community support, and predictable behavior all reduce friction. If the orchestrator adds latency or hides too much of the system, it may slow you down instead of helping. In this section, we'll look at why user feedback matters so much in AI products. It is not just a signal for improving the app. It is also a source of proprietary data that can become a real advantage. In traditional software, feedback helps you judge how well the product is working and what to improve next. In AI systems, it goes further. The feedback itself becomes valuable data, and that data helps create the flywheel that keeps improving the product over time. User feedback can personalize the model for a single person, but it can also train the next version of the model for everyone. If you collect data early and keep improving quickly, that growing data set can make it very hard for competitors to catch up. One important caution, feedback is still personal user data. That means privacy, consent, and transparency all matter, and users should understand how their data will be used before you rely on it. In this section, we'll look at how everyday conversation can become a source of feedback for AI systems. The key idea is that users often reveal what they want, what went wrong, and what they prefer without filling out a form. Explicit feedback is the kind users give on purpose, like a thumbs up, a star rating, or a yes or no question. It's easy to interpret, but it's also familiar and limited because there are only so many ways to ask for it. Implicit feedback is different. We infer it from what users do, not what they click in a feedback form. Because every application has different actions, the useful signals vary widely from one product to another. A conversational interface makes feedback easier to collect because users can correct, nudge, or praise the system in ordinary language. That language can reveal both how well the application performed and what the user actually prefers. Let's make that concrete with a travel assistant. The assistant suggests hotels in Sydney, and the user's reply becomes a clue about what matters most of them. This list gives the user a few different tradeoffs: location, price, and vibe. Notice that once the options are on the table, the next user message can reveal which of those tradeoffs is actually important. A reply like "Book the one close to galleries" points toward an interest in art and culture. A reply like "Anything under $200?" tells you the user is price sensitive and that the assistant may still be missing the mark. >> Once you can read feedback from conversation, you can use it in three ways: to measure the system, improve future models, and tailor the experience to each user. First, evaluation. For evaluation, these signals become metrics you can track over time. That helps you see whether the application is improving or drifting. Second, development. For development, feedback can guide prompt changes, data collection, or model training. In other words, today's conversations can become training signals for tomorrow's model. Third, personalization. For personalization, the same user signals help adapt the assistant to an individual. The system can learn what style, detail level, or topic mix each person tends to prefer. The challenge is that conversational feedback is mixed into normal language, so it's harder to extract reliably. You need intuition to start, but you also need data analysis and user studies to confirm what the signals really mean. This area became more visible with chatbots, but it was already active research before ChatGPT. Natural language feedback has been studied in reinforcement learning and in early voice assistants and conversational product for years. Now, let's zoom in on feedback that comes directly from the content of messages. These are the signals you can detect by reading what the user says and how they say it. Natural language feedback tells you how a conversation is going from the text itself. In production, it's worth tracking because it gives you a continuous view of user experience. One strong signal is early termination. If users stop the generation, leave the app, or abandon the exchange, that often means the conversation is going poorly. When a user cuts off a response or walks away mid-conversation, that is usually a sign of frustration or confusion. It is not proof failure, but it is a meaningful warning signal. Another useful signal is when users actively correct the model. That tells you the system misunderstood something and gives you a chance to learn from the correction. Phrases like "No" or "I meant" are classic signs that the response missed the user's intent. The user is not just continuing the conversation. They are repairing it. Users may also rephrase their request to get the model back on track. That rephrase can be detected with simple rules or with machine learning, and it is a strong clue that the first answer was off target. In the screenshot, the important thing is not just the text itself, but the pattern. The user stops, restates, and tries again. That combination is a strong signal that the system misunderstood the original request. This figure reinforces the same point visually. Early termination plus a rephrased question together make the failure mode much clearer than either signal alone. Sometimes the feedback is very explicit. The user tells the model exactly what should change. That is valuable because the correction can be fed right back into the summary or answer. This is especially common with agentic systems, where users steer the assistant toward useful actions. A small suggestion, like checking a GitHub page or a social profile, can be a big clue about what the agent missed. When users ask for confirmation, sources, or a second check, that can mean the answer is incomplete or unconvincing. It does not always mean the model is wrong, but it may mean trust is low. If users edit the model's response directly, that is a very strong correction signal. It shows exactly which output they want to changed. Those edits can also be turned into preference examples. The original answer becomes the losing response, and the user's edited version becomes the winning one. Not all feedback is corrective. Sometimes users just complain, and those complaints still tell you a lot about how the system is failing. Users may complain that the answer is wrong, irrelevant, toxic, too long, or not detailed enough. Clustering these complaints helps you discover the main categories of failure in real usage. This table shows that many complaints are about clarification, irrelevance, missing evidence, or lack of detail. That kind of breakdown is useful because it turns messy text into actionable categories. Once you know the failure mode, you can fix it more directly. If the problem is verbosity, change the prompt to be shorter. If the problem is lack of detail, ask for more specificity. A related signal is sentiment. Even when users do not explain the problem clearly, their emotion can still reveal how the conversation is going. Negative expressions like frustration or disappointment can indicate trouble, while a shift from anger to satisfaction can show the issue was resolved. In voice systems, they may even appear as changes in tone or volume. You can also infer feedback from the model's own responses. A high refusal rate often correlates with user frustration, especially when a model keeps declining instead of helping. Now, let's look at actions around the conversation, not just the messages inside it. These behaviors also tell you how users feel about the system. Some of the strongest signals are behavioral. Users often reveal their judgment through what they do with the conversation after it ends. A common action is regeneration, asking for another answer. That can mean dissatisfaction, but it can also simply mean the user wants options. Regenerating a response is ambiguous. Sometimes the first answer was not good enough, and sometimes the user just wants alternatives to compare, especially for creative tasks. Cost matters, too. If regeneration uses extra money, users will do it less often. So, you have to interpret that signal in the context of the product's pricing model. For complex requests, multiple generations are useful because they let you check consistency. If two answers disagree, that's a warning sign that neither one is fully trustworthy. Some apps go further and ask the user which response is better. That gives you direct preference data, which is exactly what you want for fine-tuning alignment. The key idea in this image is comparison. After a regeneration, the system asks the user to judge the new answer against the old one. That turns a casual interaction into structured preference data. This is a good example of how the interface can actively collect feedback instead of waiting for it. The system turns a regenerated answer into a direct better or worse choice. Beyond message content, even the way people organize conversations can be informative. Actions like deleting, renaming, sharing, and bookmarking all carry signals. Deleting a conversation often suggests it was unhelpful, though sometimes it's just about privacy. Renaming a conversation usually means the exchange was useful enough to keep, even if the default title was not. Another signal is how long the conversation lasts, but you cannot interpret the same way everywhere because a long session can mean either engagement or inefficiency. In a companion app, many turns might be a good sign because the user wants to keep talking. In customer support, the same pattern may mean the bot is making the user work too hard. Length becomes more informative when you combine it with diversity. If the conversation is long, but the model keeps repeating itself, the user may be stuck in a loop. Distinct topics or tokens help tell the difference between productive back and forth and repetitive failure. A long conversation with little variety is often a sign that the system is not moving forward. To close, remember the trade-off. Explicit feedback is easier to interpret, but it takes effort, so it may be sparse and biased toward unhappy users. Implicit feedback gives you much more data, but it is harder to interpret correctly. The same action can mean different things for different users, so you have to study the people behind the signals. The safest approach is to combine multiple signals and look at them together. When you do that, implicit conversational feedback becomes much more useful for evaluation and improvement. In this section, we move from deciding what feedback to collect to deciding how to collect it well. The key idea is to make feedback easy to give, useful to interpret, and never disruptive to the user's work. If the previous section helped you identify what to ask users, this one shows the timing and design of the request itself. Good feedback only helps if you ask at the right moment and in the right way. We're going to look at the situations where feedback is most useful, and then at the interface patterns that make that feedback reliable. The goal is to collect signals without making the product harder to use. First, let's talk about timing. Feedback can be collected throughout the user journey, but the request should appear only when it fits naturally into the experience. Users should always have a path to report errors or give input when they need to, but that option should stay out of the way, so it supports the workflow instead of interrupting it. A A place to ask for feedback is right at onboarding, when the system is still learning the user's preferences or context. But even there, you want to balance usefulness with friction. Some products need an initial setup step, like face recognition or voice enrollment. For others, it's better to start with a neutral default and learn over time, because forcing too much up front can keep people from trying a product at all. Another important moment is when the system fails. That is often when feedback is most valuable, because the user is telling you exactly where the product broke down. If the model hallucinates, blocks something valid, creates a bad image, or responds too slowly, users should be able to say so immediately. Practical controls like down votes, regeneration, or model switching turn frustration into a useful signal. Even when the model is wrong, the user should still be able to finish the task. The best products let people correct the AI or hand off to a human when collaboration with a model is not enough. In painting is a good example of collaboration. The user marks the part that needs improvement and gives a prompt to fix it. That way, the user gets a better result and the developer gets a very targeted form of feedback. Notice how the image editor focuses on a small region instead of asking the user to start over. That kind of partial correction is exactly what makes feedback feel natural inside the workflow. This figure shows the in painting idea in context. The important point is not the specific UI, but the pattern. Let users improve a result directly where the problem appears. Sometimes the right moment to ask for feedback is when the model is uncertain. In those cases, feedback can help the system choose between plausible options instead of guessing silently. If the system is unsure, it can show two candidate answers and let the user pick the better one. That gives you a comparative signal, which is especially useful for preference tuning, as long as it doesn't slow the user down too much. Here the user is comparing responses side by side. The design matters because it converts uncertainty into a choice, and that choice becomes feedback for future training or evaluation. This figure illustrates comparative evaluation in practice. The key is that the interface makes the user's preference visible without requiring a lot of extra explanation. Showing two full answers can be heavy for users, so many systems reduce the effort by showing only the start of each response. That can improve participation, but it also raises the question of whether the feedback is as reliable. This kind of partial preview lets users compare options quickly. The trade-off is simple, less reading effort versus less context for the decision. The point of this design is to keep the comparison lightweight. Users can expand what looks promising. So, the feedback starts with a quick signal instead of a long reading task. Another useful pattern is when a system asks the user to confirm whether two items are the same. That works well in things like photo organization, where the system can learn from the user's answer. This image shows a matching or confirmation task. The system is essentially saying, "I'm not sure." and inviting the user to resolve the ambiguity. Here, the feedback happens only when the model needs help. That keeps the interaction focused and makes the user's answer more informative. Now, let's switch to positive signals. Things like likes, favorites, and shares can be useful, but many interface guidelines warn against forcing users to give both positive and negative feedback all the time. Some teams still want positive feedback because it shows which features people love enough to praise. That can help focus product work on the highest impact parts of the experience. If you do ask for it, control how often the request appears. Sampling a small fraction of users can reduce annoyance, but also makes you more aware of potential bias in the feedback you collect. Once you know when to ask, the next question is how to ask. Good collection methods fit into the workflow, feel easy, and produce signals you can actually use. The best feedback systems are almost invisible. Users should be able to respond with little effort, and they should not feel forced to stop what they're doing just to help the product learn. Midjourney is often cited because its interface naturally turns user actions into feedback. The options are built into the generation flow, so the user is already making a choice instead of filling out a separate survey. First, there's an option to upscale one of the images. That tells the system which result looks most promising to the user. >> Upscaling is a strong positive signal. It says the user found one image good enough to refine rather than discard. The next option is to create variations, which is still positive but less decisive than choosing a single winner. Variations mean the user likes the direction, even if none of the images is perfect yet. That gives the model a softer signal about what to keep exploring. Then there is the option to regenerate everything. That usually means the current set missed the mark. Although sometimes users regenerate just to explore. Regeneration is useful because it signals dissatisfaction, but it is not a perfect rejection signal. Users may click it out of curiosity, not only because the images are bad. The important lesson is that each interaction carries information. Some actions are strong endorsements, some are weaker, and some only suggest that the user wants another attempt. >> This interface is designed so that every action teaches the system something. The feedback is not a separate step. It is part of a normal creative loop. This figure shows what embedded feedback looks like in practice. The system learns from what users choose, not just from what they explicitly rate. Code assistance often makes suggestions visually distinct from type text. So, accepting or ignoring them becomes a form of feedback. Even a key press can tell the system a lot about whether the suggestion was useful. Notice the lighter suggestion styling. That visual difference helps users understand what is tentative and what is already part of their work. This is a strong feedback loop because the user's action is built into the task itself. Accepting and ignoring are both informative. Stand-alone chat apps often miss this kind of rich signal because they sit outside the user's daily workflow. Integrated products, by contrast, can observe how a suggestion is actually used, edited, or sent. A thumbs up or thumbs down tells you something, but context tells you why. If you want deeper analysis, you usually need surrounding conversation history, not just the final rating. That extra context may require explicit permission, especially if it includes personal data. Some products solve this with a data donation flow, where users agree to share recent interaction history along with the feedback. People give better feedback when they understand the purpose. Be clear about whether the data is for personalization, analytics, or model training, and reassure users about privacy only when those assurances are true. Feedback requests must match what the user actually knows. If a system asks for a preference where there is no meaningful preference, the signal becomes noise. This example shows a poor feedback prompt for a factual or mathematical problem. The user may not have enough information to choose, so the interface should offer a way to say that directly. For questions with a right answer, preference is the wrong framing. The interface should support uncertainty, not force the user to pretend they know more than they do. Small design details matter a lot here. If the controls are ambiguous, users will misread them, and the feedback you collect will be misleading rather than useful. You also have to decide whether feedback is visible to others. Public signals can improve discovery, but private signals often make people more honest and more willing to participate. This kind of interface can make a user's choice visible to others. That visibility changes behavior, so privacy is part of the feedback design, not an afterthought. Here the problem is not the idea of feedback, but the presentation. If the symbols do not match the scale, users will accidentally give the wrong signal. Private feedback often improves honesty because people feel less judged, but hiding signals can also make systems harder to explain, especially when those signals affect recommendations or discovery. So, the design choice is always a trade-off. More privacy can mean better feedback, but less visibility can make the product harder to understand. Good feedback design finds the balance that matches the product and the users' expectations. In this section, we'll look at why user feedback is useful but not always trustworthy. The main theme is simple. Feedback can be biased, incomplete, and even self-reinforcing. User feedback is valuable, but is not free of cost or distortion. You have to treat it as evidence, not as perfect truth. >> Let's start with bias because it appears in almost every feedback system. If you do not account for it, the data can push you toward the wrong conclusions. Feedback has the same problem as any other data source. It reflects the people, the interface, and the context in which it was collected. So, the goal is not to assume it is clean, but to design around its known distortions. The first example is leniency bias. This is when people rate something more positively than they really feel, often because it is easier or feels more polite. If giving negative feedback creates extra work, many users will avoid it and just choose the easiest positive option. That means the feedback system is capturing convenience, not just opinion. On a five-star scale, users may treat five stars as the only truly positive rating and use four stars only when something is wrong. That compresses the data and makes the scale harder to interpret. >> That bias does not make the system useless, but it does mean you should inspect the rating distribution carefully. If almost everything is at the top, the scale may not be measuring what you think it is. One way to reduce this problem is to replace abstract numbers with labels that feel more natural. When users can respond in plain language, they often give more accurate feedback. Here begins a set of example responses that are easier for users to choose from than a numeric rating scale. This option captures a strongly positive experience, is more specific than a star rating, and easier to interpret. This continues with progressively less enthusiastic feedback, which helps users choose a description instead of forcing a number. This one sits in the middle, acceptable but not exceptional. That middle ground is often hard to express with stars alone. >> This option gives users a neutral category, which is useful when their experience was fine, but not memorable. This label is subtle. It says the experience was okay without pretending it was great. That kind of wording can produce more honest feedback. Another step down in the response scale helps separate mild dissatisfaction from serious problems. This expresses disappointment without being extreme. It gives the system a clearer signal that something fell short. And finally, the strongest option makes the negative case unambiguous, so the system can act on it. This is the most severe response in the list. It tells you not just that the experience was bad, but that the user wants the system to avoid this choice in the future. Another limitation is that feedback can be random. Not every choice reflects careful judgment. Sometimes users simply click because they are tired or uninterested. When a task is tedious, people often do not fully compare options. In that case, the feedback you collect may be more about user effort than model quality. Position bias is next. The order in which you show options can strongly affect which one gets elected, even when the options are equally good. People tend to click the first suggestion more often than the second. So, a click does not automatically mean the first item is better. You can reduce this bias by randomizing the order of the options. Another approach is to model the effect of position, so you can estimate the true success rate more accurately. Now, we move to preference bias, where people favor one answer for reasons that are easy to notice, but not necessarily relevant to quality. This is especially common in side-by-side comparisons. A longer answer may seem stronger even when it is less accurate simply because length is obvious. Recency bias can also appear where people favor the answer they saw last. The key lesson is to examine the feedback itself. Not just the scores it produces. If you understand the bias, you can avoid making decisions that look data-driven, but are actually misleading. The next limitation is more serious. Feedback can change the system in ways that reinforce its own mistakes. That is where degenerate feedback loops come in. Remember that feedback is always incomplete. Users can only react to the items you expose to them. So the data reflects your system's previous choices. In a feedback loop, the model's predictions affect what users see and that exposure affects the feedback they give. Then that feedback shapes the next version of the model, which can amplify the original bias. A recommendation system is a classic example. If one item is shown first, it gets more clicks. Those clicks strengthen the ranking, and the cycle keeps repeating until the early advantage becomes a big gap. This same mechanism can steer a product toward a narrow outcome. A small amount of early feedback can snowball into a system that overproduces one type of content while ignoring everything else. There's also a risk that the model learns to say what users want to hear rather than what is most accurate. That is why training on feedback must be done carefully. It can improve usefulness, but it can also reward agreement over truth. The bottom line is that user feedback is essential, but must be interpreted with caution. If you understand its limitations, you can use it to improve the product without letting it distort the product's direction. Let's wrap up this chapter by connecting the main ideas. We looked at AI applications as systems built on foundation models, and the key lesson is that the model is only one part of the full product. Each earlier chapter focused on one piece of AI engineering, but here we zoomed out to the whole application. The goal was to understand how the parts work together, not just how to use a model in isolation. We started with a high-level architecture as a shared mental model, even when real systems differ. That structure helps you see where the main challenges appear and which techniques solve them. The boundaries between components are useful, but they are not rigid. For example, guardrails can live in several places, and the right choice depends on the system design and the tradeoffs you want. Adding components can improve speed, safety, or capability, but also introduces new failure modes. That is why observability matters. You need to see what failed, detect it quickly, and trace it back to the source. A chat interface gives you new kinds of feedback from users, and that feedback can drive analytics and product improvement. We also looked at how to design the application so that this feedback is actually collected in useful ways. Feedback collection is no longer just a product detail because it directly affects model improvement. That is one reason AI engineering is moving closer to product. User experience and data flywheels are becoming competitive advantages. The broader message is that many AI problems are system problems. Once you step back and look at the whole system, you can find better solutions, combine components more effectively, and build safer applications. This kind of risk is real, not abstract. A user can accidentally expose sensitive company information to a model, which is exactly why guardrails and monitoring are not optional. Sometimes the system has to handle unexpected but valid user behavior, like asking for no output at all. Even small edge cases like this matter when you're designing robust applications. This note captures a common tension. Optimizing one part of the system can make another part weaker. If you cut guardrails too aggressively for latency, you may end up with a faster system that is much less safe. The scale of observability companies shows how important monitoring has become in modern systems. For AI applications, the need is even stronger because foundation models introduce new kinds of failure to measure and trace. This also connects back to earlier work on machine learning systems. The same monitoring ideas apply here, but AI applications need extra attention because model behavior can shift in ways that are harder to predict. As systems grow, orchestration tools often try to expand into gateways or full platforms. That trend reflects how closely related the layers are in a real AI stack. If you ship an open-source application, you may lose direct visibility into how people use it. That makes feedback collection much harder, even though the product may spread more easily. Feedback is not just something AI applications receive. It is also something AI can help interpret. That opens the door to faster analysis and better product decisions at scale. This example shows a practical product idea. Let users fix only the bad part instead of regenerating everything. That kind of targeted editing can make AI tools feel much more controllable and useful. Even small interface choices affect the quality of feedback you get. Showing full responses may help users judge better, but can also reduce the incentive to compare alternatives carefully. Real-world feedback systems are imperfect, and even reputation could be affected by accidental actions. That is another reason you need careful design when collecting and interpreting user signals. The final note is a reminder to treat these ideas as design patterns, not finished answers. In AI engineering, the right choice still depends on your users, your system, and your constraints.