Why we watch the work before we automate it
Tacit knowledge lives in how experts work, not in the process manual.
A process manual might say: read the request, check the account, choose the right response, update the record.
An experienced operator might read the request, notice that the sender is using an old company name, check a previous case, ask a colleague whether an account migration has finished, and delay the update until two records agree.
Both descriptions concern the same job. Only one gives an automation team enough information to build something dependable.
That gap is why observation belongs at the beginning of an AI project. Before deciding what a model should do, we need to understand what makes the work succeed: which signals matter, which exceptions change the decision, and where someone has to take responsibility.
The missing steps are often the important ones#
People compress familiar work when they describe it. “I check the account” can hide several decisions: which system to trust, how recent the information must be, whether an apparent match is sufficient, and when a discrepancy needs another person.
This is a practical reason to combine interviews with observation. Watch people work in their own environment, then ask about the decisions behind what you see. An interview may capture the steps someone remembers; following a real case gives you concrete decisions to examine together.
A manual still matters. It expresses intended policy and provides a useful starting point. Observation helps distinguish three things that are easy to confuse: the official procedure, the workaround people use, and the rule the future system should follow.
Those things will not always agree. A shortcut may be sensible, outdated, or unacceptable. The team should investigate the difference with the process owner rather than encode every observed action as a requirement.
Follow a real case from arrival to resolution#
Start with a bounded question. “Understand customer operations” is too broad. “Understand how a request to change an account contact reaches an approved update” gives the session a beginning, an end, and a decision to inspect.
Choose participants who see different parts of that journey. Include the person doing the routine work, someone who handles escalations, and the owner who can clarify policy. Watching only the fastest expert can hide the support that less experienced colleagues need.
Ask participants to use their usual tools. Watch the movement between the inbox, the record system, the internal chat, and any notes they maintain. Follow the handoff as well as the screen. A request may look finished in one queue while another team is still correcting its downstream effects.
Use familiar equipment and ordinary working conditions. Asking someone to explain every movement can interrupt the very behavior you are trying to understand. Let a short sequence unfold, then return to the decision points while they are still fresh.
Explain the purpose of the session and agree what may be captured. Prefer focused notes and redacted examples when full recordings add little value. The work should be understood with participants, and the resulting account should be something they can correct.
Ask about the evidence behind each decision#
“What did you click?” produces a sequence. “What made that safe to do?” produces a specification.
Useful follow-up questions include:
- What told you this was the right record?
- Which source would you trust if these two fields disagreed?
- What would make you stop and ask someone else?
- What information did you expect to find but could not?
- How would a new colleague know whether this was finished correctly?
Pay attention to pauses. A pause may reflect a slow application, an ambiguous request, or an important judgment. Those causes require different remedies. Faster text generation will not solve a missing approval, and an additional integration will not resolve an unclear policy.
Capture more than actions on a screen. Record the information retrieved, the person consulted, the decision made, and the condition that would have changed it. A sequence of clicks can describe a successful case while leaving the next difficult case completely unspecified.
Turn the observation into a decision record#
Consider an illustrative account-contact change. The request is brief, the requested edit is simple, and a model can extract the new name easily. The hard part is deciding whether the edit is authorized.
An observation record might look like this:
| Observed situation | Expert’s reasoning | Requirement to validate with the owner |
|---|---|---|
| The sender uses a familiar company name | The account may have been renamed | Resolve the canonical account identifier before proposing an edit |
| The request names a new contact | A plausible request does not establish authority | Check the approved authorization route independently of the request text |
| Two systems show different contacts | One may be out of date | Define the authoritative source and route unresolved conflicts to a person |
| A colleague confirms a pending migration | A temporary condition changes the next step | Represent the migration state explicitly rather than relying on informal memory |
| The update appears successful | A saved screen is not proof that every dependent system changed | Verify the stored result and surface incomplete propagation |
This is a hypothetical example, not a NexVega client case. Its purpose is to show how a few minutes of ordinary work can change the engineering brief.
The output of discovery should include a workflow map, a decision table, an exception list, and a named owner for unresolved questions. Record the source of each proposed rule. “Observed twice” and “confirmed policy” are different kinds of evidence.
Separate model work from business authority#
Once the decision record is clear, assign each step to an appropriate mechanism.
A model might interpret an unstructured request, extract candidate fields, or draft a clarification. Deterministic logic can check required fields, compare identifiers, and enforce allowed transitions. A person may need to resolve conflicting authority or approve an action with material consequences.
This allocation is a design decision, not a contest over how much of the process can be called AI. The useful question is whether the whole workflow produces the right outcome and exposes uncertainty at the right moment.
Write down the intended use, the conditions in which the assistant will operate, and the decisions that remain with a person. Observation should make those boundaries concrete: who approves the change, which evidence they need, and what happens if that evidence is missing.
If an expert cannot explain a decision immediately, capture the case for follow-up. Do not invent a rule to fill the gap. Bring similar examples to the policy owner, identify the unresolved judgment, and keep that branch under review until the boundary is clear.
Use exceptions to build the evaluation set#
Routine cases show the main path. Exceptions show whether the proposed system understands its limits.
Seek examples of missing information, conflicting records, duplicates, handoffs, and requests that should be refused or deferred. Label each with the expected action, the supporting evidence, and the reason to escalate. Include cases where the correct answer is to ask a question or leave the record unchanged.
Keep a set of cases out of prompt development so the team can evaluate a later version on examples it has not been tuned against. Remove unnecessary identifying information and preserve the distinctions that make the decision difficult. Discovery examples should improve the test set without making the test a rehearsal of the development material.
Then test the full sequence. Did the system choose the right record? Did it preserve the approval boundary? Did the saved result match the approved change? A fluent explanation is useful only if the resulting action is correct.
Check whether the new workflow reduces the total effort#
Automation can make one step faster while creating extra review and cleanup elsewhere. Measure active handling time, waiting time, rework, escalations, and reviewer effort separately. Compare cases of similar difficulty.
For an illustrative measure, track completed cases that meet the agreed quality standard per hour of combined operator and reviewer time. Pair that with error severity and the rate of incorrect changes. A gain in throughput should not conceal a transfer of work to another queue.
Evaluate the job that remains after automation. If routine cases move through automatically, reviewers may spend more of their day on difficult exceptions. Check whether they have enough context, time, and authority to resolve those cases, including during demand spikes or system outages.
Observation can move into implementation when the team has a shared account of the routine path, important exceptions, sources of authority, and conditions for stopping. It should continue after release through focused reviews of new failure patterns.
The first deliverable of automation is a clear account of the work. A team that understands why an expert hesitates, which evidence they trust, and how they recover from a bad input has the beginnings of a reliable system. Watching closely is how those requirements become visible.
