AI training that sticks: teach the real task, not the tool

Most AI training fails the same way: a vendor demos a tool, everyone nods, and three weeks later nobody uses it. The reason is that the training taught the tool, not the job. This post is about the method we use instead, where a team brings their own real, annoying task and leaves having automated a piece of it that day, with something they will actually run again on Monday.

What you'll learn

• Why feature-tour training does not survive contact with real work
• The "bring your worst task" method and how we run a session
• The four-step loop we teach instead of a tool
• How we measure whether training actually stuck
• What a team that can do this themselves makes possible

The old way: a demo, a deck, and decay

The standard AI training is a presentation. Here is the tool, here are its buttons, here are some impressive examples that were built by the person presenting, on a clean problem, with unlimited time. The team watches. The team is impressed. The team goes back to their real work, which looks nothing like the demo, and the new tool quietly dies on a tab nobody opens.

This is not a discipline problem. It is a transfer problem. People do not learn a skill by watching someone else perform it on an unfamiliar example. They learn it by doing their own task, badly, with help, until it works. A demo skips the only step that teaches anything.

The other failure is teaching the tool instead of the judgment. Tools change every quarter. The model you trained on this spring will be deprecated by autumn. If the training was "click here," it expires with the interface. If the training was "here is how to think about handing work to a model," it survives the next ten interfaces.

The new way: bring your worst task

We run training around a rule. Every person shows up with one real task they hate, something repetitive, time-eating, and undeniably part of their actual week. Not a hypothetical. The thing on their plate that makes them sigh.

The session is not us showing our work. It is us standing next to them while they automate a slice of their own. By the end of the day, each person has a working thing that touches their real workflow. That is the only outcome that survives the drive home.

The session shape

• **Morning: name the task precisely.** Most people cannot describe their own repetitive work in steps. So we make them write it out, step by step, in plain language. Half the room discovers the task is fuzzier than they thought, which is exactly why a model could not help them before.
• **Midday: hand one step to a model.** Not the whole task. One step. The boring extraction, the first-draft summary, the format conversion. We use whatever is in their stack, often Claude for the language work, and we connect it to their real data, not a sample.
• **Afternoon: build the loop and break it.** We deliberately feed it a messy input and watch it fail, because knowing how the thing fails is more useful than knowing how it succeeds.

The four-step loop we teach

We do not teach a tool. We teach a loop that works with any tool, because the tools will keep changing and the loop will not.

1. **Describe the task in plain words**, as if to a sharp new hire who knows nothing about your company. If you cannot, the model cannot help, and that is information.
2. **Hand off one step, keep the judgment.** Give the model the mechanical part. Keep the decision about whether the output is right. Never hand off the judgment in the first month.
3. **Check the output against a real standard.** Have a known-good example to compare against. "Looks fine" is not a check.
4. **Write down what worked so it repeats.** A saved prompt, a short note, a template. Otherwise the win happened once and lives in nobody's hands.

This loop is deliberately boring. Boring is the point. A team that internalizes a boring, repeatable loop will outlast every team that learned a flashy tool, because the loop is the actual skill.

The reason step two is so strict, hand off one step and keep the judgment, is that this is where most people hurt themselves. They hand the model the whole task and the decision about whether it was done right, then discover months later it has been quietly producing wrong output that nobody checked. We teach the opposite habit early: the model is a fast junior who needs its work reviewed, not a replacement for the reviewer. Keep the judgment in your hands until you have watched the thing succeed and fail enough times to know its specific blind spots. Only then, maybe, automate a check. Never the judgment itself in the first month.

A worked example from a real session

One person on our team spent every Monday morning turning a week of raw call notes into a tidy summary for the rest of us. Forty minutes of copy, paste, and reformat. Dull, necessary, hers alone.

In a session we did not build her a fancy summarizer. We made her write the task out in steps first, and she discovered her own process was inconsistent, she summarized differently depending on her mood that morning. That inconsistency was why no tool had helped her before. There was no stable task to hand off.

So step one was making the task stable: a fixed shape for the summary, three sections, same every week. Then we handed exactly one step to a model, the first-draft compression of the raw notes into those three sections. She kept every judgment: what to cut, what to flag, what was wrong.

Then we broke it on purpose. We fed it a week of notes with a half-finished sentence and a contradiction, and watched it confidently smooth over both. That failure was the most useful five minutes of the day, because now she knew the one thing she could never trust it to catch, and she checks for it every Monday.

Two weeks later she was still running it. Monday morning went from forty minutes to about twelve, and the twelve are the judgment minutes, which is the right twelve to keep. We did not teach her a summarizer. We taught her the loop, on her own worst task, and the loop is what she still has.

How we know it stuck

Most training is measured by a satisfaction survey, which measures whether the lunch was good. We measure differently, on a two-week delay.

We ask one question: are you still running the thing you built? Not "did you like it." Are you running it. If the answer is yes, the training worked. If the answer is no, we find out which of the four steps broke and fix the method, not the team.

When we taught this loop inside Temerarii Media, the things that stuck were never the impressive ones. They were the small, ugly automations a person built for a task only they cared about. Those got run every week, because the person had skin in it. The big shared "everyone should use this" tools were the ones that died. So we stopped building those in training and started insisting everyone bring their own worst task.

What this makes possible

A team that can run the loop themselves stops needing you for the next problem. That is the goal, and it is the opposite of how most training is priced, which is why most training is designed to create dependence.

Once a team owns the loop, three things happen. The backlog of "we should automate that someday" tasks starts getting cleared by the people who actually feel the pain. The team develops taste about what should and should not be handed to a model, which is a skill no demo can transfer. And when the tools change, which they will, the team upgrades without retraining, because they learned the thinking, not the buttons.

The measure of good AI training is not what the room can do at 5pm on training day. It is what they are still doing, unprompted, two weeks later. Teach the task and the loop, let the tools come and go, and the skill stays.

We run this method on our own team first, then with the people we work with. If you want to see the loop applied to real, public work, the experiments at [temerarii.xyz](https://temerarii.xyz) show it running, and the rest of the [blog](/blog/) takes the pieces apart one at a time.

See the live system →

Written by Dominique Davis, Founder, Temerarii Media.