Ep 1169 Blog 6:52 w/ Pippa & Tyler

Goal Engineering

Harrison Chase and the Actively AI founders argue that the real design problem for AI agents isn’t prompts or tools, it’s how you define their goals so humans can easily review and teach them. Their Actively case study shows that scoping agents around repeatable “motions” like outbounding, instead of around individual sales accounts, makes feedback reusable and learning faster. Pippa and Tyler unpack that shift, why it matters for anyone building agents that touch subjective work, and where the “goal engineering” frame might overreach.

Embed this episode

Paste this on any site — the player is a self-contained iframe with no cookies or trackers.

<iframe src="https://sandrise.io/exploring-next/embed/1169"
  width="100%" height="180" style="max-width:640px;border:0;border-radius:12px;overflow:hidden"
  title="Exploring Next — Episode 1169 audio player"
  loading="lazy" allow="autoplay" referrerpolicy="strict-origin-when-cross-origin"></iframe>
Embed & API docs →
Script GPT-5.1 Voice Inworld TTS 2

Transcript

Pippa So the punchline here is basically: stop engineering prompts, start engineering what the agent’s job is.

Tyler Yeah. And more specifically, engineer the job so that a human can actually teach the thing without losing their mind.

Pippa Right. The article calls it “goal engineering,” and the bit that stuck with me is this one line: design an agent’s scope to make it as easy as possible to give feedback while it’s operating.

Tyler Which sounds obvious until you notice most of us scope agents around whatever data object we already have. Accounts, tickets, repos. Their Actively AI case study is basically, “we tried that, it hurt, we moved.”

Pippa Yeah, let’s walk through that, because it’s a nice concrete story. Actively builds agents for sales teams. Version one: they had one agent per account. The goal was, quote, move this account forward. It would propose actions and the rep would approve or tweak them.

Tyler On paper that sounds fine. It’s aligned to a clear outcome, you can imagine success metrics, and the account object already has a ton of structured context sitting in the CRM.

Pippa Exactly. That’s the intuitive design: this account is a thing, so give it an agent. But what they found was, the really important context wasn’t just facts about the account, it was how to do sales-y things: how aggressive to be, what good outbound looks like, how to navigate a big org.

Tyler And that’s the kind of context that isn’t written down anywhere. It lives in the seller’s head. The article calls that out pretty clearly: documents and A P Is are easy; subjective taste and strategy are the hard part.

Pippa Yeah.

Tyler Technically, that makes sense. If you think in reinforcement learning terms, you’ve got a sparse, noisy reward signal — the rep’s corrections — and you want as many samples as possible that share structure. With one agent per account, each one is seeing a trickle of unique situations, so your feedback doesn’t generalize well and it’s annoying to provide.

Pippa And the UX was bad. They describe two specific problems: first, reps had to context-switch between a bunch of per-account agents, each with only a couple things to review at any moment. Second, there was no natural way to share a learning like, “please stop being this aggressive in outbound” across all those little agents.

Tyler So they flipped the axis. Version two: instead of account agents, they introduce these “motions.” A motion is a particular job, like outbounding or early-stage deal progression, and it runs across many accounts.

Pippa That’s the part I really like. Because a motion is just a stream of similar tasks, a salesperson can sit down, review a batch of outbound steps in one go, and their feedback obviously transfers. If they tweak tone on one email, the agent for that motion can apply it everywhere.

Tyler You basically align the agent’s goal with a human’s review session. One review screen, one type of work, lots of shared structure. That’s much closer to how humans actually want to supervise systems.

Pippa And they don’t throw away account context. They split it. The article says they now have context-gathering agents running in the background to assemble account info, while the motion agents handle progressing deals and pull in that context when needed.

Tyler Which is a nice architectural separation: background scrapers keep the world model fresh, foreground “motion” agents own the policy for how to act, and the human sits in the loop on that second part.

Pippa This also lines up with that thing we keep coming back to: the bottleneck is human oversight. They literally say the performance bottleneck is how fast agents can learn from feedback, so you should optimize the review U I, not just the model.

Tyler Yeah. And I buy that technically. If you look at why agents fail, it’s often missing or mis-weighted context, not raw generation quality. And if the missing context is subjective — brand voice, risk tolerance — the only scalable way to fix it is lots of cheap human corrections that the system can actually reuse.

Pippa So who do you think should actually care about this framing? Because it’s easy to hear “goal engineering” and go, cool new buzzword, back to my backlog.

Tyler Anyone whose agent is touching work that’s not trivially verifiable. If you’re just reconciling invoices, you can probably stick with object-centric goals and automated checks. But if you’re doing marketing copy, sales outreach, support replies with brand nuance — places where taste matters — this is a real design constraint.

Pippa Yeah, if your team is already fighting about tone in Slack, you’re in this category.

Tyler Pretty much.

Pippa From the product side, the practical move here feels pretty concrete: when you’re defining an agent, ask, what’s the smallest slice of work where a human would happily review ten of these in a row? That slice is a candidate for a motion.

Tyler And then make sure there’s a way to share that feedback across instances of the motion. Otherwise you’ve just reinvented the per-account trap with different labels.

Pippa Do you buy their stronger claim at the end, by the way, where they say prompt engineering and context engineering will fade, but goal engineering will stick around?

Tyler I think they’re overselling the disappearance part. Models might need less hand-tuned prompting over time, sure, but people will still spend real effort deciding what goes into context windows and how tools are exposed. That’s all still design. Where I do agree is that those become implementation details. The shape of the goal — what you want the agent to optimize and how humans grade it — is the durable abstraction.

Pippa Yeah, I read that more as a vibe than a literal timeline. But it’s a useful vibe. It nudges you to think in terms of goals plus review loops instead of clever one-off prompts.

Tyler And I like that their evidence isn’t just vibes. The Actively story has real friction: context switching between account agents, awkward knowledge transfer, then a redesign to motions built on Deep Agents that fixes those pain points. It’s not a randomized trial, but it’s more than a hand-wave.

Pippa It also matches what we’ve seen in other domains. Like when we talked about GitHub formalizing “skills” as reusable chunks instead of bespoke prompts per agent. Motions feel like the sales-side version of that — reuse the learning where it naturally clusters.

Tyler Yeah, skills for actions, motions for goals. Same pattern: carve the problem where supervision and reuse are easiest, not where your database schema happens to be.

Pippa So if you’re sitting there with an “account agent” or a “repo agent” that feels weirdly dumb, this might be why. The object is right, but the goal boundary is wrong.

Tyler Try rotating the axis to a motion: refactors instead of repos, outbound instead of accounts, escalations instead of tickets. Then build the review U I around that. If it suddenly feels way easier to give feedback, you probably just did some real goal engineering.

Pippa Alright, I’m going to go stare at half a dozen product specs through this lens now. Thanks for the brain-tilt, Tyler.