Rebuilding research from the ground up

Welcome to Season 2 of Think First: Perspectives on research with AI. In Season 1, we heard from researchers across the industry about how AI was reshaping their craft. This season, we go a step further diving into what changes when researchers move from bolting AI onto their existing process and start re-architecting the whole thing to be AI-native.

Most researchers still use AI for narrow tasks: paste in a transcript, get a summary, move on. The four researchers in this episode have gone further, rebuilding how they work from the ground up.

Ben Hanrahan, a principal research engineer and UX researcher on Microsoft’s Core AI team, has turned report writing into a continuous, agent-driven pipeline and started shipping his own tools. Chris Gunderson, a UX researcher who studies security admins and analysts, has made deliberate context assembly the first step of every analysis. Sara Chizari, a Senior UX Researcher on the same Core AI team, has reimagined what happens after a study ends – building agents and interactive artifacts that let stakeholders explore the evidence themselves. And Bruce Philips, a Lead Principal UX Researcher on the Fabric team, has built a telemetry tool that turns AI loose on the product to interpret user behavior, and running his workdays through an AI “chief of staff.”

They disagree on plenty – including whether you are “falling behind” if you haven’t rebuilt your workflow yet. But they converge on one idea: the unit of work is no longer the prompt. It’s the system – and, as Bruce puts it, “the context that system carries.”

The moment the old workflow broke

Each of them can point to a moment when the one-shot habit – paste a transcript, get a summary – stopped being enough.

For Ben, it was a capability jump. When a markedly more capable model arrived, he realized it could suddenly do things he had been doing by hand across the entire research process – designing studies more quickly, surfacing questions he might have missed, analyzing what he heard from participants, and pulling in external sources. That pushed him to build “a whole new pipeline for myself, with checks and guardrails.” It also let him lean on his background as an engineer. Instead of running a study for a week and then proposing directions that might get built weeks later, he could prototype changes directly and “iterate three times a week, as opposed to three times a month or two.” The shift, he says, changed “not only the pace of what I could do as a UX researcher, but also expanded the capabilities of what I can do as a UX researcher.”

“Thinking about AI in terms of one transcript or one database at a time,” Chris says, “at some point wasn’t going to scale, because the main realization is that we need more context.” For him the breaking point was scale, not speed. Work one source at a time and you only get that one source’s analysis – but studies carry more: the research brief, the project notes, the differences between participant segments. His example is a pool split between power users and new users. Without that context in the analysis, you end up “flattening things out when they really shouldn’t be.”

“The main realization is that we need more context.”

— Chris Gunderson

“No matter how good [the] prompt was in my interaction with the agent, the output didn’t feel consistent,” Sara says. For her, the breaking point was inconsistency. She had started where most people do, using AI to summarize a transcript and do a little analysis here and there, but that nagging unreliability told her something was missing. It moved her from prompting toward building context – guidelines, do’s and don’ts, “really building a system around my interaction with AI.” The payoff was control: “I have a way to sort of control the behavior of the AI… I’m seeing inconsistent results and turning it into a system that gives me more consistency.”

“We all started by putting maybe a transcript into a prompt,” Bruce says – but for him the shift came from realizing the transcript was the least of it. Now he maintains far more: not only the transcript but “why we’re running the study, what we expect to get out of it, our stakeholders’ priorities, what we know about the product and the users.” When all of that background contextualizes the transcript, it “makes the output immensely more relevant to the research question.” The differentiator wasn’t rounds of prompt iteration, it was a richer, persistent context.

Four different signals – a better model, a scaling wall, an inconsistency problem, and the discovery that context matters more than the prompt – led to the same conclusion.

What AI-native ways of working look like

If the shared realization has been “build a system,” the systems themselves look strikingly different, each shaped by the problem its owner cares about most.

Ben rebuilt reporting into something continuous. Report writing that once took a week and a half now takes one to three days – and he no longer thinks in terms of a single report at all. Instead, he maintains what he calls an LLM wiki, “constantly tweaking and adjusting that as I use it.” A chain of agents runs on a daily loop: it grabs the transcript from Teams, processes the recording through the team’s video tooling, integrates the results into the wiki, and opens a pull request. “Then I’ll review the pull request to say, oh, it missed this, it missed that,” he says, “before I merge.” He has also become a builder of tools, making “half a dozen different sorts of tools” in recent months – including a command-line tester he named Clint that takes screenshots and walks through scenarios, and which the product team adopted as one of its main testing tools. Once you know how something should work and what the architecture should be, he notes, “you can vibe-code a lot of the tool.” He frames all of this as a new baseline for the role: “the expectation now is everybody’s building tools” to shorten the distance “between starting a study and a merged PR… and tooling is a big part of how you can spread best practices, because all of this is incredibly new and we’re all making it up as we go along.”

“The expectation now is everybody’s building tools.”

— Ben Hanrahan

Chris pointed his system upstream – at the quality of what reaches participants, and at the context that reaches the model. He now pulls inputs from across content design, product design, and research for two purposes. The first is to raise the quality bar before a prototype is ever tested: manual work that used to mean heuristic evaluations and checks against content and design guidelines “can be offloaded to these AI workflows,” so the prototype starts “with a much higher quality bar before we even put [it] in front of the user.” The second is to make insights reusable at scale – recurring pain points, jobs to be done, user roles and context – “packaged at the product level” as markdown files that are “both human readable but also AI readable.” To decide what context belongs, he uses a simple test: “I imagine another researcher that I’m handing this project off to… would this be useful to that new researcher to do quality analysis? And if not, I don’t include it.” Stakeholder emails full of opinion and back-and-forth get cut; a useful timeline detail might survive. Combine the sources deliberately before AI touches them, he says, and “you’re going to get much, much better output.”

Sara aimed her system at the other end of the process – what happens after the report is “done.” She kept running into researchers who would produce a strong report and expect stakeholders to translate it into their own context, but a designer, a PM, and an engineer each need something different. So instead of another static summary, she built an agent grounded in her research that people can question in the language of their own role – a PM asking which opportunities are most consequential to invest in, a designer asking about unmet needs, an engineer asking about constraints. The point, she stresses, “wasn’t simply changing the tone for different audiences – it was changing the path for them through the evidence while keeping everyone connected to the same research.” And it was not about replacing her: “I didn’t want the agent to replace the researcher’s interpretation… I wanted to make sure my interpretation is there.” She has applied the same idea to artifacts, turning a jobs-to-be-done list that was “too long for anyone to even want to engage with it” into a lightweight, filterable website. The underlying data stayed the same; “what changed was the distance between the evidence and the stakeholder questions.” Beneath it all is a rule about sequence – she forms her own interpretation first, then brings AI in to challenge it: “here is my understanding, this is my framing – now go find contradictory evidence, find anti-patterns.” The distinction she keeps returning to is “between outsourcing judgment and extending the judgment. And I’m… more interested in extending the judgment.”

Bruce built his system to make sense of raw product behavior. He created a tool that queries https://learn.microsoft.com/en-us/kusto/query/?view=microsoft-fabric to pull sequences of user actions and surface recurring patterns – but, he says, getting AI to build the tool and do the math was the easy part. “The difficult part is having the AI make sense of the sequences.” His old method was time intensive: read a telemetry string, then manually reproduce what the user must have been doing in the product to decode it. His fix was to hand that work to the AI itself. Using Playwright, a browsing tool some models can drive, he turned the AI loose on the product to “explore all the links and connections between the different features,” screenshot every page, and join it with jobs-to-be-done data – until “the AI itself built out the topology of our product.” With that map in place, the AI can now take a string of telemetry events and predict what a user was trying to do. He applies the same context-first instinct to his own days, having built “a chief of staff of sorts” – an AI he meets for thirty minutes on a standing calendar hold each morning, which “catches me up on what I missed and what I need to do.” It isn’t good out of the box, he notes: “it’s good once you’ve taught it what’s important to you.”

“The prompt is the front door, and context engineering is the architecture of the whole entire house.”

— Sara Chizari

Prompt engineering isn’t dead, but it has grown up

We offered the group a premise: : prompt engineering was just the warm-up act, and context engineering is the real skill. None of them fully bought it..

Chris rejected the hierarchy outright, calling the two “both really important parts of the equation.” He reached for a kitchen analogy: the prompt is the recipe, the context is the ingredients, and you still need to know how hot the pan gets and how long to cook. “Good prompt engineering and good context engineering equals good outputs.” His prompt discipline is specific – he now asks for verbatim quotes with timestamps and no alterations, because without that “you sometimes will actually get paraphrases of what participants said… that sounds right, but it’s actually not what they said.”

Sara agreed with the direction but not the obituary. Prompting, for her, is how you phrase a single request for the most useful response; context engineering asks the larger question – “what does the system need to know or remember, or retrieve or verify or evaluate so that I get consistent, useful output?” In research practice that means deciding which transcripts and reports the model can access, how evidence is structured, how claims link to their quotations, and how the system handles uncertainty and contradiction. Her image for it: “the prompt is the front door, and context engineering is the architecture of the whole entire house.”

“Context is king.”

— Bruce Phillips

Ben went furthest, arguing that context engineering is itself only the next rung. Beyond the prompt and the context, he points to “the engineering of the agents and the agent flow” – the architecture, the decomposition, and the context each individual agent needs. More is not better: “giving too much context to an individual agent… starts to randomize it. What you’re looking to do… is to restrict their decision space.” Prompt engineering, in his telling, is one lever among many – critic agents, quote-grabbing agents, and the architecture that wires them together.

Bruce has leaned into context harder than anyone – yet he hasn’t abandoned the prompt. He still fires off what he calls “day-zero prompts,” with no context, “almost every day for simpler tasks such as find emails from a particular person or content on a particular topic.” Those were “wow moments a year ago” and are now just expected. But his most important work runs on accumulated context: he keeps a standing record of his priorities, stakeholders, schedule, and tools, so “I can talk to the AI like someone who knows me.” Where he once had to spell out “these are my top three priorities, please look through my inbox,” now “the AI has access to all of this information, so it does not have to be in the prompt.” Every new session used to feel “like I was being introduced to a new AI”; today he starts “with AI that’s been working with me for a year.” His verdict: “context is king,” and as models improve it is “probably taking over as the most important thing to be maintaining.”

“We ask not only does the system work — we ask what kind of judgment might it quietly remove.”

— Sara Chizari

Does the researcher’s mindset transfer to systems design?

We asked whether researchers are naturally suited to systems design, most gave a version of the same answer: partly – you already have half the muscle.

Chris argued that researchers already run systems, even if they never call them that: research intake, screener, recruitment, study design, execution, analysis, and reporting all add up to “a kind of system,” as does triangulating qualitative data, quantitative data, and support tickets against a hypothesis. The new muscle, he says, is learning “how to scale that with AI” in a way that is repeatable and quality-raising.

Ben drew a sharper line between two kinds of design. Spotting gaps and shortcomings in your own practice and imagining how to change it is, he says, “a well-exercised muscle of the UXR mindset.” Architecting the technical system is the newer one: designing what a system should do is “a very well-worn path for UX researchers,” but architecting the system itself “is probably a new muscle.”

Sara named the new muscles specifically – “orchestration, information architecture and context… evaluation, permissions and recovery” – while crediting researchers’ existing instincts for spotting breakdowns, tracing feedback loops, and working under uncertainty. She says of researchers uniquely bring – “we ask, not only does the system work – we ask what kind of behavior does the system make easier, and what kind of judgment might it quietly remove.”

Bruce was the one who declined to generalize – “I don’t have a good answer to that; it varies a lot across people.” But he named a shift the others didn’t: a new layer of systems in how researchers relate to people. Where the old world was “the human layer” – have a conversation, take notes – there is now “human to human, human to context, context to human,” and being able to “plug into and extract from those systems” is “a huge professional advantage.” It has changed his behavior concretely: “I almost never say no to a meeting,” even a peripherally interesting one, because he can capture the transcribed notes and feed them to AI. “There’s no reason to ignore information if you think that at some point it might be relevant to you.”

Where to start

For researchers who haven’t reimagined their workflow yet but are interested, the suggestions from our interviewees were telling.

Ben and Chris both said: start small, start repetitive, start low-risk. Ben’s first build was a script that reformatted and scrubbed personal information from a quote – something he does all the time, low-stakes enough to be safe, and useful for learning “how do I build an agent? How do I control what it’s doing?” Chris invoked the DRY principle – “don’t repeat yourself,” which he first read about in Hadley Wickham’s R for Data Science – and pointed at participant screeners. The old way meant keeping an Excel question bank and copying questions into each new screener by hand; an AI workflow that takes the brief, the goals, the question bank, and the target personas “can really speed up that process” and free attention for crafting higher-quality questions. Pick the thing you do every study, they agree, and start there.

Sara pushed in the opposite direction – toward the highest-leverage problem, not the lowest-risk one. Most researchers, she notes, begin with note-taking and theme organization, which saves time but doesn’t touch “our largest structural problem”: research tends to go static “at exactly the moment that organizations have a lot of questions.” Her suggestion is to take one completed study and ask “how might it remain consistent and useful six months from now” – how a PM or the design team could explore the evidence the way they actually need to. Start, in other words, with the afterlife of the research.

Coming up next

If there is a through-line across these four researchers, it’s that going AI-native is not about running the old workflow faster. It’s about asking which parts of that workflow should still exist – and then building the systems, tools, and guardrails that keep a human firmly in charge of the judgment. In the next episode, we continue Season 2 with the next group of researchers reworking their practice around AI.