AI
IT pros
Copilot for Windows Server
I designed the review-before-run script experience for Windows Server's first AI assistant, which reached private preview in July 2025 with no generated script able to run without an admin's decision.
My part
Designer: chat and script-generation experience
Duration
Mar 2024 – Jul 2025
Crew
PM, engineering, design, responsible AI research
Ships on
Windows Admin Center
Context & role
Copilot for Windows Server is an LLM-based assistant for IT admins who run production servers, built into Windows Admin Center. It answers plain-language requests for event log analysis, performance trends, update troubleshooting, best practices assessments and PowerShell scripts. The work ran from the first platform spec in March 2024 to private preview in July 2025, with PM, an engineering feature crew, two fellow designers and a responsible AI research team. My role: I designed the chat and script-generation experience, meaning how a generated PowerShell script is shown, explained, labeled and handed back to the admin. A design colleague led the early scenario research, and the responsible AI team designed the overreliance study.
Stakes
A script that deletes the wrong files or stops the wrong process can take a production server down, and because generative AI is non-deterministic, some wrong scripts look entirely plausible. The audience was wary: in a 146-person survey, event log analysis was the task most often named as worth AI help, but the second most common answer was "none". Admins were already pasting scripts from general chatbots with no guardrails, and for a skeptical audience, one destructive script would end the product's credibility.
Reframe
Asked: add a chat panel. Reframed: keep a human between generate and run.
The problem as first given was capability: a set of skills and a chat panel, and an early spec even listed letting the assistant execute OS changes as a top requirement. I treated the real problem as trust under lopsided costs: make risky actions visible by default and keep a human decision between generating a script and running it, without slowing experts so much that they go back to a chatbot in another window.
Illustration: an assistant that runs scripts itself, versus one that labels, explains and waits for the admin.
Evidence
People don't verify AI output unless the design does it for them
Three sources shaped the direction: the 146-person survey (MaxDiff ranking plus open questions), five scenario interviews showing admins already scripting with general chatbots, and earlier findings that people rarely do optional verification, so safety had to live in the default layout. A responsible-AI experiment then ranked script errors by harm (syntax errors low, wrong information medium, unintended actions high) and tested highlighting risky commands against PowerShell's -WhatIf dry run. In the pilot, every participant responded well to -WhatIf, and some added it unprompted.
01
Admins were already taking the risk. Most interviewees wrote scripts with general chatbots, with no guardrails and no knowledge of their environment.
02
Optional checks get skipped. People rarely do optional verification, so safety has to live in the default layout.
03
Not every error is equal. Syntax errors fail safely; a plausible script that runs the wrong action is the one that takes a server down.
Key decision
The assistant proposes. The admin always runs.
There were three options: let the assistant execute changes directly, as the early spec allowed; show the script with an optional verify step; or make review the default by explaining the script, flagging risk and never running it without an admin's action. Direct execution was the fastest path to the "junior admin in a box" promise. I chose review-before-run because a plausible wrong script on a production server can take it down, accepting one extra step for expert admins on every script. Optional verify was ruled out by the research: an extra button is a step people skip. On a production server, that extra step is the product.
Second call
Flag risk with simple rules, not more AI
The tempting way to flag risky scripts was to ask the model to judge its own output, which adds a second layer of non-deterministic error on top of the first. Instead, the safeguard matches action commands against a known list by simple word matching, so the flag is predictable and explainable, accepting that it will miss risks a smarter check might catch. In the experiment pilot, a dry run with PowerShell's -WhatIf parameter was the most welcome mitigation, so the design treats it as part of the normal path rather than an expert trick.
Illustration: three options for where the human decision sits.
Craft
Every step of the script flow has a designed state
The entry point moved from a floating Copilot button into the top-level search area, using a prompt input, so asking felt like part of the console. A disclaimer and a loading state cover the wait while a script is generated, and the screen reader announces progress. The generated state shows the explanation and the script; a script with action commands shows a warning and scrolls to the flagged lines. After Run, output appears in an integrated PowerShell session, so the admin sees exactly what happened. Questions outside Windows Server are declined rather than answered badly. One error state is still open: there was no way yet to show when a question went to the wrong skill.
Illustration: a designed state for every step of the script flow.
System
Four trust rules for AI that touches a server
Label every response ("AI-generated content may be incorrect"). Show the work: each script comes with an explanation, and the event log skill shows the query it ran. Flag risk with deterministic rules, not more AI: action commands are matched against a known list, so the safeguard adds no new layer of AI error. Never auto-run: the admin edits, runs or cancels, and can dry-run with -WhatIf. Because other teams were invited to add skills, safety couldn't be a per-feature decision; it had to be a rule every skill inherits. I later carried the same rules into an Azure DevOps planning concept.
Illustration: four trust rules every skill inherits, including skills other teams add later.
What shipped
What shipped
The assistant entered private preview in July 2025 as the Windows Server assistant in Windows Admin Center, with seven skills from event log analysis to PowerShell script generation. Every generated script arrives labeled, explained and waiting for the admin to edit, run or cancel, with a -WhatIf dry run available, and output appears in an integrated PowerShell session. The entry point sits in the console's top-level search area rather than a floating button. Every preview-blocking accessibility bug was fixed before release, including keyboard access across the script section and correct status announcements. The overreliance experiment was designed and piloted, so the team chose safeguards on evidence rather than opinion.
Outcome
What changed
The assistant entered private preview in July 2025 with seven skills, from event log analysis to PowerShell script generation, and every script behind an admin decision. Each step of the script flow has a designed state, from loading to the flagged-risk warning to PowerShell output after Run, and questions outside Windows Server are declined rather than answered badly. Accessibility was treated as part of trust: every high-severity issue found in testing was fixed before preview. Trust measures such as edit, cancel and dry-run rates are still to come.
Reflection
Measure trust, not just use
I would push earlier for trust measures in the preview plan: the edit rate on scripts with action commands, dry-run use and cancelled runs say more about safety than queries per user. I would also move the safety experiment ahead of the UI lock, so its findings could shape the first version. The principle I took away: in high-stakes tools, AI design is about where the human decision sits and how cheaply errors become visible. Design for the costliest failure first, and prefer simple deterministic safeguards over more AI.
