How Swan built an agentic software factory that finds production bugs, reaches consensus on the fix, opens the PR, and proves it works before a human merges.
Two years ago, there were no agentic commits at Swan. Today, the vast majority of our code is produced by autonomous or human-driven agents powered by agentic security and QA pipelines. Bookmark this and feed these ideas to your agent to build your stack.
Back in March, I wrote about Cygnet, our in-house agentic coworker: Slack thread partner, Linear project manager, and unattended code pipeline. That first version was an agent you had to tag in Slack. Since then, we’ve built it to find bugs on its own, work out what’s wrong, and build the fix for human review before anyone asks. We’ve added routines, reporters, and pipelines to help us ship faster and more accurately.
Let’s take a look at Cygnet in action today:
On July 9th, a partner webhook started sending a massive number of events to our database, so the job failed, retried, and failed again. A Cygnet job automatically queried Datadog, found the burst, and opened a ticket that included the query it had run, the version, the time window, and a guess at the cause.
A second job picked up that ticket and sent out three investigators. Each one re-ran the query, counted how many events matched that shape, and proposed a hypothesis. Two of them landed on the same answer while the third disagreed, and all three write-ups went onto the ticket along with the dissent. After that, another group of agents designed a fix. An engineer read the resulting spec and approved it by adding a label.
That afternoon, the pipeline opened a PR and tried to prove the fix worked. The proof run failed, so the agent handed the PR to a human, who reproduced the bug in a local container, confirmed the fix, pushed a test, obtained a second approval, and merged the change. A post-deploy routine verified that the fix was in by automatically reviewing logs and reporting back.
In March, every run began with someone tagging Cygnet on a ticket. Anyone on the team can still do that, though most of what it does now begins on its own. A discovery job opens tickets for production errors. Feeder jobs pull those tickets through triage, spec, implementation, and review. A human is tagged in at the stages where we need oversight.
Cygnet now also operates a set of automated workflows that cover weekly domain watchtowers, daily release checks, invariant checks, security news alerts that ask whether Swan is in the blast radius, and a nightly knowledge cycle we call Dreaming. Smaller jobs handle chores like refreshing the product wiki and updating knowledge for the bug pipeline to self-improve.
The bug-fix pipeline
Our original code pipeline takes a ticket all the way to a PR in a single job. The bug-fix pipeline was designed to have more human input because the bug triaging process itself is automated, so we use ticket labels as the state machine tracker. A stage claims a ticket, does its piece of work, and exits by writing whatever label the next stage is waiting for.
Discovery runs on a cron powered by GitHub Actions. Triage, spec, implement, and review get dispatched as tickets become ready. If something stalls, a resume job retries a couple of times before parking the ticket somewhere a human will notice it. A canceled or timed-out job puts its ticket back in the queue. Stale labels get swept off closed tickets, and a wrong diagnosis gets cleaned up so it can’t confuse the next stage.
Bug tickets come with evidence attached, which usually includes a monitor, an error signature, and a time window. Three investigators run in parallel. Each one re-runs that evidence against live data, works out a root cause on its own, and comes back with a structured verdict covering whether the evidence held up, what it thinks the cause is, and a confidence score.
A script reads those verdicts and decides where the ticket goes. If two or more say the evidence is bad, the ticket gets canceled. If the root causes agree at high confidence, the findings are merged, and the ticket advances. Anything in between goes to a human with the competing candidates listed, so whoever picks it up doesn’t have to redo the investigation.
Spec production works the same way, with three bugfix designers. Each verifies the root cause against the code, proposes a fix, and answers whether the change would actually ship: which flags guard the path, whether each file is part of a release build, and which versions are affected. A proposal missing that section fails the stage. If the proposals diverge, or reachability couldn’t be established, the ticket goes to a human even if the original bugfix looked easy. Using consensus for the fix helps prevent us from accidentally bandaiding a bigger problem with a simple fix, which is a common problem with AIs maintaining our systems.
Once a pipeline opens PRs by itself, you need a number that tells you how often it’s right. Every PR gets a single verdict from the human reviewer: correct fix, wrong root cause, or not a real bug. Precision is calculated as the number of correct fixes divided by the total, and that’s the figure we look at when building pipelines so that we don’t create too much AI noise and can increase the confidence for human reviewers.
Once the fix is accepted, the diagnosis is counted based on the label. But that doesn’t always tell us if the diagnosis was arrived at efficiently, or whether we ended up merging something slightly different from what it suggested. To plug that gap, another job investigates what actually merged. It compares the recorded diagnosis with the merged diff and the reviews. This helps us understand if we had to tweak the code at the end in relation to the original diagnosis. If a gap is found here, we add a lesson to the product knowledge base to avoid making a similar mistake in the future. This knowledge is then loaded into future consensus loops.
Automated Proof of Work and PR Response
Before we ship changes to production, we provide a Proof of Work, usually in the form of unit test output, screenshots, videos, or other artifacts demonstrating that the feature is working. We used to do this by hand, but we have increasingly been experimenting with an automated stage.
The Proof of Work stage boots an offline environment, drives the changed flows through a real browser, takes screenshots, and posts them on the ticket. Anyone can point to that same workflow from any PR. Backend changes typically get quick smoke tests, while UI changes go through slower browser-based testing. Not all Proof of Work is equal, and sometimes the agents may struggle to produce it. This happened on July 9th, and we automatically flagged the ticket for human follow-up.
Several review systems touch our PRs, including a multi-stage deterministic QA pipeline that covers about nine different categories from security to semantics. The bug-fix review stage responds to these comments. It fixes failing CI and implements every actionable comment, declining only when a comment is wrong or out of scope, and then it replies to explain why. It resolves the threads it addressed, reviews the full diff, and posts one verdict: ready for a human to merge or not, with a reason.
Slack reactions and Cygnet Dreaming
After a few months of running Cygnet, we came up with a few improvements. A freeform request with no Linear ticket now creates one, starts an agent that explores the codebase and writes a spec, and lets that agent ask questions back in the thread.
We added reactions. A green check tells Cygnet to build the plan it proposed in the ticket, directly from Slack. A downvote makes the bot open a ticket describing what went wrong. A trash can reaction redacts a bot message in case it produced something sensitive.
“Remember this,” writes a memory page, gated behind an approval that needs a human reply and hard-blocks anything that looks like a secret. We wanted to have human approval for memory writes because memory poisoning can lead to security issues or worse outcomes once an incorrect memory is written. The live memory index sits in the chat prompt, so what one person taught it last week is in context for everyone this week.
Sessions are keyed by Slack thread, so that Cygnet doesn’t lose context while talking to multiple people in multiple threads. Every session ends with a short retrospective on the thread, which is later used for knowledge building during the Dreaming cycle.
Dreaming is the process of digesting a bunch of runs from the day that contain raw experience: retrospectives, corrected threads, canceled tickets, skills that loaded and then got overruled. It runs overnight on the larger models with a large pre-fetched bundle of Slack, tickets, skill telemetry, and its own observations log. It gathers signal, lints for contradictions, catalogs candidates, writes the wiki, and logs its dreams so that we can inspect the cycle.
Most observations are noise and live in an append-only log. An observation that hardens into a rule becomes a memory page, and each cycle appends supporting or contradicting evidence. When a block has enough evidence from enough sources, it gets promoted into a skill, through a ticket and a PR that a human reviews. Skills have the greatest potential to impact a large number of agents and humans, so we treat skill updates with care to avoid poisoning.
Every skill load is logged and matched to the thread it happened in. If it loaded and the session succeeded, we leave it alone. If it loaded and a human corrected the result, the content is wrong. If nothing loaded and a human had to step in, the trigger is too narrow. If the wrong skill is loaded, the trigger is too broad. One human correction is enough to create a block, because when someone says “no, not like that,” we don’t need to wait for it to happen again. Each skill lives in one home, the repo whose domain it serves. An hourly sync publishes it into a plugin that agents and human agent sessions can load in as needed.
When Dreaming is unsure, it asks questions in the Slack summary. Once humans reply, the next cycle reads the whole thread. A canceled Dreaming ticket counts as negative evidence, since someone looked at the idea and said no.
Reporters and routines
One of the major upgrades we made to Cygnet since we launched it as an on-demand coworker was the concept of routines and reporters. We used GitHub Actions to kick these off and built a framework that makes adding new jobs as simple as writing a YML file, which agents are very good at. Here are a few that we use regularly:
Watchtowers gather domain-specific data on Monday morning, and a composer turns those rows into an exec summary an hour later. For example, they might look at the new user pipeline or trading activity to ensure that things are running smoothly. They help us spot anomalies and trends across various business domains, using real-time data from Datadog and other tools. They write reports and give us color-coded summaries in Slack channels for teams to investigate.
Shipwatch checks that recently shipped tickets actually work in production and can test whatever hypothesis the change was meant to prove.
Security News Alerts is event-driven, based on a security feed. It triages security news, assesses whether it could affect something at Swan given our technology stack, and flags the news item for human review when it thinks we should look more closely.
We are also experimenting with an agentic security response, where agents can gather security signals from a half-dozen security systems, investigate patterns, and enhance the reports with prior knowledge. This saves humans significant time in gathering context to investigate a security alert.
Agentification is a Journey
We started the agentification of our business back in April of 2025, and we’ve found that the more reliable we make our tooling, the more people use the tooling to deliver business value.
We’ve also found that making our baseline experience better, e.g., by putting a Slack agent in right away, empowered many more people to solve their own problems. For example, we have observed humans in client services directly ask questions of Cygnet instead of tagging ops personnel as they would have in the past. They get their answers faster and with more context, which lets them get back to clients with the right solutions sooner.
The people who used to work in a “go-between” role, such as ops, now increasingly produce their own tooling to automate their jobs. To help people transition into engineering, we instituted a program called Wingspan, which pairs experienced engineers with new agentic coders. It seems very easy to build prototypes with LLMs, but getting them to the level that would pass muster with Swan’s secure coding standards and operate within its secure coding environment is something else entirely. Over the past two years, we went from zero committers using AI-marked commits to the entire engineering team using agents to deliver code, plus ten people with no prior engineering experience from across the company.
Having a guided pairing program is important for organizations that are considering empowering people with little or no engineering experience to ship code with LLMs. This program helped us educate and provide resources to people building internal tools for themselves, while freeing up more time for senior and experienced engineers to actually build the pipelines, guardrails, and tooling so that more people could ship.
We know there are more and more off-the-shelf enterprise agentic systems, but every enterprise is different, and there is a lot of value in building a system that suits the shape of your company. It also keeps us independent from the frontier labs or third-party vendors so that we don’t end up building too much infrastructure directly on top of a company that can raise its prices or suffer an outage. We recommend using some of the ideas in this article to bootstrap your own agentic system and play around.
Building is now cheap, but security is still hard and expensive, so we devote a disproportionate amount of time to ensuring that things are secure. If you’re just getting started on this effort, please refer to some of our prior articles:
After the Coldcard exploit, everyone is asking the same question: is my Bitcoin custody setup safe? Swan’s CTO opens up the hardware behind Swan Vault and answers it.
