Rise of the Agentic Workflows

A Quiet Announcement, Big Implications

In February, the GitHub team announced that Agentic Workflows were entering technical preview.

This announcement went relatively unnoticed for many in the industry, it seems. Perhaps due to the relatively obscure nature of the concept.

But for those working in software engineering and testing fields, I think we are seeing the beginning of a huge transition in conventional CI models and perhaps more broadly testing approaches in general.

For software engineering and testing teams, this feels like the beginning of a major transition in how CI can be used.

What Agentic Workflows Actually Change

Agentic Workflows are, in effect, a prompt-driven CI workflow (or pipeline in old speak). Instead of your workflow calling a pre-defined script (such as an NPM script for calling a Playwright test suite), you can instead provide a prompt to an LLM in the same way you would locally when working with an Agent.

This opens the door to vast possibilities around creating non-deterministic style instructions. My first thoughts when I heard about the news were centred around how coding agents in GitHub could, in effect, perform a form of exploratory testing of websites or services. This, combined with setting these workflows to run on a schedule (cron, for example), opens up a whole world of possibility…

Prompt-driven CI introduces a practical way to run exploratory-style testing continuously, not just during manual test sessions.

Using Playwright MCP in CI

You can use the workflows alongside toolsets such as Playwright MCP, which means you have the ability to tell an Agent to use Playwright to test whether a website is working and/or bug-free. The prompt can be as broad or as specific as you like with regards to focus, methodology, and overall context.

For example, I used a template Accessibility workflow and tweaked it for my needs. It serves a site locally on the CI machine, then prompts an Agent to inspect the site using Playwright MCP for accessibility issues. If it finds any, it logs them as issues directly in GitHub (configurable).

This is a super powerful workflow and one that didn’t take particularly long to get working after a bit of play and the odd GitHub issue raised along the way. What it represents is rather staggering: a fully prompt-driven exploratory accessibility testing workflow. It will not necessarily approach the task the same way each time it runs the MCP. But this isn’t necessarily a bad thing. Like a manual tester, it might catch things in a different way or check pages in a different order. The end result, though, is that it finds accessibility issues without any need for a scripted series of accessibility tests.

A fully prompt-driven exploratory accessibility workflow can find issues without relying on a scripted set of accessibility test cases.

Scripted and Agentic Testing Together

There is, of course, always going to be a need for scripted tests (probably). These ensure you know precisely what is being covered, and these should perhaps target core business journeys and functionality going forward.

Where This Could Lead

Where Agentic Workflows come in, in my view, is with the more exploratory side of the mix. You can use them to perform smoke tests of deployed sites, ensure no glaring regression issues have occurred, or simply to check your app is up and running. This might eventually replace conventional monitoring/synthetic tests as a result because they could act as a more dynamic alternative to scripting such tests and relying on such third party services.

Final Thoughts

In summary, I’m very excited by the potential Agentic Workflows bring to CI. If you work in the testing field, particularly if you are an Automation Engineer / SDET, it is probably time to familiarise yourself with this exciting development.

If you are registered with the Ministry of Testing site, you can watch a recent ‘call for insights’ video I did about this topic here.