When your newest analyst is an agent

Like most security teams, we began with an abundance of everything except clarity: tools, detections, alerts and noise. Our architecture was fragmented, procurement decisions were disconnected, and we had no clear view of how our controls interconnected. The security operations centre (SOC) carried a high false-positive rate, which translated directly into analyst burnout and skilled people doing low-value work. I was convinced that time saved here would pay back across the whole of cyber defence — freeing capacity for proactive threat hunting and the strategic work only humans can do.

That conviction set us on the road to generative and, eventually, agentic AI. What follows is not a technical blueprint for building agents; those resources already exist. It is a leadership account of the planning, enablement and cultural challenges that vendor blogs rarely mention, distilled into a handful of lessons for security leaders who are just beginning, midway through, or planning their own AI journey.

Tools without workflows are just noise

Our first attempt was humbling. We took a security-specific generative AI assistant and dropped it into the environment to help with repetitive tasks: summarising incident tickets, drafting reports, analysing threat intelligence. The results were, frankly, ‘meh’. We had introduced the tool in a stand-alone fashion, without embedding it in workflows, without baselines and without clear guidelines. Usage became inconsistent and entirely dependent on individual analyst preference.

Vendors market these tools as plug-and-play. In practice they demand a deep understanding of your data flows, process gaps, data quality and organisational context — none of which comes in the box. The lesson I now repeat to anyone who will listen: start by asking what problem AI is solving, not what technology is available.

The turning point came when we aimed the assistant at a specific, painful target: our detection rule base. Rules were inconsistently named, poorly governed, hard to search and barely mapped to MITRE ATT&CK. Using the assistant, we accelerated our migration to detection-as-code, improving both efficiency and code quality far faster than manual effort alone could have. A vague tool became a valuable one the moment it had a real problem to solve.

Foundations before frontier

We talked a lot about becoming a ‘frontier organisation’ — one that moves beyond basic AI tools to embed advanced, agentic systems deeply. My view has not changed: frontier status is an aspiration, not a switch you flip. You cannot leap from a traditional model straight to the frontier. You first have to build the foundation of strategic vision, clean data, mature processes and human readiness.

Agents perform poorly in environments with poor data, broken processes or missing context. Before we built our detection engineering agent, we used the assistant to clean and standardise the rule base — an unglamorous prerequisite that made everything afterwards possible. The richer the context we gave an agent, the more accurate and useful its outputs became.

The single biggest success factor, though, was something simpler: protected time. Rather than treating agent development as a side-of-desk activity squeezed between operational duties, we dedicated two people full-time to designing and building our custom systems. Freed from competing priorities, they could focus on the design, the outcomes and the relentless iteration that quality demands. That decision, more than any tool, is what produced agents we could trust.

Agents are team mates

The most profound shift in our thinking came when we stopped treating agents as tools and started treating them as team members. They generate content, make decisions, execute logic continuously and interact with one another — much like humans on shift. Once we saw them that way, everything changed.

We now give our agents ‘job descriptions’ with clear objectives, defined collaboration points and access controls. We onboard them the way we onboard a new hire: coaching, feeding context, correcting mistakes and building trust gradually rather than assuming it from day one. When we upgrade a model, it feels like a promotion — the agent suddenly has new capabilities, which prompts us to revisit its permissions, role and place in the architecture.

We built three systems on this principle, each designed around human-in-the-loop oversight rather than full autonomy. An incident analysis system performs initial triage, deep analysis across data sources, limited threat hunting and structured recommendations for human analysts. A detection engineering agent manages the full rule lifecycle — validation, optimisation, overlap checks, ATT&CK mapping and threat-intelligence integration — and analysts can simply converse with it. A threat intelligence agent ingests and operationalises intelligence, generates hypotheses, runs targeted hunts and creates tasks for our proactive teams, linking directly to the other two. The improvements are not theoretical: less burnout, faster response and more time for strategic work.

How we built it

People often ask which model we used and how long we spent training it — and the honest answer tends to surprise them. We took a capable, general-purpose large language model with strong agentic and tool-use abilities and shaped it: a tightly defined job, a small set of tools, and a clear set of rules. The intelligence was, in effect, off the shelf; the value we added was context and constraint. For most security teams that is the real unlock — you are not hiring data scientists to train algorithms, you are configuring and guiding a capable general system to reason the way your best analysts do.

That clarity also drove the build-versus-buy decision. The market is full of vendors promising an ‘AI SOC analyst’ in a box, but a bought product gives you someone else’s judgement. We built our own because an agent that sits inside your defence team has to understand your environment, your playbooks and your risk appetite — and, as the rule-based work taught us, that context is the whole point.

So ‘training’ meant something closer to coaching, just analysts walking each agent through real, sanitised scenarios, correcting its reasoning and tightening its instructions until its judgement held up. The timescale is worth stating plainly for anyone planning their own: we had useful output within a few weeks, but earning the team’s confidence took a couple of months of running the agents in parallel with humans and watching the gap close. Like any good hire, an agent embedded in a live team is a relationship to maintain, not a project to close.

Getting comfortable

The elephant in the room is job anxiety. Even as a leader, I felt the tension: if agents handle the routine work, what happens to roles? My honest answer to the team was that AI will not take your job — but someone who uses AI well might. The work is evolving towards the more nuanced, complex and collaborative, including industry-wide threat collaboration and societal-level defence.

For you

Be part of something bigger, join BCS, The Chartered Institute for IT.

We addressed this head-on with a dedicated ‘People Workstream’ covering governance, assurance, job-description updates and time-reallocation planning. But comfort was not only a people problem; it was an engineering one too. As the accountable leader, I had to be personally confident the agents were safe, auditable and properly controlled before they could make decisions in live workflows.

That meant multi-layered controls: authentication — especially for agent APIs — identity management for inter-agent and agent-to-tool communication, governance of secrets and tokens, content filtering to mitigate prompt injection and jailbreaks, and defence-in-depth authorisation backed by role-based access controls. We added independent verification and validation: deterministic pre-checks before any AI-driven decision, confidence scoring on machine-generated detections, conservative downgrade logic, and purple-teaming to test that the controls held against real threat behaviours. For high-risk actions, such as call-out recommendations, we required documented rationale, parallel running with humans and shift-handover summaries of agent decisions. We treated trust like onboarding: gradual, evidence-based and never assumed.

Proving the value

AI enablement has to deliver a measurable return. We kept our traditional SOC metrics — triage time, mean time to respond, false-positive rates — and added agent-specific ones: detection coverage gaps closed, intelligence-to-action time, manual hours saved, and unnecessary call-outs avoided, meaning the times the system correctly decided not to wake a human for a false positive. We also track employee engagement and workload, because reducing burnout was the whole point. Costs — prompt usage, tooling, engineering time — are tracked against those gains so we can see return on investment holistically rather than as a vanity metric.

What comes next

We are still early. Agent-to-agent interactions, long-term role evolution and — soberingly — the parallel development of adversarial agents by threat actors all remain open questions. In regulated, critical national infrastructure environments, assurance expectations around transparency, auditability and accountability are only tightening. We will need new layers of purple-teaming for an AI-versus-AI world, continuous career-path redesign so that people are not deskilled, and mature cost governance so that savings are reinvested in strategy rather than quietly absorbed as headcount cuts.

The principles, I suspect, will stay constant across this and the next technology revolution: start with the problem, build the foundation, treat your agents as team mates you remain accountable for, and care for your people first. This is my ‘ithihaasa’ — a Sanskrit word meaning ‘that is how it happened’. Every journey will be different, but I hope ours helps other leaders navigate theirs with a little more clarity, confidence and humanity.

About the author

Reena Pathak, Director of Cyber Security, works in a critical national infrastructure environment. The opinions expressed are entirely the author’s own and do not reflect the views, policies or practices of any current or former employer. All examples, metrics and technical details have been generalised and anonymised for illustrative purposes.