Penetration testing now describes at least three different things: a fully automated assessment, a hybrid engagement where AI assists a human tester, or a fully consultant-led, manually delivered test.
The NCSC defines a penetration test as "A method for gaining assurance in the security of an IT system by attempting to breach some or all of that system's security, using the same tools and techniques as an adversary might." By that definition, all three qualify; but they differ in how that assurance is established and what confidence it provides.
The problem for buyers is that all three produce superficially similar deliverables (a report full of findings), and a proposal doesn't always say which one you're getting. The confidence you can place in an assessment's results depends on how the testing was carried out.
This article explains what separates the three approaches, where AI fits into the picture, and what to ask a provider before you authorise an assessment.
What penetration testing looks like today
A penetration test is not a compliance exercise, and it is not a vulnerability scan. Testers provide assurance by doing what an attacker would do: attempting to breach and compromise the target.
There are a few different types of penetration testing commonly sold today:
- Manual penetration testing (or "pentesting"): A qualified consultant performs the test, reasoning about how an attacker might exploit your systems, and providing context and remediation guidance for each finding.
- AI-assisted penetration testing: A consultant still leads and delivers the test, but uses AI tooling to speed up parts of the work, such as processing reconnaissance data or drafting sections of the report.
- Automated penetration testing: Agentic AI tooling is used to perform reconnaissance, enumeration, and even exploitation, with limited or zero human involvement.
The difference between these approaches is in the depth of coverage, the context of the findings, and the level of assurance that comes from having a qualified person responsible for the work.
Manual, consultant-led penetration testing
Manual pentesting is performed by a qualified consultant who understands the application or environment being tested, and can reason about how an attacker might exploit it. They can identify business logic flaws, access control weaknesses, and other issues that a scanner or AI tool is unlikely to find.
A consultant can also ask your team questions mid-test, flag something unexpected the same day it's found, and adjust the approach based on what they observe during testing.
There's also a practical difference in how an engagement is performed. A manually delivered pentest remains within the boundaries of the scope and adheres strictly to the rules of engagement, because the person doing the work has read them and is accountable to them.
To be fair to the automated side of the comparison, manual testing has a weakness of its own: consistency. The quality of a consultant-led pentest depends on the individual doing it, and a provider's team can span everyone from a junior tester building experience to a senior consultant with more than a decade of it. Two people given the same scope may not always find the same things due to their level of experience, speciality, or their own personal methodology.
This is one reason that vulnerability scanning is commonly used during an infrastructure penetration test to supplement manual vulnerability discovery, by ensuring that any low-hanging fruit isn't missed.
Scanners are fast, repeatable, and are great at discovering missing patches, weak TLS configurations, exposed admin panels, and other issues that can be reported at face value. Combining the two in practice lets the automated tools catch the obvious while the consultant goes deeper into the areas a tool can't feasibly analyse.
Where AI fits into penetration testing today
The newer arrivals in the pentest market are agentic AI tools. When given a set of targets they work through the reconnaissance, enumeration, and even the exploitation and reporting phases largely (or entirely) on their own.
These tools range from guided assistants that suggest the next technique, through to systems that can discover and chain automated exploitation attempts on their own. The value to clients is faster delivery, broader coverage, and a lower price than a manual engagement.
AI tooling does have genuine strengths in specific parts of a testing engagement. Modern LLMs are well suited to processing large volumes of reconnaissance data and correlating findings across a big attack surface. As large statistical models with an enormous amount of pre-trained knowledge, which can also be fed information on the fly, this is exactly the kind of work they're built for.
When used to support an experienced tester, AI can make an engagement much more efficient without changing the shape of their pentest methodology. Some providers will even use these tools for drafting the descriptive parts of a report.
Where the concern with AI penetration testing comes in is when these tools are used to replace a human tester, rather than as an assist to one. Compliance standards, like PCI DSS, recommend that a penetration test is performed by a qualified person (OSCP, GIAC, CREST, and so on) with previous experience conducting testing against a similar environment.
AI tools don't meet that bar; they aren't qualified, and they don't have experience. They don't know what your network architecture or application is meant to do without significant prompting, and they don't have the lateral thinking skills that a human tester develops through years of experience. Novel attack techniques are unlikely to be found by a tool that only knows what has been seen before, and a tool that can generate new attack ideas is likely to be generating a lot of false positives too.
Scope creep is a concern of its own. An agentic tool that misinterprets its instructions, or hallucinates a target, can drift outside the agreed boundaries of an engagement and start testing systems that were never in scope. In a production environment, that's unplanned attack traffic against live infrastructure; if you're evaluating a provider that uses agentic tooling, it's worth verifying what technical controls stop the tool at the scope boundary, rather than relying on the tool to interpret it correctly.
However, none of this means that AI tooling is useless in a testing context. The value depends entirely on how it's used: as a way to make a human tester faster or more efficient, or as a replacement for the tester altogether.
Manual vs automated at a glance
The table below summarises the key differences between vulnerability scanning and the three types of penetration test.
| Capability | Vulnerability scan | Automated (AI) pentest | AI-assisted pentest | Manual pentest |
|---|---|---|---|---|
| Known CVEs, missing patches, and misconfigurations | ||||
| Business logic and access control flaws | ||||
| Findings verified by a person before reporting | ||||
| Report written entirely by the tester | Varies (AI may draft sections) | |||
| Consistent, repeatable coverage | Same checks every run | Varies between runs | Varies by consultant and AI tooling | Varies by consultant |
| Continuous, always-on coverage | ||||
| Cost and speed | Lowest cost, fast | Low cost, fast | Priced on consultant time, often faster delivery | Priced on consultant time |
| Stays within the agreed scope | Risk of scope drift | |||
| Your data stays between you and your provider | Not typically (for cloud-hosted models) | Depends on the provider's AI policy | ||
| Suitable where AI use is restricted or prohibited | ||||
| Named person accountable for the work |
Why they cost different amounts
A scanner or an AI tool has a low marginal cost to run once it's built. A manual test is priced on the time that a skilled consultant will spend working through your specific application or environment; this is why the size of a target network, the number of user roles for an application, and the complexity of any workflows involved can all directly affect the price of a pentest.
Automated AI pentests are usually sold as a platform subscription or a per-asset fee, so the price can be based on how many targets you point it at and how often it runs, rather than anyone's effort. That's why they can undercut a consultant-led engagement, and the lower price reflects the lower cost of running a tool, not equivalent depth.
An AI-assisted engagement, by contrast, is still priced on consultant time, and you shouldn't assume the efficiency gain reaches you. If AI tooling means four days of effort now covers what used to take five, some providers will still quote five and keep the difference as margin. Faster consultants don't automatically mean cheaper quotes.
Where a consultant's time is what you're buying, how that time is sold also shapes the engagement delivery:
Day-rate pricing is transparent about effort, but it places the scoping risk on the client. If the scope is underestimated, the engagement can end before coverage is complete. If it's overestimated, the client pays for time that isn't needed.
Fixed-price scope based engagements move that risk to the provider, and it's on the provider to ensure the scope is covered if the assessment takes longer than expected.
Fixed pricing has its own trade-offs, as this can give the provider an incentive to finish quickly, which is why the scope document and methodology are so important. A well-defined scope sets what 'covered' means clearly enough that you can hold the provider to it.
Whichever model you're quoted for, you should ask "what happens if the scope isn't fully tested in the time allocated?"
For a full breakdown of what drives cost and typical price ranges by service type, see our penetration testing pricing page, or use the cost calculator for an indicative range based on your own scope.
How to tell which one you're being sold
Before you accept a quote, ask the provider these questions directly:
- Who performs the testing, and is any part of it outsourced or subcontracted? If the work is subcontracted, the credentials and experience you evaluated when choosing the provider may not belong to the person who's actually testing your systems. Ask for the name and qualifications of the consultant who's assigned to your engagement.
- Is testing manual, automated, or AI-assisted? Ask them to describe, in plain terms, what a human is doing during the engagement versus what a tool is doing.
- Can I see a sample report? A report built from scanner or AI output tends to read differently to one written by a consultant who understood the application. Look for findings that reference your application's specific logic, and not generic descriptions of a vulnerability class.
- What methodology or framework do you follow? Providers doing manual work will usually be able to point to something like the OWASP Web Security Testing Guide, NIST SP 800-115, OSSTMM, or PTES, and explain how it applies to your specific engagement.
- How is a critical finding handled if one comes up mid-test? This tells you whether there's a person actively watching the engagement or a tool is running unattended until the report is generated.
An AI tool that's interacting with your systems raises questions about where your data is transmitted and processed during an assessment. If you're using an AI pentest service, or if it's an AI-assisted pentest, do you really want the information that's captured during an assessment being sent to a cloud LLM? Our AI penetration testing risks article has a fuller set of questions to put to a vendor on that front.
A manual, consultant-led engagement is more expensive than a scan or AI tool, but it also provides a higher level of assurance that the findings are accurate, contextual, and worth acting on.
Where automation and AI-assisted testing can be useful
Automated tools still have a place in penetration testing. Your vulnerability management programme should include a mix of automated and manual testing, and the right balance depends on your business requirements, the criticality of your systems, and the resources you have available.
An AI-led platform-based pentest can be a reasonable fit for continuous coverage between full engagements, picking up newly disclosed vulnerabilities and changes across a large attack surface, as long as you treat that as breadth rather than a substitute for the depth of a consultant-led test.
When used appropriately, AI can also make a manual engagement more efficient: accelerating reconnaissance, drafting reports, or acting as a knowledge base for the testers themselves by turning a ten-minute research detour into a few seconds.
The argument of "automation vs no automation" comes down to whether a qualified person is doing the testing and taking responsibility for what's found. If something goes wrong during an AI-led pentest, who takes responsibility, and how quickly can the testing be stopped? With a consultant-led engagement a named person is accountable for the work, and a phone call halts testing immediately. That's harder to guarantee when a tool is running unattended.
AI also has a valuable role earlier in the process by reviewing code for vulnerabilities before they reach production. Catching an issue before it's merged is a far better position to be in than paying a pentest to find it once it's live, though that's more of a development control than an independent assurance exercise.
Our own position is straightforward: every Exploitr pentest is consultant-led and manually delivered, and AI is not used to perform testing. We're a signatory of the CREST AI Charter and support transparency in AI-delivered cyber security services. See our AI policy for the details.
If you want to know exactly what that looks like in practice, our penetration testing methodology page sets out how every Exploitr engagement is scoped, tested, and reported. You're welcome to ask us any of the questions above before you commit to anything, and we guarantee you'll get an answer from a human being.
