QA Audit

The problem

Teams often know quality feels fragile and still cannot say why. Automation exists but nobody trusts it. There is no list of what is actually tested before a release. AI features have no regression set. Leadership asks for “more QA” without a diagnosis.

A QA audit is that diagnosis. It is not a test cycle. It is a time-boxed review of how you currently find (or miss) defects, and a written plan for what to change.

When to audit instead of hiring

Start here when:

• You are about to outsource QA or hire a dedicated QA team and do not want to scale a broken process
• Releases keep slipping and everyone has a different story about testing
• You inherited a suite of tests and cannot tell signal from noise
• AI or LLM features shipped without an evaluation habit
• A customer, board, or new CTO asked “how do we actually test this?”

If you already know you need execution, skip the audit and go to outsourcing or QA as a service. An audit is for uncertainty.

What we look at

A typical QA audit covers:

• Release cadence vs. actual test activity
• Coverage of critical user journeys (including payments, permissions, and data) — including whether a requirements traceability matrix exists and is trusted
• Manual process: who tests, in what environment, with what evidence
• Automation: what runs, what is skipped, what is flaky, who owns it
• Defect flow: from find to fix to verify
• AI testing gaps if you ship models, prompts, agents, or RAG
• Environments, test data, and blockers that waste tester time
• Reporting: whether anyone outside QA can see residual risk

We do not sell a numbered “QA maturity score” as if it were a certification. You get specific findings on your product.

How a QA audit runs

1. Access and interviews. We talk to engineering, product, and whoever currently tests — and we look at the repo, tracker, and pipelines, not only slides.
2. Sample the work. We trace a recent release: what was tested, what escaped, what was untestable.
3. Findings. Risks ranked by user or business impact, not by how impressive they sound.
4. Plan. A practical sequence: what to stop, what to automate, what to staff, whether managed testing or a dedicated team is even the right next hire.

The audit can stay independent. You are not obligated to buy delivery from us afterward — though many teams do, once the gaps are named.

What a good audit output looks like

You should be able to hand the findings to an engineering lead who was not in the interviews and have them understand: what is untested, why, what to stop doing, and what to staff next. Vague advice such as “shift left” or “increase automation coverage” is not a QA audit.

We would rather list ten specific gaps — for example, no regression on prompt changes, no staging data for the billing path, automation that is skipped in CI — than produce a long maturity matrix. If we recommend outsourcing, a dedicated team, or managed testing, that recommendation will point at the gaps it is meant to close.

What we need for an honest review

• A recent release we can reconstruct (tickets, builds, what was tested)
• Access to whoever currently tests, even if that is “the developers”
• Pipeline and environment reality, not the architecture diagram
• Permission to say uncomfortable things in the readout

If the only artefact you will accept is a green report, do not commission an audit.

What’s at risk if you skip the diagnosis

• Outsourcing a mess and paying testers to follow a bad checklist
• Buying tools before you have an operating model
• Hiring a senior QA lead and then giving them no mandate
• Discovering in production that nobody owned AI regressions

Deliverables

• Written findings: process, coverage, tooling, and residual release risk
• Prioritized recommendations (stop / start / staff)
• Suggested engagement shape if you want Nexus Eclipse to do the follow-on work
• A readout your engineering lead can argue with — that is the point

We do not invent metrics, pass marks, or compliance stamps. If you need SOC 2, HIPAA, or ISO audit evidence, that is a different specialist; we will say so.

Suitable for

Founders, CTOs, and product leads who need a clear picture of testing before they spend on people or tools — including teams adding AI features on top of an existing app.

FAQs

What is a QA audit?

It is an independent review of how your team tests software today, and a written plan for what to change. It is not the same as executing a full regression.

How long does it take?

Long enough to see a real release path, short enough that it does not become a consulting theatre. Exact duration depends on product size and access. We set that before we start, on a consultation.

Will you need production access?

We need enough access to see how you actually test: staging, pipelines, ticketing, and (where relevant) prompt or evaluation docs. We do not need to become admins of your production systems to audit process.

Can you audit only our AI features?

Yes. That is a scoped audit: evaluation sets, hallucination handling, tool-call checks, and what happens after a model or prompt change. The rest of the app can stay out of scope.

Do we have to hire you for testing afterward?

No. The audit stands on its own. If the findings point to outsourcing, a dedicated team, or managed testing, those are separate decisions.