QA / Test Engineer Assessment

Why QA engineers are hard to hire well

Quality assurance is the function organisations reach for when they want quality without changing how they build. That framing follows the role into the hiring process. The job specification asks for someone who will “own quality”, which is not a thing a single person can own, and the interview then struggles to describe what the person will actually do, because the honest answer depends on how much authority they will be given and nobody has decided.

The title also spans two jobs that share almost no skills. At one end is exploratory manual testing, which is a genuine analytical discipline built on curiosity, domain knowledge and a talent for imagining what a user might do wrong. At the other is test engineering, which is software development with a different target. Plenty of hiring processes advertise the first and interview for the second, or the reverse, and then conclude the market is thin.

There is a third difficulty that hiring managers rarely name. Good testers spend their working lives delivering unwelcome news, and doing it in a way people accept is a social skill as much as a technical one. A tester who is right and abrasive gets routed around. A tester who is agreeable and imprecise gets ignored. Neither failure is visible in a conversation about test frameworks.

What separates the best from the rest

The strongest testers start from risk rather than from coverage. Coverage is a number that can be raised by testing things that were never going to break. Risk is a judgement about where failure would be most likely and most expensive, and it produces a completely different test suite: thinner in places, much deeper around payment, permissions, data migration and anything touching money or identity. Ask a candidate what they would not test and the weak ones have no answer.

They also treat flakiness as a first-class defect. A suite with a ten percent intermittent failure rate does not cost ten percent of its value, it costs almost all of it, because once people start rerunning failures the suite has stopped being a signal and become a ritual. Good test engineers know this and are ruthless: they fix the test, fix the underlying race, or delete it. Weak ones add retries, which converts a visible problem into an invisible one.

The third marker is where in the lifecycle they engage. Testers who receive finished features and look for problems find problems late, when they are most expensive to fix. The best ones are in the refinement conversation asking what happens when the file is empty, when the user has two accounts, when the request arrives twice. Much of the value in this role is defects that were never written, and none of that work looks like testing.

Why interviews alone fall short

The standard QA interview asks a candidate to describe their testing approach, and every candidate describes a sensible one, because a sensible approach is well documented and easy to recite. Boundary values, equivalence partitions, the test pyramid. It is a vocabulary check. It does not distinguish someone who applies those ideas under a deadline from someone who read about them.

The practical exercise, “here is a login form, tell us how you would test it”, fails differently. It has no cost. Real testing is entirely about cost: you have a day, a hundred plausible cases, and you have to choose. Remove the constraint and every candidate produces an impressively long list, which tells you about their imagination and nothing about their prioritisation. Situational scenarios restore the constraint. Every candidate faces the same clock, the same incomplete information and the same unhappy stakeholder, and the differences that emerge are the ones that will show up in your release process.

Common hiring mistakes in QA recruitment

• Advertising for a tester and interviewing for a developer - filtering on data structures excludes the exploratory testers who find the defects automation never will
• Treating automation percentage as a quality metric - a high automation rate over the wrong scenarios is expensive and reassuring, which is the worst combination
• Never asking about a defect that was rejected - how a candidate handles being told “that is not a bug” tells you more about their effectiveness than any test plan
• Hiring QA to compensate for engineering practice - a tester cannot inspect quality into a codebase with no unit tests, and the good ones will leave once they realise that is the assignment
• Ignoring reporting quality - a defect report that a developer can act on without a conversation is the actual deliverable of this role, and it is almost never assessed

The work this role is assessed against

• Author UI automation with Selenium, Cypress, or Playwright, and API tests with Postman or REST Assured
• Configure CI pipelines (Jenkins, GitHub Actions) to run tests in parallel and triage failures
• Create structured test cases in TestRail or Zephyr and link them to stories in Jira or Azure DevOps
• Perform exploratory testing, and file clear, prioritised defects with evidence (logs, screenshots, videos)
• Execute performance tests (JMeter, k6), analyse bottlenecks, and recommend remediation

Tools and outputs this role works with

Jira, Azure DevOps, TestRail, Zephyr, Selenium WebDriver, Cypress, Playwright, Appium, Postman, REST Assured, JUnit, TestNG, pytest, Jenkins, GitHub Actions, GitLab CI, Git, JMeter, k6, BrowserStack, Sauce Labs, Cucumber, SpecFlow, SonarQube, IntelliJ IDEA, Visual Studio Code, and Docker.