Analytics Engineer Assessment

Why Analytics Engineers are hard to hire well

Analytics engineering is a young enough discipline that most organisations hiring for it have not yet agreed what it is. Some are hiring a very good analyst who knows dbt. Some are hiring a data engineer who will also build dashboards. The job specification usually reflects whoever wrote it, the interview loop is assembled from whichever teams had capacity, and the candidate who gets hired is evaluated against three incompatible mental models of the role.

The work also has an unusual failure signature. Bad analytics engineering does not break anything on day one. It accumulates. Logic gets duplicated across models because copying was faster than refactoring. Definitions diverge because nobody owns the semantic layer. Two years later the project has several hundred models, nobody can say which are still consumed, and the cost of any change is dominated by fear rather than effort. Every individual decision that produced that state looked reasonable and small.

Then there is the discipline gap. This role applies software engineering practice to SQL: version control, code review, testing, environments, continuous integration. Plenty of otherwise excellent candidates have written a great deal of SQL and have never had any of it reviewed, tested or promoted through an environment. They are not weak analysts. They simply have not done this job, and a conversation about dbt will not distinguish them from somebody who has.

What separates the best from the rest

The strongest analytics engineers optimise for the next person’s change, not their own build. They will accept a slightly longer model today to avoid a fork tomorrow. They name things so somebody who was not in the meeting can work out what they mean. They write the test that catches the failure they have not thought of yet, because the whole point of a test is that the future is unknown.

They are equally distinguished by what they remove. Warehouses grow monotonically by default: adding is safe and easy, deleting requires knowing who depends on something. Good analytics engineers instrument consumption so they can answer that question, then actually delete. A team without that habit will spend an increasing share of every sprint maintaining assets that nothing reads.

The third marker is how they treat upstream. Weaker candidates regard source systems as weather: unpredictable, unaccountable, and something to mop up after. Better ones treat every source as a contract with an owner, get schema changes announced in advance, and build tests that fail their own pipeline loudly rather than letting bad data flow quietly to a dashboard. Data downtime is mostly an organisational problem, and this is where it gets solved.

Why interviews alone fall short

A live SQL exercise for this role tests roughly the least important skill it needs. Almost every serious candidate can write a window function. What determines whether the hire works out is a series of design judgements: where a piece of business logic should live, when to make a model incremental, whether two similar tables should be conformed or kept apart. Those decisions have no single right answer and cannot be graded in a coding screen.

Portfolio conversations fail differently. A candidate describes a warehouse they built, and it sounds coherent, because they are describing the intended architecture rather than the version that exists after eighteen months of pressure. You cannot see the shortcuts, the models added under a deadline, or what they did when a stakeholder wanted a metric that broke the semantic layer. Situational scenarios put those specific trade-offs in front of every candidate identically, so you are comparing design judgement rather than recall.

Common hiring mistakes in analytics engineering recruitment

• Interviewing for a data engineer and hiring for a modeller - streaming infrastructure and Kubernetes questions filter out exactly the candidates who are strongest at the dimensional modelling this role lives on
• Treating dbt experience as the qualification - the tool is learnable in a fortnight, and the modelling judgement it exposes takes years
• Never asking about version control and review - a candidate who has only ever run SQL from a query editor will bring habits that do not survive a team of four
• Skipping the cost conversation - warehouse spend is one of the few places where an individual contributor’s design decisions show up directly on an invoice, and many candidates have never seen the bill
• Ignoring deletion and consolidation - hiring only builders produces a project that grows forever, and the maintenance burden compounds long after the hiring decision is forgotten

The work this role is assessed against

• Implement dbt models, tests, sources, seeds, and macros; manage environments and documentation
• Build and operate orchestration workflows (Airflow, Prefect, Dagster); resolve failures and reduce data downtime
• Refactor schemas into star and snowflake patterns; implement incremental models and slowly changing dimensions
• Define and maintain metric layers and LookML; standardise definitions across BI tools
• Tune queries and storage, manage permissions, and enforce coding standards via pull requests and CI/CD

Tools and outputs this role works with

SQL, dbt Core and dbt Cloud, Snowflake, BigQuery, Amazon Redshift, Databricks, Apache Airflow, Prefect, Dagster, GitHub, GitLab, Bitbucket, GitHub Actions, GitLab CI/CD, CircleCI, Great Expectations, Soda, Monte Carlo, Looker, Tableau, Power BI, Mode, Fivetran, Stitch, Airbyte, AWS (S3, IAM), GCP (GCS, IAM), Azure (ADLS, Entra ID), Terraform, Docker, DataHub, Amundsen, Alation, and Collibra.