Lead Data Engineer

Lead Data Engineer (BigQuery Consolidation, Migration & Architecture) – Contract

Location:

Hybrid - London

Contract Length:

Initial 4-week engagement with extension

Day Rate:

£700 per day (Outside IR35)

Start Date:

ASAP

An ambitious, data-driven organisation is seeking a highly experienced Lead Data Engineer to assess and potentially lead the consolidation of a complex, fragmented data landscape into a single Google BigQuery platform.

The business currently operates across multiple technologies and data warehouses, creating challenges around data consistency, reporting accuracy, and stakeholder trust. The objective is to establish a scalable, future-proof architecture that delivers a true single source of truth while supporting an AI-enabled data strategy.

Initially, this will be a four-week exploratory engagement focused on validating the migration approach, identifying risks, and proving technical feasibility. For the right individual, there is potential to lead the wider migration and transformation programme.

The Challenge

The current environment consists of multiple platforms, including:

Google BigQuery

Snowflake

Microsoft Fabric

Over time, the data estate has become increasingly fragmented, resulting in:

Multiple versions of reporting across the business

Reduced confidence in data quality and consistency

Complex data flows and transformations

Increasing operational overhead and technical debt

The organisation believes there is a significant opportunity to simplify the architecture and align around a strategic Google Cloud and AI-first approach.

What You'll Be Doing

Architecture & Strategy Assessment

Conduct a deep assessment of the existing data architecture, pipelines, and reporting landscape.

Challenge assumptions and identify architectural risks, bottlenecks, and potential migration pitfalls.

Define a target-state architecture centred on Google BigQuery.

Evaluate scalability, maintainability, performance, and future AI-readiness.

Migration Validation

Perform hands-on analysis of critical data pipelines and transformations.

Validate that complex workloads can be successfully migrated to BigQuery without loss of fidelity, performance, or business value.

Assess migration complexity, business impact, and delivery timelines.

Develop a practical roadmap for consolidation.

Data Quality & Governance

Define frameworks for validating data integrity and achieving data parity across platforms.

Establish measurable success criteria for migration and reporting accuracy.

Recommend approaches for automated testing and validation at scale.

Executive Recommendations

At the conclusion of the engagement, provide a clear assessment covering:

Migration feasibility

Key risks and mitigation strategies

Cost-benefit analysis

Delivery roadmap and timeline estimates

Go / No-Go recommendation for the wider programme

What We're Looking For

Essential Experience

Proven experience leading large-scale data platform migrations and consolidation programmes.

Deep expertise in Google BigQuery architecture and best practices.

Strong understanding of modern cloud data platforms including:

Google BigQuery

Snowflake

Microsoft Fabric

Experience designing scalable, enterprise-grade data architectures.

Strong understanding of modern data engineering tooling and practices, including dbt and cloud-native services.

Technical Mindset

Highly hands-on and technically credible.

Comfortable diving into existing pipelines, transformations, and codebases rather than operating solely at a strategic level.

Able to quickly understand complex environments and identify practical solutions.

Strong analytical and problem-solving skills with the confidence to challenge assumptions.

Desirable Experience

Retail or e-commerce experience.

Exposure to AI-enabled data platforms and modern analytics ecosystems.

Experience supporting organisations through cloud-first or AI-driven transformation initiatives.

Technology Environment

Google Cloud Platform (GCP)

BigQuery

Gemini

AI-enabled analytics and decision-making

The successful consultant will play a key role in shaping the future direction of the organisation's data platform.

Interview Process

Single-stage interview process.

Direct engagement with senior leadership.

Interviews will be arranged quickly, often within the same week of application.

This role would suit a senior, pragmatic data engineering leader who has successfully delivered complex migrations before and enjoys combining strategic thinking with hands-on execution.