Senior Data Engineer

Job Title: Senior Data Engineer

Location:
 Lahore, Pakistan – On Site OR Remote only for EXCEPTIONAL candidates.

Company:
 Confidential.

Reports to:
 Head of Technology

Who We Are

We're building the central nervous system for enterprise AI. We're not creating another chatbot or a simple workflow tool. We are engineering a sophisticated, agentic infrastructure that allows businesses to deploy autonomous AI agents that can reason, plan, and execute complex tasks.

This is a ground-floor opportunity to build the data backbone of a system that will fundamentally change how businesses operate. We're a small, deeply technical team, and we're looking for a true builder to join us as our first dedicated data expert.

The Role: This Isn't Your Average Data Job

We need an exceptional data engineer to architect, build, and own our entire data infrastructure from scratch. You will be the single point of ownership for how data flows, is stored, processed, and made available to our AI agents and machine learning models.

Your work won't just support the product; it will 
be
 the product. The quality, speed, and reliability of your data pipelines will directly determine the performance of our AI agents. You will design the foundation that everything else is built upon.

What You'll Do Day-to-Day

• 
Architect from Zero:
 Design and implement a scalable, secure, and robust data architecture on Azure using the Databricks Lakehouse Platform. This is your green-field project.

• 
Build the Pipelines:
 Develop and manage high-performance data pipelines (ETL/ELT) to ingest, process, and transform massive volumes of structured and unstructured data from diverse enterprise sources (e.g., APIs, databases, flat files, event streams).

• 
Own the Lakehouse:
 Implement and manage our Databricks environment, including Delta Lake for reliable data storage, Unity Catalog for governance, and optimizing Spark jobs for performance and cost.

• 
Enable AI Agents:
 Create the data systems that feed our AI agents. This includes everything from feature engineering pipelines for traditional models to building retrieval-augmented generation (RAG) pipelines for Large Language Models (LLMs).

• 
Ensure Data Integrity:
 Establish best practices for data quality, validation, and governance. You'll be the champion for clean, trustworthy data across the entire company.

• 
Data Security and RBAC
: Secure access to individuals and groups based on designated entitlements and roles, using Microsoft/Azure security best practices.

• 
Infrastructure as Code (IaC):
 Define and manage our data infrastructure using tools like Terraform or Bicep. We believe in automated, repeatable, and version-controlled environments.

• 
Collaborate and Lead:
 Work directly with the founding team and ML engineers to understand data requirements, providing expert guidance on what's possible and how to get there. As the team grows, you'll set the standard for all future data hires.

• 
Quality Control and Monitoring:
 Implement QA measures and monitoring and improving data pipeline performance and quality across the stack. Must be proficient in building or using off the shelf notification and alerting systems.

Our Tech Stack

You'll be working with, and making key decisions about, the following technologies:

• 
Cloud:
 Microsoft Azure (Azure Data Lake Storage, Azure Data Factory, Azure Functions, etc.)

• 
Data Platform:
 Databricks (Delta Lake, Spark, Unity Catalog, MLflow)

• 
Languages:
 Python (Expert-level), SQL (Mastery-level), and likely some Scala or PySpark.

• 
Infrastructure & DevOps:
 Terraform/Bicep, Docker, GitHub Actions / Azure DevOps, Azure Databricks, DBT.

• 
Data Types:
 Everything from clean tabular data to messy, unstructured text and logs.

This Role Is for You If...

• You hold a 
Bachelor's in Computer Science or Computer Engineering
, or you have a compelling, 
demonstrated history of building impactful software
 that speaks for itself.

o Nice to Have: Masters or PhD in Computer Software or Engineering

• With 
8+ years of experience in software and data engineering
, you have a proven track record of designing and building complex data platforms from the ground up, preferably in an Azure environment.

• 
MUST HAVE:
 Demonstrable experience with medallion architecture in data warehouse solutions.

• You are a 
Databricks and Spark expert
. You don't just use it; you understand how it works under the hood and can optimize it for any workload.

• You've built 
data systems specifically for AI/ML applications
. You understand the unique challenges of versioning datasets, workflows, data ops pipelines, managing features, and building low-latency data feeds for models.

• You live and breathe the "
you build it, you run it
" mentality. You have a deep sense of ownership and are comfortable managing the entire lifecycle of the data infrastructure.

• You are 
pragmatic and product-focused
. You know when to build a perfect, scalable system and when to ship a simple solution to learn faster.

• You thrive in 
ambiguity
 and love the challenge of having no playbook. You see a blank slate as an opportunity, not a problem.

Why Join Us?

1. 
Massive Impact:
 You will be a founding member of the engineering team. The decisions you make in the first six months will define our technical trajectory for years to come.

2. 
Solve Hard Problems:
 We are working on the frontier of applied AI. The challenges you'll face are novel, complex, and intellectually stimulating.

3. 
No Red Tape:
 You'll be working with a small, elite team where you can ship code daily and see its immediate impact without layers of bureaucracy.

4. 
Performance Based Incentives:
 This is more than a job. You'll receive significant incentive as a co-builder of our vision, along with performance-based bonuses tied to key company milestones. Your success is our success, and you will be rewarded accordingly.

If you are a world-class data engineer who wants to build your masterpiece, we want to talk to you.