Senior Data Engineer

About the Role

We are looking for a Senior Data Engineer
who can
build scalable pipelines for ML training, anomaly detection, and feature engineering
—and lead in designing world-class data infrastructure for AI/ML applications. If you thrive at the intersection of
data architecture, AI/ML readiness, and cloud-native engineering
, this role is for you. You'll be architecting and implementing modern data solutions that power
large-scale machine learning workflows
and
business intelligence
.

Key Responsibilities
Data Architecture & Infrastructure

• Design and implement scalable, fault-tolerant data pipelines for structured and unstructured datasets.
• Architect
  data lakes
  ,
  data warehouses
  , and
  hybrid lakehouse
  architectures.
• Define infrastructure blueprints for both
  real-time
  and
  batch processing
  .

AI/ML Data Workflows

• Develop pipelines for
  feature engineering
  ,
  data normalization
  ,
  augmentation
  , and
  model-ready datasets
  .
• Enable
  training
  ,
  validation
  , and
  inference
  data flows for AI/ML models.
• Integrate with ML platforms (SageMaker, MLflow, Vertex AI, etc.) and support various model data formats (Parquet, TFRecords).

ETL/ELT & Data Pipeline Development

• Lead development of
  ETL/ELT workflows
  using orchestration tools like
  Airflow, dbt, Prefect, or Luigi
  .
• Build reusable preprocessing frameworks ensuring performance, scalability, and automation.

Data Management & Governance

• Implement and maintain
  metadata repositories
  ,
  data catalogs
  ,
  lineage tracking
  , and
  data discovery
  tools.
• Build frameworks for
  data validation
  ,
  anomaly detection
  , and
  quality scoring
  .
• Enforce governance, lineage, privacy, and security protocols (
  GDPR, HIPAA
  , etc.).

Collaboration & Business Enablement

• Partner with
  Data Scientists
  ,
  ML Engineers
  ,
  Analysts
  , and
  Product teams
  to align solutions with business needs.
• Translate analytical and research requirements into
  production-ready pipelines
  .

Operational Excellence

• Monitor and optimize
  Snowflake
  data warehouse performance and schema design.
• Write and tune advanced
  SQL
  queries for complex transformations and reporting.
• Maintain
  high data quality
  and observability across systems.

Bonus Areas

• Build interactive dashboards using
  Power BI
  or
  Tableau
  .
• Automate infrastructure provisioning with
  Terraform
  or other IaC tools.
• Contribute to
  predictive modeling pipelines
  and collaborate on ML models in production.

Required Technical Skills

Data Engineering & Development

• Advanced
  SQL
  and deep understanding of
  data modeling
  (star/snowflake schemas, dimensional models).
• Experience in ETL/ELT pipeline development using
  Python
  or equivalent.
• Hands-on expertise in
  Snowflake
  and familiarity with columnar storage systems.

Big Data & Distributed Systems

• Strong experience with
  Apache Spark / PySpark
  ,
  Kafka
  ,
  Hadoop/Hive
  , and
  Flink/Beam
  .

Cloud & Storage Platforms

• Expertise in one or more:
• AWS
  (Glue, S3, Redshift, EMR, Lambda)
• GCP
  (BigQuery, Dataflow, Cloud Composer)
• Azure
  (Data Factory, Synapse Analytics, Data Lake)

Machine Learning & AI Readiness

• Familiarity with
  feature stores
  ,
  data versioning
  , and ML-centric pipelines.
• Understanding of
  model latency vs. data freshness
  tradeoffs.

Key Competencies

• Architectural Thinking:
  Ability to define scalable, modular, and reusable systems from scratch.
• ML-First Mindset:
  Understands how data impacts model performance & business decisions.
• Data Quality Focus:
  Ensures trustworthiness, observability, and validation of data.
• Leadership & Collaboration:
  Leads discussions and influences across technical & business teams.
• Problem Solving:
  Expert at debugging large-scale systems & automating resilient workflows.
• Compliance Awareness:
  Understands privacy laws & builds systems for
  responsible AI
  .

Qualifications

• Bachelor's or Master's degree in
  Computer Science
  ,
  Information Systems
  , or related field.
• 4–6+ years
  of hands-on experience in
  data engineering
  with leadership responsibilities.
• Strong programming skills in
  Python
  ,
  SQL
  , and optionally
  Java/C++
  .
• Proven experience with
  cloud data platforms
  ,
  big data ecosystems
  , and
  ML infrastructure
  .
• (Preferred) Experience with
  Power BI
  ,
  Tableau
  , and
  Terraform
  .