Lead Data Engineer

We are seeking a Lead Data Engineer to own the design, development, and management of data pipelines, orchestration, and cloud operations for all data initiatives. In this role, you will work closely with cross-functional teams to ensure the seamless integration of data across platforms, with a strong focus on both traditional analytics and our AI-driven data initiatives. The ideal candidate will have extensive experience in data engineering, cloud platforms, and machine learning operations, and will bring strong leadership and technical expertise to help scale our data infrastructure.

Key Responsibilities:

1. Design and Architecture: Lead the design and architecture of cloud-based data solutions and data pipelines to support a range of analytics and AI applications.
2. Cloud Infrastructure: Engineer and manage cloud-based data infrastructure, primarily within Google Cloud Platform (GCP), but experience with other cloud platforms (AWS, Azure) is a plus.
3. Data Pipelines: Build and manage proprietary data pipelines from first-party and third-party platforms, ensuring data quality, scalability, and efficiency.
4. Machine Learning Operations (MLOps): Oversee the ingestion, monitoring, and serving of data required for machine learning operations.
5. CI/CD Automation: Implement and manage automated CI/CD pipelines to ensure fast and efficient deployment of data engineering solutions.
6. Cross-Functional Collaboration: Work closely with data scientists, software engineers, and product teams to ensure data is easily accessible, accurate, and aligned with business goals.
7. Cloud Data Warehousing: Administer and optimize cloud data warehouses, ensuring they are scalable and optimized for both real-time and batch processing.
8. Vendor Management: Collaborate with third-party software vendors and data providers to manage integrations and data products.
9. Leadership & Mentorship: Provide technical leadership and mentorship to junior engineers and foster a collaborative team environment.
10. Innovation: Proactively identify new technologies, processes, and opportunities to improve data infrastructure and drive business outcomes.
11. Product Lifecycle: Contribute to the full product lifecycle, from ideation to deployment, including taking products from initial concept (zero to one) to scaling and maintaining them (one to ten and beyond).

Required Qualifications:

1. 10+ years of experience in Data Engineering and Cloud Engineering, with at least 2 years in a senior technical role.
2. Strong experience with cloud architecture and cloud engineering, preferably GCP (Google Cloud Platform). Familiarity with AWS or Azure is a plus.
3. Advanced proficiency in SQL and Python for data manipulation, ETL, and automation.
4. Extensive experience in Cloud Data Warehousing solutions, such as BigQuery, Redshift, or Snowflake.
5. Proven track record in building and managing proprietary data pipelines for both structured and unstructured data.
6. Foundational understanding of Machine Learning Operations (MLOps), including managing data ingestion, monitoring, and serving data for machine learning models.
7. Experience in developing and implementing CI/CD pipelines for data products.
8. Hands-on experience with the architecture, engineering, and administration of commercial software platforms.
9. Ability to manage third-party software vendors, ensuring data product integration aligns with business objectives.
10. Strong problem-solving skills with the ability to think creatively and proactively in ambiguous environments.
11. Excellent communication and collaboration skills, with a strong ability to work across teams and drive alignment on technical solutions.
12. Passion for innovation and willingness to step outside of your comfort zone to solve complex challenges.
13. Experience in the ecommerce and digital marketing industry including the data available from third-party APIs such as Clickstream, Shopify, Google Analytics, Meta, TikTok, Google Ads, etc.

Desired Skills & Experience:

1. Experience with data orchestration tools (e.g., Apache Airflow, etc.).
2. Familiarity with containerization (e.g., Docker) and Kubernetes for managing scalable data workloads.
3. Background in software engineering practices and agile methodologies.
4. Knowledge of data security and compliance best practices in cloud environments.
5. Exposure to AI/ML frameworks such as TensorFlow, PyTorch, or scikit-learn.