Data Engineering Lead

13 hours ago


Lahore, Punjab, Pakistan Confiz Full time

We are seeking a passionate and experienced Data Engineering Lead to join our dynamic team, responsible for managing and optimizing our data lake platform. The ideal candidate will have extensive experience in data engineering, particularly in managing complex data pipelines and ensuring data quality, performance, and availability. This role requires a proactive leader who owns end-to-end operations of the platform and can drive solutions and improvements while effectively communicating with stakeholders.

Responsibilities:

  • Diagnose and resolve pipeline issues while implementing effective solutions to minimise outages, enhance performance, and increase platform resilience.
  • Implement and maintain monitoring solutions to assess platform performance, health, and data quality.
  • Establish and track KPIs to ensure timely data availability.
  • Collaborate with an engineering team to ensure the best practices for data ingestion, storage, and processing in Delta Lake.
  • Optimize resource usage and platform costs, ensuring a balance between performance and cost-effectiveness as the platform scales.
  • Drive initiatives for data quality assurance, proactively identifying and resolving issues before they impact stakeholders.
  • Manage deployments of pipeline changes using Azure DevOps, ensuring consistency and reliability through CI/CD best practices.
  • Identify and recommend enhancements to processes, tools, and technologies to improve operational efficiency.
  • Maintain regular updates with stakeholders, ensuring transparent communication of progress and challenges.

Requirements:

  • Bachelor's degree in Computer Science or a related field
  • 8+ years of experience in data engineering, with hands-on experience in:
  • Apache NiFi, Databricks, Delta Lake, Azure Synapse Analytics, Azure Data Lake Storage, Azure Data Factory, and Power BI.
  • Proven track record in managing and troubleshooting complex data pipelines, with a focus on performance improvement and operational efficiency.
  • Experience in setting up and integrating open-source tools for ETL and data visualization/analytics.
  • Experience in setting up and maintaining Kubernetes clusters for deploying and managing data applications.
  • Experience working in Agile/Scrum (SAFe) environments, participating in sprints, stand-ups, and cross-functional team collaboration.
  • Excellent problem-solving skills and the ability to think critically under pressure.
  • Strong communication skills, with a knack for presenting information to both technical and non-technical stakeholders.
  • Self-driven and accountable, with a proactive mindset and strong sense of ownership.
  • Microsoft Azure Data Engineer Associate (DP-203) or Databricks certification is preferred.
  • Prior experience working in the telecom industry or supporting large-scale, data-driven telecom platforms is preferred.

We have an amazing team of 700+ individuals working on highly innovative enterprise projects & products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast-growth fintech, and multiple Silicon Valley startups.

What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 (QMS), ISO 27001:2022 (ISMS), ISO 20000-1:2018 (ITSM) and ISO 14001:2015 (EMS) Certified. We have a vibrant culture of learning via collaboration and making workplace fun.

People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.

To know more about Confiz Limited, visit: Confiz LinkedIn

#J-18808-Ljbffr

  • Lahore, Punjab, Pakistan Data Wise Full time

    About the Position:Data Wise is looking for an experienced quality assurance engineer lead to join our dynamic team. The ideal candidate will have a solid background in software testing, with expertise in automation and quality engineering.Responsibilities:Collaborate with cross-functional teams to identify testing requirements, risks, and...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    At Data Pilot, we're more than just a data product development and consulting services company – we're your strategic partner on the journey from data to decisions. Our mission is simple: to help businesses unlock the full potential of their data assets and maximize business value through our expert engineering and business acumen.In the dynamic landscape...

  • Data Engineer Lead

    2 weeks ago


    Lahore, Punjab, Pakistan Devsinc Full time

    Job OverviewDevsinc is a cutting-edge technology company seeking an experienced Data Engineer to lead the development of our data infrastructure. As a key member of our team, you will be responsible for designing, implementing, and maintaining efficient data pipelines that enable seamless data flow from various sources to storage systems.ResponsibilitiesData...


  • Lahore, Punjab, Pakistan Data Wise Full time

    About the Team:Data Wise is looking for a software quality engineering manager to lead our quality assurance team. The ideal candidate will have a solid background in software testing, with expertise in automation and quality engineering.Responsibilities:Develop and implement automated test frameworks and scripts using Selenium and Playwright.Mentor and...

  • Data Engineering Lead

    4 hours ago


    Lahore, Punjab, Pakistan beBee Careers Full time

    About the RoleWe are seeking a passionate and experienced data engineering professional to manage and optimize our data lake platform.The ideal candidate will have extensive experience in data engineering, particularly in managing complex data pipelines and ensuring data quality, performance, and availability.This role requires a proactive leader who owns...

  • Data Engineer Lead

    6 days ago


    Lahore, Punjab, Pakistan NorthBay Solutions Full time

    We are looking for an experienced Senior Data Architect to join our team at NorthBay Solutions. As a key member of our data engineering group, you will be responsible for designing and developing cutting-edge big data technologies to build critical, highly complex distributed systems from scratch.About the Role:The Ideal Candidate Will Have: 4 to 6+ years of...


  • Lahore, Punjab, Pakistan Avanceon Limited Full time

    Are you ready to shape the future with your ML expertise and have fun while doing it?About the RoleWe're looking for a talented Machine Learning Engineer to join our team at Avanceon Limited. As a key member, you'll develop cutting-edge ML models, collaborate with Data Engineering to build efficient ETL architectures, and guide stakeholders through powerful...

  • Lead Engineer

    2 weeks ago


    Lahore, Punjab, Pakistan Descon Engineering Limited Full time

    Lead Engineer Descon Engineering Limited, Pakistan Responsibilities:Responsible for monitoring progress and quality of the design work as per project requirements.Monitors and ensures progress and quality of the design work carried out in-house or sub-contracted as per project requirements.Ensures the provision of all necessary inputs for proposal...


  • Lahore, Punjab, Pakistan LEAD Concept Full time

    LEAD Concept is looking for a skilled Software Engineer for Frontend Development. The ideal candidate will have a thorough understanding of ReactJS and be able to develop high-quality user interfaces.Job ResponsibilitiesCreate engaging user experiences using ReactJS.Built reusable components and front-end libraries for future use.Translate UI/UX design...

  • Data Engineer

    3 weeks ago


    Lahore, Punjab, Pakistan Strategic Systems International Full time

    JOB TITLE: Data EngineerDEPARTMENT: Software DevelopmentREPORTS To: Team Lead / Project ManagerPURPOSE:The Data Engineer will be responsible for supporting data engineering and cloud-related tasks, ensuring efficient data processing and troubleshooting in production environments. This role is ideal for individuals with a strong problem-solving mindset, a...


  • Lahore, Punjab, Pakistan Devsinc Full time

    We are seeking a talented Data Architecture Specialist to join our data engineering team and drive business growth through innovative data solutions. As a key member of our team, you will be responsible for designing, developing, and maintaining cutting-edge data processing systems and data pipelines.About the Role:Design and Develop Data Architectures:...

  • Engineering Team Lead

    2 weeks ago


    Lahore, Punjab, Pakistan Descon Engineering Limited Full time

    Job Title: Engineering Team LeadWe are seeking an experienced Engineering Team Lead to join our team at Descon Engineering Limited in Pakistan. This is a supervisory role that requires strong leadership and technical skills.About the RoleThe successful candidate will be responsible for monitoring progress and quality of design work as per project...

  • Data Engineer

    2 weeks ago


    Lahore, Punjab, Pakistan Systems Limited Full time

    Senior Data Engineer:Systems Limited is looking for a Senior Data Engineer with expertise in PySpark, Airflow, and Databricks to join our growing team.Responsibilities:Design, build, and maintain scalable data pipelines using PySpark for ETL/ELT processing, ensuring data is ingested, transformed, and loaded efficiently across multiple platforms.Use Apache...

  • Data Engineer

    4 weeks ago


    Lahore, Punjab, Pakistan Tkxel Full time

    Tkxel is a leading software development company located in Reston, Virginia. We are committed to developing innovative software solutions for leading enterprises in the world, helping them grow their businesses using the latest technology solutions.Job DescriptionWe are seeking a skilled and experienced Data Engineer with expertise in PySpark and Azure Data...

  • IT Test Engineer

    2 days ago


    Lahore, Punjab, Pakistan LEAD Concept Full time

    Job OverviewWe are seeking an experienced Software Tester Developer to join our IT team at LEAD Concept. The ideal candidate will have a strong background in software testing and development, with experience in automation tools and methodologies.Key ResponsibilitiesDevelop and execute comprehensive test plans to ensure software quality.Collaborate with...


  • Lahore, Punjab, Pakistan Tkxel Full time

    Tkxel OverviewWe are Tkxel, a leading software development company headquartered in Reston, Virginia. Our mission is to deliver innovative software solutions that help leading enterprises grow their businesses using the latest technology.Job DescriptionWe are seeking an experienced Data Engineer with expertise in PySpark and Azure Data Factory. The ideal...

  • Data Engineering Lead

    2 weeks ago


    Lahore, Punjab, Pakistan Arbisoft Full time

    About UsArbisoft is a dynamic and innovative company that thrives on delivering cutting-edge solutions. We're seeking a highly motivated Senior Data Engineer to join our team.Job DescriptionThe ideal candidate will have hands-on experience with Python, SQL, and AWS data services. This role involves designing, maintaining, and optimizing data pipelines,...

  • Cloud Data Engineer

    2 weeks ago


    Lahore, Punjab, Pakistan Devsinc, LLC Full time

    We are seeking a Senior Data Engineer to lead our data team. The successful candidate will have extensive experience in designing and implementing large-scale data architectures, as well as collaborating with cross-functional teams to ensure seamless data integration.Main Responsibilities:Data Architecture: Develop and implement data architectures that...

  • Data Engineer

    13 hours ago


    Lahore, Punjab, Pakistan PRG Pakistan Full time

    We are seeking a skilled Data Engineer with AI/ML expertise to design, build, and optimize scalable data pipelines that power machine learning models and AI-driven applications. You will work closely with data scientists, analysts, and software engineers to ensure efficient data ingestion, transformation, and storage—enabling real-time analytics and AI...


  • Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    We are seeking an experienced Data Engineering Leader to join our team at NorthBay Solutions LLC. As a Senior Big Data Engineer, you will have the opportunity to design and develop cutting-edge Big Data technologies, build critical distributed systems from scratch, and lead complex ETL jobs and pipelines.About the Role:Design and develop using Big Data...