L2 Data Pipeline Expert

3 days ago


Karachi, Sindh, Pakistan NorthBay Solutions Full time
Senior AWS Data Pipeline Specialist

We are seeking an experienced Senior AWS Data Pipeline Specialist to join our team at NorthBay Solutions. As a key member of our data engineering team, you will design, build, and maintain scalable, efficient, and reliable data pipelines using Amazon Web Services (AWS). Your primary responsibilities will include monitoring and troubleshooting data pipelines, investigating job failures, and collaborating with cross-functional teams to implement corrective actions. You will also be responsible for re-running failed jobs, retrying data ingestion processes, and maintaining documentation of troubleshooting steps, common issues, and resolutions. In addition, you will participate in incident response and recovery procedures as needed and provide technical support for production data pipelines, ensuring service level agreements (SLAs) are met.

Key Responsibilities
  1. Design, build, and maintain scalable, efficient, and reliable data pipelines using AWS services.
  2. Monitor and troubleshoot data pipelines using AWS Glue, Lambda, Step Functions, Athena, Redshift, or Airflow.
  3. Investigate job failures, analyze CloudWatch logs, and collaborate with L2/L3 teams to implement corrective actions.
  4. Re-run failed jobs and retry data ingestion processes while ensuring data consistency.
  5. Maintain documentation of troubleshooting steps, common issues, and resolutions.
  6. Participate in incident response and recovery procedures as needed.
  7. Provide technical support for production data pipelines, ensuring SLAs are met.

Requirements
  • Bachelor's degree in Computer Science, Data Science, Information Technology, or a related field.
  • Up to 2+ years of experience in Data Processing / ETL.
  • Hands-on experience with any two of AWS Glue, Athena, Lambda, Step Functions, Redshift.
  • Proficiency in Python and SQL for data querying and processing (preferred).
  • Familiarity with PySpark and ETL development (a plus).
  • Experience in monitoring and debugging AWS Glue jobs, Step Functions, and Lambda logs (a plus).
  • Strong analytical and problem-solving skills to diagnose and resolve data processing failures.
  • Ability to work independently while effectively collaborating with cross-functional teams.
  • AWS certifications such as AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect Associate (preferred but not mandatory).
  • Excellent communication and interpersonal skills.
  • Prior experience in L1/L2 support (a plus).

Career Progression
  • Senior Data Engineer (L2/L3 Support / ETL Development)
  • Big Data Engineer / AWS Data Architect
  • Data Engineering Lead / Architect

  • Data Engineer

    3 days ago


    Karachi, Sindh, Pakistan NorthBay Solutions Full time

    Job OverviewWe are seeking a highly skilled Data Engineer - L2 Support to join NorthBay Solutions. This role involves monitoring, troubleshooting, and optimizing AWS-based data pipelines while ensuring data integrity and pipeline continuity. The successful candidate will work with various AWS data services, including Glue, Lambda, Step Functions, Athena,...


  • Karachi, Sindh, Pakistan NorthBay Solutions Full time

    We are seeking a Level 2 AWS Data Engineer to join our team. This role involves monitoring, troubleshooting, and optimizing AWS-based data pipelines while ensuring data integrity and pipeline continuity. You will work with AWS Glue, Lambda, Step Functions, and other AWS data services to investigate job failures, re-run ETL jobs, and support production...


  • Karachi, Sindh, Pakistan NorthBay Solutions LLC Full time

    Job OverviewWe are seeking a Level 2 AWS Data Engineer to join our team. This role involves monitoring, troubleshooting, and optimizing AWS-based data pipelines while ensuring data integrity and pipeline continuity. You will work with AWS Glue, Lambda, Step Functions, and other AWS data services to investigate job failures, re-run ETL jobs, and support...

  • Data Architect

    4 days ago


    Karachi, Sindh, Pakistan NorthBay Solutions LLC Full time

    About the JobWe are seeking a highly skilled senior ETL developer to join our team at NorthBay Solutions LLC. This role involves designing, developing, and maintaining efficient ETL processes using AWS Glue, Lambda, and Step Functions.You will work closely with data architects and L2/L3 engineers to develop and implement data pipelines that ensure data...


  • Karachi, Sindh, Pakistan NorthBay Solutions Full time

    Job DescriptionAs a Senior AWS Data Processor at NorthBay Solutions, you will play a critical role in the success of our team by designing, building, and maintaining scalable, efficient, and reliable data pipelines using Amazon Web Services (AWS). Your primary responsibilities will include monitoring and troubleshooting data pipelines, investigating job...


  • Karachi, Sindh, Pakistan NorthBay Solutions Full time

    Data Engineer Job SummaryWe are seeking an experienced Data Engineer to join our team at NorthBay Solutions. As a key member of our data engineering team, you will design, build, and maintain scalable, efficient, and reliable data pipelines using Amazon Web Services (AWS). Your primary responsibilities will include monitoring and troubleshooting data...

  • Cloud Data Engineer

    4 days ago


    Karachi, Sindh, Pakistan NorthBay Solutions LLC Full time

    Job OverviewWe are seeking a skilled AWS data engineer to join our team at NorthBay Solutions LLC. This role involves monitoring, troubleshooting, and optimizing AWS-based data pipelines while ensuring data integrity and pipeline continuity.You will work with AWS Glue, Lambda, Step Functions, and other AWS data services to investigate job failures, re-run...


  • Karachi, Sindh, Pakistan BlueBerry Tech inc Full time

    Required Skills and QualificationsDatabase Proficiency: Proficiency in SQL and NoSQL databases.Data Pipeline Tools: Experience with data pipeline and ETL tools (e.g., Apache Kafka, Airflow).Programming Languages: Strong programming skills in Python, Java, or Scala.Cloud Platforms: Familiarity with cloud platforms (AWS, Azure, GCP) and big data technologies...


  • Karachi, Sindh, Pakistan Horizon Technologies Full time

    Horizon Technologies is looking for an experienced Sales Pipeline Management Expert to optimize our sales performance and drive business growth. As a key member of our team, you will be responsible for analyzing and interpreting complex data sets to inform strategic decisions.The ideal candidate will have expertise in CRM management, sales reporting, and...


  • Karachi, Sindh, Pakistan NorthBay Solutions LLC Full time

    Job DescriptionWe are seeking an experienced AWS data pipeline specialist to join our team at NorthBay Solutions LLC. This role involves monitoring, troubleshooting, and optimizing AWS-based data pipelines while ensuring data integrity and pipeline continuity.You will work with AWS Glue, Lambda, Step Functions, and other AWS data services to investigate job...


  • Karachi, Sindh, Pakistan Cloudlit Full time

    About the Role:">We are seeking an experienced ML Operations Pipeline Developer to join our team. In this role, you will design and implement efficient machine learning operations pipelines using industry-standard tools such as MLflow, Kubernetes, and Pachyderm.">Main Responsibilities:">Pipeline Development:Develop and maintain efficient machine learning...


  • Karachi, Sindh, Pakistan Gymboree Play and Music Full time

    ResponsibilitiesThe Data Engineer will be responsible for assembling large, complex sets of data that meet non-functional and functional business requirements by designing and implementing ETL/ELT architecture. Additionally, they will collaborate with the data governor to implement source data into the designed target model.Key RequirementsBachelor's degree...


  • Karachi, Sindh, Pakistan 10Pearls, LLC Full time

    Role OverviewWe are seeking an exceptional Data Science Solutions Expert to join our team. The ideal candidate will have expertise in .NET, React, Azure, and Data Science and be able to design and develop scalable, data-driven web applications while integrating advanced data science techniques.Main ResponsibilitiesLead the development of full-stack...

  • Data Engineer

    2 days ago


    Karachi, Sindh, Pakistan Office Field Full time

    Direct message the job poster from Office FieldHuman Resources Executive at Office Field | HR Policies Implementation, Tech RecruitingOffice Field is looking for a Data Engineer to join our growing team of analytics experts. They will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and...

  • Data Architect

    2 days ago


    Karachi, Sindh, Pakistan Office Field Full time

    About Office FieldOffice Field is a leading IT services and consulting company that aims to provide innovative solutions for businesses. We are dedicated to delivering high-quality services and building long-term relationships with our clients.Job DescriptionWe are seeking an experienced Data Engineer to join our team of analytics experts. As a key member of...

  • Big Data Expert

    1 week ago


    Karachi, Sindh, Pakistan Horizon Technologies Full time

    Job SummaryHiring for the position of Cloud Data Architect at Horizon Technologies!About UsHorizon Technologies is a cutting-edge technology firm that leverages innovative solutions to drive business growth. As a Cloud Data Architect, you will play a key role in shaping our cloud strategy and architecting scalable data platforms.About the RoleThis role...


  • Karachi, Sindh, Pakistan Ahmed Towel Full time

    Data Management Expert NeededAhmed Towel is seeking an experienced Data Management Expert to join its accounting team. The successful candidate will be responsible for managing and analyzing financial data, ensuring accuracy and integrity of financial records, and preparing reports and financial statements.Key qualifications include:Proficiency in Microsoft...


  • Karachi, Sindh, Pakistan Al Nafi International College Full time

    About this RoleWe are looking for an exceptional Senior Big Data Specialist/Analyst to lead our data strategy at Al Nafi International College. This individual will play a vital role in developing and implementing data-driven solutions that drive business growth and improvement.The successful candidate will have extensive experience in Big Data engineering,...


  • Karachi, Sindh, Pakistan Cloudtastic Private Limited. Full time

    We are seeking a highly skilled Microsoft 365 Technical Support Expert to join our team at Cloudtastic Private Limited. In this role, you will play a crucial part in managing our Managed Microsoft 365 Support Services, providing technical assistance and resolving Microsoft 365-related issues.Key ResponsibilitiesProvide L2/L3 support for Microsoft 365...


  • Karachi, Sindh, Pakistan Wenawa Software Inc. Full time

    We are looking for a highly skilled .NET product engineering expert to join our team.Key Responsibilities:You will be responsible for developing high-quality software solutions using .NET and cloud-based technologies.Design and implement robust .NET solutions for complex problems.Develop and maintain scalable data processing pipelines.Collaborate with...