Senior Cloud-Based Data Pipelines Developer

2 days ago

Karachi, Sindh, Pakistan beBeeDataEngineer Full time 1,200,000 - 1,800,000
Senior Data Engineer Position

We are seeking a highly skilled and experienced Senior Data Engineer to join our team. This role demands a strong understanding of end-to-end data pipelines, data modeling, and advanced data engineering practices across diverse data sources and environments.

The successful candidate will play a pivotal role in building, deploying, and optimizing data infrastructure and pipelines in a scalable cloud-based architecture.

Main Responsibilities:

  • Design, develop, and maintain large-scale data pipelines using modern big data technologies and cloud-native tools.
  • Build scalable and efficient distributed data processing systems using Hadoop, Spark, Hive, and Kafka.
  • Work extensively with cloud platforms (preferably AWS) and services like EMR, Glue, Lambda, Athena, S3.
  • Design and implement data integration solutions pulling from multiple sources into a centralized data warehouse or data lake.
  • Develop pipelines using DBT (Data Build Tool) and manage workflows with Apache Airflow or Step Functions.
  • Write clean, maintainable, and efficient code using Python, PySpark, or Scala for data transformation and processing.
  • Build and manage relational and columnar data stores such as PostgreSQL, MySQL, Redshift, Snowflake, HBase, ClickHouse.
  • Implement CI/CD pipelines using Docker, Jenkins, and other DevOps tools.
  • Collaborate with data scientists, analysts, and other engineering teams to deploy data models into production.
  • Drive data quality, integrity, and consistency across systems.
  • Participate in Agile/Scrum ceremonies and utilize JIRA for task management.
  • Provide mentorship and technical guidance to junior team members.
  • Contribute to continuous improvement by making recommendations to enhance data engineering processes and architecture.

Required Skills & Experience:

  • 5+ years of hands-on experience as a Data Engineer.
  • Deep knowledge of Big Data technologies – Hadoop, Spark, Hive, Kafka.
  • Expertise in Python, PySpark and/or Scala.
  • Proficient with data modeling, SQL scripting, and working with large-scale datasets.
  • Experience with distributed storage like HDFS and cloud storage (e.g., AWS S3).
  • Hands-on with data orchestration tools like Apache Airflow or StepFunction.
  • Experience working in AWS environments with services such as EMR, Glue, Lambda, Athena.
  • Familiarity with data warehousing concepts and experience with tools like Redshift, Snowflake (preferred).
  • Exposure to tools like Informatica, AbInitio, Apache Iceberg is a plus.
  • Knowledge of Docker, Jenkins, and other CI/CD tools.
  • Strong problem-solving skills, initiative, and a continuous learning mindset.

Preferred Qualifications:

  • Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.
  • Experience with open table formats such as Apache Iceberg.
  • Hands-on with AbInitio (GDE, Collect > IT) or Informatica tools.
  • Knowledge of Agile methodology, working experience in JIRA.

Soft Skills:

  • Self-driven, proactive, and a strong team player.
  • Excellent communication and interpersonal skills.
  • Passion for data and technology innovation.
  • Ability to work independently and manage multiple priorities in a fast-paced environment.

Benefits:

This role offers the opportunity to work with a talented team, develop your skills, and contribute to the success of our organization.

How to Apply:

If you are a motivated and experienced professional looking to take on new challenges, please submit your application.

],

  • Chief Data Pipeline Architect

    11 hours ago

    Karachi, Sindh, Pakistan beBeeDataEngineering Full time 1,500,000 - 2,500,000

    Senior Data Engineering Specialist Job Description   We are seeking a highly skilled Senior Data Engineering Specialist to design, develop and optimize high-performance data pipelines and ensure seamless integration of cloud platforms.Key Responsibilities:Develop and maintain efficient data pipelines using Informatica Cloud technology to deliver high...

  • Senior Data Pipeline Architect

    11 hours ago

    Karachi, Sindh, Pakistan beBeeDataEngineering Full time 1,500,000 - 2,000,000

    Job OverviewWe are seeking a highly skilled Senior Data Engineer to join our team. The ideal candidate will have hands-on expertise in building robust data pipelines, working with large-scale datasets, and leveraging modern Big Data and cloud technologies.The successful candidate will be responsible for designing and developing complex ETL pipelines and data...

  • Highly Skilled Senior Data Pipeline Developer

    11 hours ago

    Karachi, Sindh, Pakistan beBeeDataEngineer Full time 2,000,000 - 2,500,000

    Key Senior Data Engineer RoleWe are seeking a highly skilled senior data engineer to develop and maintain complex data pipelines and solutions using big data technologies.Job Description:The successful candidate will be responsible for designing, developing, and implementing scalable data workflows using cloud-native technologies. This role requires...

  • Data Engineering

    4 weeks ago

    Karachi, Sindh, Pakistan Xloop Digital Full time

    Senior Data Engineer Karachi Full timeRemoteWe are seeking an experienced Senior Data Engineer with strong expertise in Informatica Cloud to joinour data engineering team. In this role, you will be responsible for designing, developing, and optimizinghigh-performance data pipelines and ensuring seamless data integration across cloud platforms. Yourtechnical...

  • Data Migration Developer

    4 weeks ago

    Karachi, Sindh, Pakistan Virtej Technologies Full time

    Data Migration Developer (Azure | C#) - Reloaction Job to Malaysia15 hours ago - Be among the first 25 applicantsGet AI-powered advice on this job and more exclusive features.Type: Contract (Renewable)Budget: RM 15,000 to 16,000Experience: 8-10 yearsAre you passionate about solving complex data challenges and driving seamless migrations in the cloud? We're...

  • Chief Data Architect

    11 hours ago

    Karachi, Sindh, Pakistan beBeeDataEngineer Full time 1,500,000 - 2,500,000

    Job Title: Data EngineerOverview of the RoleWe are seeking a skilled professional to design, develop, and maintain large-scale data infrastructure and pipelines in a cloud-based architecture.Main ResponsibilitiesData Pipeline Development: Design and build efficient distributed data processing systems using modern big data technologies and cloud-native...

  • Data Engineer

    2 weeks ago

    Karachi, Sindh, Pakistan NorthBay - Pakistan Full time

    Job Title: Data EngineerJob Title: Data EngineerLocation: Karachi, Lahore , Islamabad (Hybrid)Experience: 5+ YearsJob Type: Full-TimeJob OverviewWe are looking for a highly skilled and experienced Data Engineer with a strong foundation in Big Data, distributed computing, and cloud-based data solutions. This role demands a strong understanding of end-to-end...

  • Senior ETL Data Engineer

    3 weeks ago

    Karachi, Sindh, Pakistan NorthBay - Pakistan Full time

    3 weeks ago Be among the first 25 applicantsGet AI-powered advice on this job and more exclusive features.Job Title: Senior ETL Data Engineer (Ab Initio / Informatica)Experience: 4+ YearsLocation: Lahore / Karachi / Islamabad (Hybrid)Job Type: Full-TimeAbout The RoleWe are looking for a highly skilled Senior ETL Data Engineer (Ab Initio / Informatica) with...

  • Senior ETL Data Engineer

    4 days ago

    Karachi, Sindh, Pakistan NorthBay Solutions Full time

    Job Title: Senior ETL Data Engineer (Ab Initio / Informatica)Experience: 4+ YearsLocation: Lahore / Karachi / Islamabad (Hybrid)Job Type: Full-TimeAbout the Role:We are looking for a highly skilled Senior ETL Data Engineer (Ab Initio / Informatica) with strong experience in building robust data pipelines, working with large-scale datasets, and leveraging...

  • Cloud Infrastructure Architect

    11 hours ago

    Karachi, Sindh, Pakistan beBeeStrategist Full time $100,000 - $120,000

    Infrastructure StrategistThe ideal candidate will be responsible for designing, implementing, and managing our infrastructure and deployment pipelines to drive business growth and efficiency.Developing and maintaining CI/CD pipelines using tools like Jenkins, Docker, and Kubernetes to ensure seamless application delivery.Deploying containerized microservices...