Data Pipeline Specialist

2 days ago

Islamabad, Islamabad, Pakistan beBee Careers Full time

Data Pipeline Specialist Role

">

The ideal candidate should have a strong background in data engineering and architecture, with experience in designing and implementing scalable data pipelines. The role involves collaborating with cross-functional teams to develop, deploy, and operate ETL pipelines, ensuring data quality, monitoring, and setting up alerts, managing CI/CD, governance, and access control.

">Key Responsibilities
  • Develop, construct, test, and maintain optimal data pipeline architecture.
  • Assemble large, complex data sets meeting functional and non-functional business requirements.
  • Improve data reliability, efficiency, and quality.
  • Prepare data for predictive and prescriptive modeling.
  • Identify automation opportunities within data tasks.
  • Build, deploy, and operate ETL pipelines, ensure data quality, monitor, and set up alerts, manage CI/CD, governance, and access control.
  • Design and implement scalable data pipelines for ingestion, transformation, and storage using Snowflake and other technologies.
  • Stay updated on industry best practices and emerging technologies related to data architecture and Snowflake/AWS.
  • Enhance internal processes by automating manual tasks, optimizing data delivery, and redesigning infrastructure for scalability.
  • Monitor process performance and recommend improvements.
  • Collaborate with stakeholders across executive, product, data, and design teams to support data infrastructure needs.
  • Create data tools to aid analytics and product development.
  • Work with data and analytics experts to improve data system functionality.
  • Demonstrate proficiency in data management and automation in Spark, Hadoop, HDFS environments.
  • Knowledge of data science/machine learning, analytics, or data warehousing is a plus.

Required Skills

  • Excellent communication skills.
  • Experience with Snowflake, Hadoop, Hive, Spark, Airflow, Livy, Scala, Java.
  • Experience with AWS EMR (HDFS, S3, HBase), Athena, PySpark or related technologies.
  • Experience with streaming technologies like Kafka.
  • Proficiency in programming languages such as Scala, Java, SQL, Python, R.
  • Experience managing data in relational databases and developing ETL pipelines.
  • Exposure to enterprise services like Cloudera, Databricks, AWS, SSIS, SQL.
  • Knowledge of AWS data services such as EC2, EMR, Kinesis, Lambda, DynamoDB is advantageous. Informatica experience is a plus.
  • Relevant certifications in Snowflake, AWS, or data architecture are highly desirable.

  • Data Pipeline Support Specialist

    2 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Data Pipeline Support RoleThis is a challenging opportunity for an experienced AWS Data Engineer to join our team in providing technical support for production data pipelines. You will be responsible for monitoring, troubleshooting, and optimizing AWS-based data pipelines.The ideal candidate will have hands-on experience with AWS Glue, Lambda, Step...

  • Data Engineer

    2 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Job Title: Data Engineer – Pipeline Specialist

  • Senior Data Pipeline Specialist

    2 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    About the Role:This is an exciting opportunity for a Senior Data Engineer to join our team and contribute to the development of our data infrastructure. As a key member of the engineering team, you will be responsible for designing, developing, and maintaining data pipelines that ingest, validate, and transform telemetry and audio signature data.You will...

  • Data Pipeline Support Specialist

    2 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    **Key Responsibilities**Monitor, troubleshoot, and support AWS data pipelines using any two of AWS Glue, Lambda, Step Functions, Athena, Redshift, or Airflow.Investigate job failures, analyze CloudWatch logs, and collaborate with L2/L3 teams to implement corrective actions.Re-run failed jobs and retry data ingestion processes while ensuring data...

  • Data Engineer

    2 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Job DescriptionWe are seeking a highly motivated Associate Software Engineer – Data Engineering to fill this key role. This position is ideal for recent graduates or early-career professionals passionate about data engineering, big data technologies, and building scalable data pipelines.Core Responsibilities:Assist in designing, developing, and maintaining...

  • Senior Data Pipeline Developer

    2 weeks ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    About the JobThis role involves working on a high-profile project that requires designing, building, and maintaining large-scale data processing systems. As a member of the data and analytics team, you will work closely with other professionals to develop and implement cutting-edge technologies.Key Responsibilities:Develop and maintain data pipelines using...

  • Pipeline Design Engineer

    1 week ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Key responsibilities of this position include designing and developing ETL/ELT data pipelines, implementing data models optimized for performance and scalability, working with Oracle Data Integrator (ODI) to automate data integration processes, ensuring data quality and integrity, optimizing SQL queries and database performance, collaborating with...

  • Data Pipeline Architect

    2 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    ResponsibilitiesDesign and develop AI system architecture that is fully integrated into our predictive inventory planning platformLead the deployment of AI models and algorithms on cloud platforms, primarily Azure, ensuring efficient, secure, and scalable systemsDevelop and optimize APIs to integrate AI models into our inventory planning tools, ensuring...

  • Data Pipeline Tester

    2 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Data transformation is underway to ensure accurate data availability. As a QA professional, your role will involve participating in testing activities, assessing changes in data workflows, and maintaining test cases.Main ResponsibilitiesDevelop test scripts for data pipeline validation, transformation, and integration.Maintain test strategies including...

  • Big Data Specialist

    7 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    This is a unique opportunity to work on international projects with a diverse clientele and to contribute to the growth and success of our company. As a Senior Data Engineer, you will design, develop, and deploy data pipelines, transform and enrich data, and ensure the quality and accessibility of data for stakeholders across the...