Big Data Integration Specialist

1 day ago

Islamabad, Islamabad, Pakistan beBee Careers Full time

About the Opportunity

We are seeking a highly skilled Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining high-performance, large-scale complex data architectures using Python, SQL, PySpark, Azure, Databricks, Synapse, Azure Data Lake, DevOps, and cloud-based large-scale data applications.

Your primary focus will be on architecting, designing, developing, testing, and maintaining big data products, constructing and fine-tuning Apache Spark jobs and clusters within the Databricks platform, migrating out of Azure Synapse to Azure Data Lake or other technologies, and evaluating and implementing new technologies and tools to improve data integration, data processing, storage, and analysis.

To succeed in this role, you will need a strong background in data engineering with expertise in Databricks, Azure, DevOps, and experience with cloud-based large-scale data applications. You will also be skilled in programming languages such as Python, Scala, and proficient in writing efficient and optimized code for data integration, migration, storage, processing, and manipulation.

A strong understanding of big data principles, techniques, and best practices is essential, along with experience with scalable and distributed Data Processing Technologies such as Spark/PySpark, DBT, and Kafka. You will also be skilled in designing and implementing efficient ELT/ETL processes in Azure and Databricks and using open-source solutions being able to develop custom integration solutions as needed.

Responsibilities

  • Architect, design, develop, test, and maintain high-performance, large-scale, complex data architectures.
  • Contribute to detailed design, architectural discussions, and customer requirements sessions.
  • Actively participate in the design, development, and testing of big data products.
  • Construct and fine-tune Apache Spark jobs and clusters within the Databricks platform.
  • Migrate out of Azure Synapse to Azure Data Lake or other technologies.
  • Assess best practices and design schemas that match business needs for delivering a modern analytics solution.

  • Big Data Quality Assurance Specialist

    17 hours ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Job Description:We are looking for a skilled Big Data Quality Assurance Specialist to participate in our team's data transformation efforts. As a key member, you will be involved in testing sprint tickets, new features, and providing progress updates on assigned tasks.The ideal candidate will have a strong focus on Big Data testing, particularly in Data Lake...

  • Big Data Specialist

    2 weeks ago

    Islamabad, Islamabad, Pakistan Clustox Inc. Full time

    Data Engineering RoleWe are a mission-driven organization building an innovative cloud-based platform to combat coral reef degradation. As a Senior Data Engineer, you will play a critical role in designing and optimizing data systems that power our conservation efforts.Your responsibilities will include creating scalable ETL/ELT pipelines using Azure...

  • Big Data Engineer

    2 weeks ago

    Islamabad, Islamabad, Pakistan Evamp & Saanga Full time

    As a Big Data Engineer specializing in Java development, you will play a crucial role in designing, developing, and implementing large-scale data processing systems. Your primary focus will be on leveraging big data technologies to build robust and efficient solutions for managing and analyzing vast amounts of data. You will collaborate with cross-functional...

  • Lead Big Data Engineer

    24 hours ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    About the PositionWe are seeking a Lead Big Data Engineer to join our team. The successful candidate will be responsible for architecting, designing, and developing large-scale data systems using big data technologies such as Apache Spark and Azure Databricks.The ideal candidate will have strong experience with cloud-based large-scale data applications and...

  • Data Integration Specialist

    5 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Key Responsibilities:The Senior Data Engineer will be responsible for meeting with stakeholders to understand the big picture and goals, recommending architecture aligned with the product/organization's objectives, participating in detail design and architectural discussions, and collaborating with developers on the team to meet product deliverables....

  • Big Data Expert

    8 hours ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    About the RoleWe are seeking a highly skilled and experienced Big Data Engineer to join our team. As a Big Data Engineer, you will play a crucial role in designing, developing, and implementing large-scale data processing systems.Your primary focus will be on leveraging big data technologies to build robust and efficient solutions for managing and analyzing...

  • Big Data Quality Assurance Automation Specialist

    1 week ago

    Islamabad, Islamabad, Pakistan Confiz Full time

    Confiz is a leading enterprise with a vibrant culture of learning and collaboration.About the RoleWe are seeking a highly skilled Big Data Quality Assurance Automation Specialist to join our team. The successful candidate will be responsible for designing, developing, and maintaining automated testing frameworks for our big data projects.The ideal candidate...

  • Data Integration Specialist

    2 hours ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    About the Role:We are seeking a skilled Data Integration Specialist to join our team. As a key member, you will be responsible for designing, building, and deploying data integration solutions using Informatica Cloud.Our ideal candidate will have hands-on experience in developing data integration workflows, writing and optimizing SQL queries, and working...

  • Data Automation Expert for Big Data Projects

    3 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Job Description:The purpose of this role is to have a Quality Assurance (QA) expert who can participate in day-to-day QA activities such as sprint calls, providing updates on assigned tickets, and improving knowledge in the relevant domain. This individual will be responsible for creating and identifying regression testing suites to remove manual effort from...

  • Big Data Professional

    1 week ago

    Islamabad, Islamabad, Pakistan ieng Group Full time

    ieng Group is a cutting-edge technology company that focuses on delivering innovative data engineering solutions. We are currently seeking a talented Big Data Expert to join our team and contribute to the development of our data-driven products.