Sr Big Data Engineer

7 days ago

Islamabad, Islamabad, Pakistan Datamatics Technologies Full time

Direct message the job poster from Datamatics Technologies

Talent Acquisition Lead | Global Technical Recruitment | Technical Recruitment | EMEA & MENA

Job Summary:

We are looking for a highly skilled Big Data Engineer to spearhead the development of a scalable Big Data Platform. The ideal candidate will have extensive experience with Kafka, Flink, Spark, Python, and Talend, along with expertise in Apache Ranger, Prometheus, and strong SQL skills. This role involves designing and implementing data pipelines, ensuring security, monitoring system performance, and collaborating with cross-functional teams to deliver high-performance data solutions.

Key Responsibilities:

  1. Design, develop, and optimize a Big Data Platform leveraging Kafka, Flink, Spark, and Talend.
  2. Architect and implement real-time and batch data processing pipelines.
  3. Ensure secure data access and governance using Apache Ranger.
  4. Monitor and optimize system performance with Prometheus.
  5. Develop and maintain ETL workflows and data ingestion pipelines.
  6. Write efficient and scalable SQL queries for data transformation and analysis.
  7. Collaborate with Data Scientists, Analysts, and DevOps teams to align data infrastructure with business goals.
  8. Implement data security, compliance, and access control policies.
  9. Troubleshoot and resolve issues related to data processing, storage, and system performance.

Required Skills & Qualifications:

  1. 5+ years of experience in Big Data Engineering.
  2. Hands-on expertise in Kafka, Flink, Spark, Python, and Talend.
  3. Strong proficiency in SQL, with experience in query optimization and performance tuning.
  4. Familiarity with Apache Ranger for data security and access control.
  5. Familiarity with Prometheus for monitoring and performance optimization.
  6. Deep understanding of distributed computing, data partitioning, and pipeline orchestration.
  7. Experience with cloud-based big data solutions (AWS, Azure, or GCP) is a plus.
  8. Strong problem-solving and analytical skills.
  9. Excellent communication and leadership abilities.

Preferred Qualifications:

  • Experience with Docker, Kubernetes, and containerized big data deployments.
  • Knowledge of workflow orchestration tools like Airflow.
  • Hands-on experience with NoSQL databases (e.g., Cassandra, HBase).
Seniority level

Mid-Senior level

Employment type

Full-time

Job function

Engineering and Information Technology

Industries

IT Services and IT Consulting

Referrals increase your chances of interviewing at Datamatics Technologies by 2x

#J-18808-Ljbffr

  • Senior IT Director

    2 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    As a Sr. Executive IT, you will be responsible for designing and developing ETL pipelines, investigating unusual patterns, and fixing data quality issues. You will also provide operational support and lead data engineering initiatives as a technical lead for big data platform projects.Eligibility Criteria:Bachelor's degree or above in Computer...

  • Big Data Engineering Expert

    3 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    A Senior Big Data Engineer is required to design and develop complex ETL jobs and pipelines using Big Data technologies. The ideal candidate will have over 4 years of experience in Data Engineering, with expertise in designing and developing large-scale ETL pipelines using tools such as Python, Hadoop, Spark, and Hive.Requirements:Strong software development...

  • Senior Big Data Engineer

    4 weeks ago

    Islamabad, Islamabad, Pakistan NorthBay Solutions Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a...

  • Senior Big Data Engineer

    3 weeks ago

    Islamabad, Islamabad, Pakistan NorthBay Solutions Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a...

  • Big Data Architect

    1 week ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Job TitleA Big Data Engineer plays a crucial role in designing, developing, and implementing large-scale data processing systems.

  • Big Data Systems Developer

    3 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Job OverviewAs a Big Data Engineer, you will play a key role in designing, developing, and implementing large-scale data processing systems. Your primary focus will be on leveraging big data technologies to build robust and efficient solutions for managing and analyzing vast amounts of data.

  • Executive Leader

    3 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Job Title: Sr. Executive IT Job DescriptionWe are seeking a highly skilled and experienced IT professional to join our team as Sr. Executive IT. The successful candidate will be responsible for leading data engineering initiatives, managing large analytical datasets, and providing operational support.Key Responsibilities:Design and develop ETL pipelines and...

  • Big Data Architect

    3 weeks ago

    Islamabad, Islamabad, Pakistan Confiz Full time

    Join to apply for the Big Data Architect role at ConfizWe are looking for a savvy Data Architect to join our growing team of analytics experts. The hire will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross-functional teams.ResponsibilitiesCreate and maintain optimal...

  • Big Data Platform Developer

    7 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Data Engineer - Scalable Platform DevelopmentThe ideal candidate will have extensive experience with Kafka, Flink, Spark, Python, and Talend, along with expertise in Apache Ranger, Prometheus, and strong SQL skills.Job RequirementsDesign and implement data pipelines, ensuring security and monitoring system performance.Collaborate with cross-functional teams...

  • Senior Big Data Tester

    7 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Senior Big Data TesterWe are seeking an experienced Senior Big Data Tester to join our team. In this role, you will be responsible for designing and executing tests for data validation, storage, and retrieval using Azure services like Data Lake, Synapse, and Data Factory. You will collaborate with development teams to assess changes in data workflows and...