Big Data Engineer

4 weeks ago

Islamabad, Islamabad, Pakistan Datamatics Technologies Full time

Get AI-powered advice on this job and more exclusive features.

Direct message the job poster from Datamatics Technologies

Job Title: Big Data Engineer

Job Summary:

We are looking for a highly skilled Big Data Engineer to spearhead the development of a scalable Big Data Platform. The ideal candidate will have extensive experience with Kafka, Flink, Spark, Python, and Talend, along with expertise in Apache Ranger, Prometheus, and strong SQL skills. This role involves designing and implementing data pipelines, ensuring security, monitoring system performance, and collaborating with cross-functional teams to deliver high-performance data solutions.

Key Responsibilities:

  • Design, develop, and optimize a Big Data Platform leveraging Kafka, Flink, Spark, and Talend.
  • Architect and implement real-time and batch data processing pipelines.
  • Ensure secure data access and governance using Apache Ranger.
  • Monitor and optimize system performance with Prometheus.
  • Develop and maintain ETL workflows and data ingestion pipelines.
  • Write efficient and scalable SQL queries for data transformation and analysis.
  • Collaborate with Data Scientists, Analysts, and DevOps teams to align data infrastructure with business goals.
  • Implement data security, compliance, and access control policies.
  • Troubleshoot and resolve issues related to data processing, storage, and system performance.

Required Skills & Qualifications:

  • 5+ years of experience in Big Data Engineering.
  • Hands-on expertise in Kafka, Flink, Spark, Python, and Talend.
  • Strong proficiency in SQL, with experience in query optimization and performance tuning.
  • Familiarity with Apache Ranger for data security and access control.
  • Familiarity with Prometheus for monitoring and performance optimization.
  • Deep understanding of distributed computing, data partitioning, and pipeline orchestration.
  • Experience with cloud-based big data solutions (AWS, Azure, or GCP) is a plus.
  • Strong problem-solving and analytical skills.
  • Excellent communication and leadership abilities.

Preferred Qualifications:

  • Experience with Docker, Kubernetes, and containerized big data deployments.
  • Knowledge of workflow orchestration tools like Airflow.
  • Hands-on experience with NoSQL databases (e.g., Cassandra, HBase).
Seniority level
  • Seniority levelMid-Senior level
Employment type
  • Employment typeFull-time
Job function
  • Job functionEngineering and Information Technology
  • IndustriesIT Services and IT Consulting

Referrals increase your chances of interviewing at Datamatics Technologies by 2x

Sign in to set job alerts for "Big Data Developer" roles.Senior/Staff Software Developer - (Angular +Java/Python + AWS) - Morning ShiftFull Stack Developer (Python, ADB, Playwright) — WhatsApp Automation SaaSPython Developer with ERPNEXT Experience

We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

#J-18808-Ljbffr

  • Senior Big Data Engineer

    4 weeks ago

    Islamabad, Islamabad, Pakistan NorthBay Solutions Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a...

  • Big Data Platform Owner

    3 days ago

    Islamabad, Islamabad, Pakistan Confiz Full time

    We are looking for an experienced and driven Big Data Platform Owner (Engineering Manager) to lead our Big Data engineering team. This role is perfect for a technical leader with a strong background in Azure Big Data technologies who can drive strategy, foster team growth, and ensure the successful delivery of high-quality data solutions. You will be...

  • Chief Architect

    3 days ago

    Islamabad, Islamabad, Pakistan beBeeData Full time 1,800,000 - 2,500,000

    Big Data Engineering Leadership OpportunityWe are seeking an experienced Big Data Platform Owner to lead our engineering team. This role is ideal for a technical leader with a strong background in Azure Big Data technologies who can drive strategy, foster team growth, and ensure the successful delivery of high-quality data solutions.Key...

  • Data Engineer

    3 days ago

    Islamabad, Islamabad, Pakistan NorthBay Solutions LLC Full time

    Job Title: Data EngineerLocation:Karachi, Lahore , Islamabad (Hybrid)Experience:5+ YearsJob Type: Full-TimeJob Overview:We are looking for a highly skilled and experienced Data Engineer with a strong foundation in Big Data, distributed computing, and cloud-based data solutions. This role demands a strong understanding of end-to-end Data pipelines, data...

  • Senior Data Engineer

    3 weeks ago

    Islamabad, Islamabad, Pakistan TenX Full time

    JOB SUMMARY:Data Engineers are responsible for designing and creating the data warehouse and all related extraction, transformation, and load of data functions. He should be able to analyze and digest what a business wants to accomplish with its data and design the best possible ETL/BI process around those goals.DESCRIPTION:Data Warehouse /ETLParticipates in...

  • SM EDW and Big Data Development

    2 weeks ago

    Islamabad, Islamabad, Pakistan PTCLal Full time

    Get AI-powered advice on this job and more exclusive features.Sign in to access AI-powered advicesContinue with Google Continue with GoogleDirect message the job poster from PTCL.OfficialTalent Acquisition | Talent & Organizational Development | Total Rewards | HR Transformation | Reporting & AnalyticsARE YOU READY TO RISE WITH PTCL GROUPWith our unwavering...

  • Strategic Data Engineer

    1 week ago

    Islamabad, Islamabad, Pakistan beBeeData Full time 1,200,000 - 1,500,000

    Senior Data EngineerWe are seeking an experienced Senior Data Engineer to join our team.Design and develop complex data pipelines using Big Data and cloud-native technologies.Leverage tools such as Ab Initio, Informatica, DBT, and Apache Spark to build scalable data workflows.Implement distributed data processing using Hadoop, Hive, Kafka, and Spark.Build...

  • Data Scientist/Engineer

    6 days ago

    Islamabad, Islamabad, Pakistan SwipBox Full time

    Job Summary: We are looking for a skilled Data Engineer/Data Scientist with strong Python programming and analytical skills, who can work across the data pipeline from ingestion and processing to modelling and reporting. The ideal candidate has experience handling large-scale (big) datasets, building predictive models, working with live and runtime data...

  • Senior Data Specialist

    3 days ago

    Islamabad, Islamabad, Pakistan beBeeData Full time 1,200,000 - 1,500,000

    Job Opportunity: Data EngineerWe are seeking a highly skilled and experienced professional to join our team as a Data Engineer. This role will be responsible for designing, developing, and maintaining large-scale data pipelines using modern big data technologies and cloud-native tools.

  • Senior Data Infrastructure Specialist

    2 weeks ago

    Islamabad, Islamabad, Pakistan beBeeDataEngineer Full time

    Job Description">We're seeking an ambitious and detail-oriented Associate Software Engineer – Data Engineer to play a key role in developing, optimizing, and maintaining our data infrastructure.This is an exciting opportunity for recent graduates or early-career professionals passionate about data engineering, big data technologies, and building scalable...