Big Data Engineer

1 week ago

Islamabad, Islamabad, Pakistan Datamatics Technologies Full time

Get AI-powered advice on this job and more exclusive features.

Direct message the job poster from Datamatics Technologies

Job Title: Big Data Engineer

Job Summary:

We are looking for a highly skilled Big Data Engineer to spearhead the development of a scalable Big Data Platform. The ideal candidate will have extensive experience with Kafka, Flink, Spark, Python, and Talend, along with expertise in Apache Ranger, Prometheus, and strong SQL skills. This role involves designing and implementing data pipelines, ensuring security, monitoring system performance, and collaborating with cross-functional teams to deliver high-performance data solutions.

Key Responsibilities:

  • Design, develop, and optimize a Big Data Platform leveraging Kafka, Flink, Spark, and Talend.
  • Architect and implement real-time and batch data processing pipelines.
  • Ensure secure data access and governance using Apache Ranger.
  • Monitor and optimize system performance with Prometheus.
  • Develop and maintain ETL workflows and data ingestion pipelines.
  • Write efficient and scalable SQL queries for data transformation and analysis.
  • Collaborate with Data Scientists, Analysts, and DevOps teams to align data infrastructure with business goals.
  • Implement data security, compliance, and access control policies.
  • Troubleshoot and resolve issues related to data processing, storage, and system performance.

Required Skills & Qualifications:

  • 5+ years of experience in Big Data Engineering.
  • Hands-on expertise in Kafka, Flink, Spark, Python, and Talend.
  • Strong proficiency in SQL, with experience in query optimization and performance tuning.
  • Familiarity with Apache Ranger for data security and access control.
  • Familiarity with Prometheus for monitoring and performance optimization.
  • Deep understanding of distributed computing, data partitioning, and pipeline orchestration.
  • Experience with cloud-based big data solutions (AWS, Azure, or GCP) is a plus.
  • Strong problem-solving and analytical skills.
  • Excellent communication and leadership abilities.

Preferred Qualifications:

  • Experience with Docker, Kubernetes, and containerized big data deployments.
  • Knowledge of workflow orchestration tools like Airflow.
  • Hands-on experience with NoSQL databases (e.g., Cassandra, HBase).
Seniority level
  • Seniority levelMid-Senior level
Employment type
  • Employment typeFull-time
Job function
  • Job functionEngineering and Information Technology
  • IndustriesIT Services and IT Consulting

Referrals increase your chances of interviewing at Datamatics Technologies by 2x

Sign in to set job alerts for "Big Data Developer" roles.Senior/Staff Software Developer - (Angular +Java/Python + AWS) - Morning ShiftFull Stack Developer (Python, ADB, Playwright) — WhatsApp Automation SaaSPython Developer with ERPNEXT Experience

We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

#J-18808-Ljbffr

  • Big Data Engineer

    2 days ago

    Islamabad, Islamabad, Pakistan beBeeDataEngineering Full time

    We are seeking an experienced Data Engineer to join our team.The ideal candidate will have a strong background in data engineering, particularly in big data environments on Azure's cloud platform.Job DescriptionThis is a mid-senior level full-time position that requires a bachelor's degree in computer science or a related field and 3-6 years of...

  • Big Data Software Engineer

    7 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Data Scientist - Big DataWe are looking for a talented Data Scientist with expertise in big data engineering to join our team.The successful candidate will be responsible for designing, developing, and implementing data processing systems using big data technologies. This includes building robust and efficient solutions for managing and analyzing vast...

  • Senior Big Data Engineer

    7 days ago

    Islamabad, Islamabad, Pakistan NorthBay Solutions Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a...

  • Big Data Developer

    7 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    We are looking for a skilled Data Engineer to join our team. The ideal candidate will have experience in data engineering and a passion for Big Data.You will have the opportunity to participate in exciting and challenging projects, work with our exceptional team of Technology professionals, and contribute to the growth and development of our global,...

  • Big Data System Developer

    7 days ago

    Islamabad, Islamabad, Pakistan beBeeData Full time

    Job Summary: We are seeking a skilled Data Engineer to lead the development of a scalable Big Data Platform. The ideal candidate will have extensive experience with Kafka, Flink, Spark, Python, and Talend.This role involves designing and implementing data pipelines, ensuring security, monitoring system performance, and collaborating with cross-functional...

  • Associate Software Engineer- Data Engineer

    3 weeks ago

    Islamabad, Islamabad, Pakistan Devsinc Full time

    We are looking for a highly motivated Associate Software Engineer – Data Engineering to join our team. This is an excellent opportunity for recent graduates or early-career professionals who are passionate about data engineering, big data technologies, and building scalable data pipelines. In this role, you will work with experienced engineers to develop,...

  • Senior Data Engineer

    2 weeks ago

    Islamabad, Islamabad, Pakistan TenX Full time

    Join to apply for the Senior Data Engineer role at TenXSenior Talent Acquisition Executive @ TenX | 360° HR Resource | Scouting Talent | C-level Recruitment | Your hiring PartnerJOB SUMMARY:Data Engineers are responsible for designing and creating the data warehouse and all related extraction, transformation, and load of data functions. They should be able...

  • Data Warehouse Engineer

    2 weeks ago

    Islamabad, Islamabad, Pakistan beBeeDataWarehouse Full time

    Key Responsibilities:Design and develop data warehouses and ETL processes to extract, transform, and load data from various sources.Analyze business goals and objectives to create optimal ETL/BI processes.Data Warehouse/Etl Specialist Requirements:Bachelor's degree in computer science or related field; a master's degree is preferred.3-4 years of experience...

  • Senior Data Engineering Expert

    2 days ago

    Islamabad, Islamabad, Pakistan beBeeDataEngineer Full time

    About the Role:We are seeking a highly skilled ETL Data Engineer with experience in building robust data pipelines, working with large-scale datasets, and leveraging modern Big Data and cloud technologies.Key Responsibilities:Design and develop complex ETL pipelines and data solutions using Big Data and cloud-native technologies.Leverage tools such as Ab...

  • Data Scientist/Engineer

    3 weeks ago

    Islamabad, Islamabad, Pakistan SwipBox Full time

    Job Summary: We are looking for a skilled Data Engineer/Data Scientist with strong Python programming and analytical skills, who can work across the data pipeline from ingestion and processing to modelling and reporting. The ideal candidate has experience handling large-scale (big) datasets, building predictive models, working with live and runtime data...