Big Data Professional

7 days ago


Islamabad, Islamabad, Pakistan NorthBay Solutions Full time

We are NorthBay Solutions, a leading provider of innovative data solutions. We are currently seeking a highly skilled Big Data Professional to join our team.

About the Role:

  • We are looking for an individual with extensive knowledge of big data technologies, including AWS Glue, Lambda, Step Functions, and others, to design, develop, and maintain our data pipelines.
  • You will be responsible for investigating job failures, analyzing CloudWatch logs, and collaborating with cross-functional teams to implement corrective actions.
  • As a Big Data Professional, you will work closely with our L2/L3 engineers and data architects to escalate and resolve complex issues.

Key Responsibilities:

  • Design, develop, and maintain big data pipelines using AWS Glue, Lambda, Step Functions, Athena, Redshift, or Airflow.
  • Investigate job failures, analyze CloudWatch logs, and collaborate with L2/L3 teams to implement corrective actions.
  • Re-run failed jobs and retry data ingestion processes while ensuring data consistency.
  • Maintain documentation of troubleshooting steps, common issues, and resolutions.
  • Participate in incident response and recovery procedures as needed.
  • Provide technical support for production data pipelines, ensuring SLAs are met.

Qualifications:

  • Bachelor's degree in Computer Science, Data Science, Information Technology, or a related field.
  • Up to 2+ years of experience in Data Processing / ETL.
  • Hands-on experience with any two of AWS Glue, Athena, Lambda, Step Functions, Redshift.
  • Proficiency in Python and SQL for data querying and processing (preferred).
  • Familiarity with PySpark and ETL development (a plus).
  • Experience in monitoring and debugging AWS Glue jobs, Step Functions, and Lambda logs (a plus).
  • Strong analytical and problem-solving skills to diagnose and resolve data processing failures.
  • Ability to work independently while effectively collaborating with cross-functional teams.
  • AWS certifications such as AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect Associate (preferred but not mandatory).
  • Excellent communication and interpersonal skills.
  • Prior experience in L1/L2 support (a plus).

Career Path Opportunities:

  • Senior Data Engineer (L2/L3 Support / ETL Development).
  • Big Data Engineer / AWS Data Architect.
  • Data Engineering Lead / Architect.

  • Big Data Engineer

    23 hours ago


    Islamabad, Islamabad, Pakistan Evamp & Saanga Full time

    As a Big Data Engineer specializing in Java development, you will play a crucial role in designing, developing, and implementing large-scale data processing systems. Your primary focus will be on leveraging big data technologies to build robust and efficient solutions for managing and analyzing vast amounts of data. You will collaborate with cross-functional...

  • Big Data Architect

    2 days ago


    Islamabad, Islamabad, Pakistan ieng Group Full time

    Job Overview:The ieng Group is seeking a highly skilled Big Data Architect to design and implement large-scale data processing systems. This individual will work closely with the data engineering team to ensure the successful delivery of data-driven projects.Main Responsibilities:System Design: Design and develop scalable, reliable, and efficient data...

  • Big Data Developer

    3 weeks ago


    Islamabad, Islamabad, Pakistan Datamatics Technologies Full time

    Direct message the job poster from Datamatics TechnologiesTalent Acquisition Lead | Global Technical Recruitment | Technical Recruitment | EMEA & MENAJob Summary:We are looking for a highly skilled Big Data Engineer to spearhead the development of a scalable Big Data Platform. The ideal candidate will have extensive experience with Kafka, Flink, Spark,...


  • Islamabad, Islamabad, Pakistan Datamatics Technologies Full time

    Job Summary:We are seeking an experienced Big Data Engineer to lead the development of a scalable Big Data Platform. The ideal candidate will have extensive experience with Kafka, Flink, Spark, Python, and Talend, along with expertise in Apache Ranger, Prometheus, and strong SQL skills.This role involves designing and implementing data pipelines, ensuring...


  • Islamabad, Islamabad, Pakistan Confiz Full time

    Job DescriptionThe purpose of this role is to have a highly skilled QA Automation Expert who can participate in day-to-day QA activities such as sprint calls and providing updates on assigned tickets. The ideal candidate will improve their knowledge in the relevant domain and create and identify the regression testing suite to eliminate manual effort from...


  • Islamabad, Islamabad, Pakistan Fusemachines Full time

    Job SummaryAbout FusemachinesFusemachines is a leading provider of AI solutions and products to various industries. Our company was founded by Sameer Maskey, Ph.D., an Adjunct Associate Professor at Columbia University, who aims to democratize AI and leverage its global impact. With a robust presence in four countries and a team of over 400 employees, we...


  • Islamabad, Islamabad, Pakistan Fusemachines Full time

    **About the Role**Fusemachines is a global AI company with a presence in four countries, committed to democratizing AI and harnessing its power to transform businesses worldwide. We are currently seeking a highly skilled Sr. Data Engineer to join our team and lead the development of big data infrastructure using Microsoft Azure in the Media domain.This...


  • Islamabad, Islamabad, Pakistan Confiz Full time

    Role OverviewThis role is focused on leveraging big data technologies to improve our QA process. We are seeking a highly skilled QA Automation Expert to join our team and contribute to the success of our company.Responsibilities:Developing and automating processes for gathering expected results from data sources and comparing data results from...


  • Islamabad, Islamabad, Pakistan Datamatics Technologies Full time

    About the Job:Datamatics Technologies is looking for a highly skilled Big Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing and implementing a scalable Big Data Platform.Responsibilities:Design, develop, and optimize Big Data Platform using Kafka, Flink, Spark, and Talend.Architect and implement...

  • Big Data Engineer

    1 week ago


    Islamabad, Islamabad, Pakistan VisionX Technologies, Inc. Full time

    Key QualificationsTo succeed as a Senior Data Engineer at VisionX Technologies, Inc., you should possess:A minimum of a bachelor's degree in Computer Science, Information Technology, or a related field.A deep passion for data and proven ability to create high-quality data products.At least 8+ years of experience in implementing Big Data, Data Lakehouse, and...


  • Islamabad, Islamabad, Pakistan Fusemachines Full time

    About the RoleData Engineering LeadWe are seeking a highly skilled Senior Data Engineer to lead, design, build, and maintain the infrastructure required for data integration, storage, processing, and analytics (BI, visualization and Advanced Analytics) using Microsoft Azure in the Media domain. The ideal candidate will possess hands-on experience with...


  • Islamabad, Islamabad, Pakistan Confiz Full time

    The purpose of this job is to have a QA Automation Expert who can participate in day-to-day QA activities such as sprint calls and providing updates on assigned tickets. You will improve your knowledge in the relevant domain and, with your abilities and skillset, create and identify the regression testing suite to eliminate manual effort from the existing QA...


  • Islamabad, Islamabad, Pakistan Fusemachines Full time

    **Job Description**Fusemachines is a leading AI company that specializes in delivering state-of-the-art AI products and solutions to various industries. As a Sr. Data Engineer, you will be responsible for designing, building, and maintaining the infrastructure required for data integration, storage, processing, and analytics using Microsoft Azure in the...


  • Islamabad, Islamabad, Pakistan Confiz Full time

    About This OpportunityWe are looking for a talented QA Automation Expert to join our team. In this role, you will be responsible for participating in day-to-day QA activities and providing updates on assigned tickets. You will also improve your knowledge in the relevant domain and create and identify the regression testing suite to eliminate manual effort...


  • Islamabad, Islamabad, Pakistan Confiz Full time

    About the RoleWe are seeking an experienced QA Automation Expert to join our team. As a key member of our QA team, you will be responsible for participating in day-to-day QA activities and providing updates on assigned tickets. You will also improve your knowledge in the relevant domain and create and identify the regression testing suite to eliminate manual...


  • Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines is a leading provider of AI strategy, talent, and education services. With a global presence and a team of experts, we seek to democratize AI and bring our expertise to companies worldwide.About the RoleThis contract position involves designing, building, and maintaining data infrastructure for integration, storage, processing,...


  • Islamabad, Islamabad, Pakistan Confiz Full time

    Confiz is a leading technology company that has been at the forefront of innovation, delivering cutting-edge solutions to businesses across various industries. We are currently seeking an experienced Machine Learning Engineer with expertise in big data processing to join our team.The ideal candidate will have strong programming skills in Python or R,...


  • Islamabad, Islamabad, Pakistan VisionX Technologies, Inc. Full time

    VisionX Technologies, Inc. OverviewVisionX is a global provider of innovation solutions, partnering with Fortune 1000 companies to drive business growth. Our team leverages agile methodologies, technology accelerators, and Intellectual Property creation to deliver high-quality products.We have been recognized as one of the Top 10 Most Innovative Companies of...


  • Islamabad, Islamabad, Pakistan Confiz Full time

    We are undertaking a Data Transformation and as data becomes fundamental to what we are doing, the ability to have the right data in the right place at the right time, that is accurate and consistent is essential.This role will play a really important central role within this evolving landscape, working alongside the Technology Team and the Product Owner to...


  • Islamabad, Islamabad, Pakistan Confiz Full time

    Job OverviewThe Principal Data Engineer at Confiz is responsible for designing and implementing cloud-based data warehousing solutions using Azure Synapse. This role requires a professional with expertise in data engineering and strategic thinking to deliver scalable and efficient data solutions to meet complex business requirements. The ideal candidate will...