Expertise in Big Data Processing

1 week ago


Lahore, Punjab, Pakistan Systems Limited Full time

Job Summary

">

We are seeking an experienced Senior Data Engineer to join our team. As a key member of our organization, you will be responsible for designing, implementing, and optimizing large-scale data pipelines using PySpark, Apache Airflow, and Databricks.

Key responsibilities include:

">
  • ">
  • Designing and maintaining scalable data pipelines using PySpark for ETL/ELT processing;">
  • Using Apache Airflow to automate, schedule, and monitor complex data workflows;">
  • Leveraging Databricks for building and deploying production-grade data pipelines;">
  • Developing and optimizing data pipelines and storage solutions on cloud platforms like AWS, Azure, or Google Cloud;">
  • Monitoring and optimizing the performance of data pipelines;">
">

Requirements

">

To succeed in this role, you must have:

">
  • ">
  • A Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related field;">
  • 8+ years of experience in Data Engineering with a proven track record of designing, implementing, and optimizing data pipelines;">
  • Expertise in PySpark for large-scale data processing and transformation;">
  • Extensive experience with Apache Airflow for managing and orchestrating complex data workflows;">
  • Strong hands-on experience with Databricks, including building notebooks, clusters, and end-to-end pipelines;">
  • Proficiency in SQL and working knowledge of relational databases;">
  • Cloud platform experience, including proficiency in cloud-based data storage and computing services;">
  • Knowledge of data warehousing concepts and experience with modern data warehouses;">
  • Strong experience with ETL/ELT processes and data pipeline orchestration;">
  • Familiarity with version control systems like Git;">
  • Experience working in Agile development teams following Scrum or Kanban methodologies;">

  • Full Stack Developer

    19 hours ago


    Lahore, Punjab, Pakistan Big Data Inc Full time

    Overview:Big Data Inc is a cutting-edge technology company that offers innovative solutions for big data processing and analysis. We are currently seeking an experienced Full Stack Java Developer to join our team in Lahore, Pakistan.About the Role:The incumbent will be responsible for designing, testing, and implementing large-scale enterprise applications...

  • Big Data Specialist

    3 days ago


    Lahore, Punjab, Pakistan Crypthon Technologies Full time

    We're looking for a detail-oriented IT Infrastructure Developer to join our team. As a key member, you will be responsible for designing and implementing scalable data solutions.Key Responsibilities:You'll work closely with cross-functional teams to define data requirements and implement efficient pipelines.Bulk up your skillset by working with cutting-edge...

  • Big Data Engineer

    3 weeks ago


    Lahore, Punjab, Pakistan Arbisoft Full time

    Software Engineering - Arbisoft, Lahore, Islamabad, KarachiWe are looking for a skilled Big Data Engineer with 5+ years of experience in building scalable data solutions. The ideal candidate will have strong coding skills (preferably in Python or Java) and experience in managing the end-to-end development lifecycle—from development to deployment in...

  • Big Data Architect

    3 days ago


    Lahore, Punjab, Pakistan Devsinc Full time

    Job Title: Senior Data SpecialistDevsinc is seeking a highly skilled Senior Data Specialist to design, implement, and maintain efficient data pipelines. This position requires expertise in data modeling, ETL frameworks, big data tools, and programming languages.Key ResponsibilitiesData Pipeline Development: Design and implement scalable and secure data...


  • Lahore, Punjab, Pakistan HR POD Careers Full time

    About the RoleWe are looking for a talented Technical Product Owner - Big Data to lead the development of our big data solutions. As a product owner, you will be responsible for defining product vision, roadmap, and backlogs, ensuring alignment with business objectives.Your expertise in agile environments, lean methodologies, and technical product ownership...


  • Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    Avanceon Middle East & South Asia is seeking an experienced Big Data Implementation Specialist to join our team. In this role, you will be responsible for designing and implementing big data solutions using cloud-based technologies.Key Responsibilities:Implement big data solutions using cloud-based technologies such as Azure Data FactoryManage data ingestion...


  • Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    NorthBay Solutions LLC is hiring a skilled Cloud Architect for Big Data Solutions. As a Senior Big Data Engineer, you will be responsible for designing and developing cloud-based Big Data solutions using AWS services such as EMR, Glue, Lambda, and Athena.About the Role:Design and develop cloud-based Big Data solutions using AWS services.Develop and manage...


  • Lahore, Punjab, Pakistan HR POD Careers Full time

    Requirements:4+ years of technical QA experience in Big Data technologies.2-3 years of experience with test automation frameworks.Proficient in Python, Pytest, Robot, and SQL for Big Data testing.Experience with Apache Spark and Big Data cloud platforms.Strong knowledge of Data Testing Strategies (data, process, outcome validation, code coverage).Experience...


  • Lahore, Punjab, Pakistan Devsinc Full time

    As a Big Data Infrastructure Specialist, you will play a vital role in designing, developing, and maintaining our big data infrastructure. You will work closely with our cross-functional teams to ensure that our data systems are scalable, reliable, and efficient.Main Responsibilities:Designing and Developing Big Data Infrastructure for High Performance and...


  • Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    We are seeking an experienced Big Data Engineering Expert to join our team at NorthBay Solutions LLC. As a Senior Big Data Engineer, you will have the opportunity to design and develop cutting-edge Big Data technologies, build critical distributed systems from scratch, and lead complex ETL jobs and pipelines.About the Role:Design and develop using Big Data...


  • Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    **About the Role:**We are seeking a highly skilled Senior Big Data Engineer to join our team at Avanceon Middle East & South Asia. As a Senior Big Data Engineer, you will be responsible for developing and implementing big data solutions using cloud-based technologies.Responsibilities:Big Data Solutions: Develop and implement big data solutions using...

  • Big Data Specialist

    6 days ago


    Lahore, Punjab, Pakistan Devsinc, LLC Full time

    The Senior Data Engineer will play a vital role in shaping our data strategy. We seek an experienced professional who can design, develop, and maintain large-scale data pipelines and databases, as well as collaborate with cross-functional teams to ensure seamless data integration.Key Responsibilities:Data Pipeline Development: Design, develop, and maintain...

  • Big Data Specialist

    6 days ago


    Lahore, Punjab, Pakistan Devsinc Full time

    About the RoleThe successful candidate will have proven experience (3+ years) as a Data Engineer or similar role in data management, processing, and analytics. They will possess experience with data modeling, ETL frameworks, and big data tools like Apache Hadoop, Spark, Kafka, etc.They will also demonstrate proficiency in programming languages such as...

  • Big Data Engineer

    1 week ago


    Lahore, Punjab, Pakistan Burq Full time

    We value diversity and are an equal opportunity employer. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.The Ideal Candidate Will HaveExperience working with data engineering tools and platformsStrong programming skills in Python and...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    Empowering Businesses with Real-Time Insights through Advanced Data ArchitecturesData Pilot is committed to helping organizations unlock the full potential of their data assets and maximize business value through our expert engineering and business acumen. As an Associate Data Architect, you will play a crucial role in designing and implementing scalable,...


  • Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, participate in a large variety of exciting and challenging projects,...


  • Lahore, Punjab, Pakistan NorthBay Solutions Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a...

  • Big Data Engineer

    2 days ago


    Lahore, Punjab, Pakistan Gtradecenter Full time

    InvoZone is a fast-growing company that offers a range of services to clients worldwide. As a Big Data Engineer, you will play a key role in developing innovative business solutions using Apache Spark.ResponsibilitiesDevelop and implement big data solutions using Apache SparkWork with cross-functional teams to identify business needsDesign and implement data...


  • Lahore, Punjab, Pakistan HR POD Careers Full time

    Job Description:We are seeking a highly skilled Data Quality Engineer to join our team at HR POD Careers. As a key member of our big data testing team, you will be responsible for developing and automating processes for gathering expected results from data sources and comparing data results from testing.About the Role:Develop and automate test scripts to...


  • Lahore, Punjab, Pakistan Big Byte Insights Full time

    About Big Byte InsightsAt Big Byte Insights, we're dedicated to empowering businesses with innovative software solutions that drive growth, efficiency, and customer satisfaction. Our team of experts collaborates closely with clients to understand their unique challenges and develops tailored solutions that exceed their expectations.Our ExpertiseWe specialize...