Big Data Pipeline Developer

5 days ago


Lahore, Punjab, Pakistan Confiz Full time

Requirements:

  • Bachelor's degree from an accredited institution or equivalent work experience.
  • 5+ years of experience in data engineering, warehousing, publishing, and visualization throughout the data lifecycle.
  • At least 2 years of experience as an Azure Data Engineer or similar role in a technology or IT consulting environment.
  • 3 years of experience in data profiling, cataloging, and mapping for designing and building technical data flows.
  • 3 years of experience defining ELT/ETL architecture and process design.
  • 2 years of experience implementing end-to-end data warehousing analytics solutions on MS or Azure platforms.
  • 2 years of experience working with Azure.
  • Experience in creating data pipelines using ETL tools like Airflow, ADF.
  • Experience with distributed computing technologies like Apache Spark, Databricks.
  • Experience with stream-processing systems like Spark Streaming, Eventhub.
  • Proficiency in one of the object-oriented and functional languages: Python, Java, Scala, etc.
  • Experience in writing effective and maintainable unit and integration tests for ingestion pipelines.
  • Experience in using static analysis and code quality tools.
  • Experience in using code version control repositories like Git.
  • Experience in security best practices including encryption of sensitive data, encryption at rest, and encryption in flight.
  • Experience in building CI/CD pipelines & DevOps orchestration tool like Jenkins.
  • Familiarity with Data Lakehouse, Delta Lake, Apache Hudi.

  • Big Data Architect

    6 days ago


    Lahore, Punjab, Pakistan ACE Money Transfer Full time

    Job SummaryACE Money Transfer is seeking a Big Data Architect to join our team. The ideal candidate will have extensive experience in designing and implementing big data solutions, including Hadoop, Spark, and NoSQL databases. You will be responsible for leading the design and development of complex big data architectures and optimizing data processing...


  • Lahore, Punjab, Pakistan Arbisoft Full time

    We are looking for a highly skilled Big Data Engineer to join our team at Arbisoft. The successful candidate will have 5+ years of experience in building scalable data solutions using cloud-native technologies.Responsibilities:Develop and maintain robust data pipelines and platforms using big data technologies.Design and implement CI/CD pipelines with GitHub...

  • Big Data Architect

    6 days ago


    Lahore, Punjab, Pakistan ACE Money Transfer Full time

    We are looking for a talented Big Data Architect to lead the design and implementation of our big data infrastructure. The successful candidate will have hands-on experience with cloud services such as AWS, Azure, or GCP, and will be able to develop and maintain scalable data pipelines, data models, and ensure data quality.Main Responsibilities:Cloud Service...

  • Big Data Developer

    6 days ago


    Lahore, Punjab, Pakistan ibex Full time

    We are seeking a highly skilled and motivated individual to join our dynamic Reporting Team at ibex as a Big Data Developer.About the JobThis is an exciting opportunity to work with a talented team of professionals who are passionate about delivering high-quality insights to support business decision-making.Design, develop, and maintain efficient and robust...


  • Lahore, Punjab, Pakistan ACE Money Transfer Full time

    Role Summary:We are seeking a highly skilled Big Data Solutions Architect to join our team at ACE Money Transfer. As a key member of our data architecture team, you will be responsible for designing and implementing scalable and high-performance data systems to support our business operations.Key Responsibilities:Big Data Solutions: Design and implement big...


  • Lahore, Punjab, Pakistan Big Data Inc Full time

    Job OverviewWe are seeking a highly skilled Full Stack Java Developer to join our team at Big Data Inc in Lahore.


  • Lahore, Punjab, Pakistan Tkxel LLC Full time

    Key ResponsibilitiesLead the roadmap and enhancements for the enterprise Big Data Platform, ensuring scalability, reliability, and cost efficiency.Architect end-to-end data solutions that fulfill business intelligence and analytics needs while ensuring best practices in security, compliance, and operational efficiency.Lead and mentor a team of Data...


  • Lahore, Punjab, Pakistan Confiz Full time

    Confiz is looking for an experienced Big Data expert to lead its Big Data platform's design, development, and maintenance. As a Data Analytics Director, you will collaborate with data engineers and Big Data Business Analysts across various platform layers, demonstrating a commitment to collaborative problem-solving, sophisticated design, and delivering...


  • Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    About the RoleWe are seeking an experienced Data Engineer to join our team at Avanceon Middle East & South Asia. As a Data Engineer, you will be responsible for designing and developing scalable data solutions that meet the company's requirements. This includes managing large datasets, implementing data lake solutions, and ensuring seamless integration with...


  • Lahore, Punjab, Pakistan Devsinc Full time

    Senior Data EngineerWe are looking for an experienced Senior Data Engineer to join our dynamic team at Devsinc.The ideal candidate will have a deep understanding of data architecture and engineering principles, along with hands-on experience in designing, developing, and maintaining large-scale data processing systems and databases.This role is critical to...

  • Big Data Specialist

    6 days ago


    Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    Job DescriptionRole Summary:This position requires a highly skilled Data Engineer to work on designing, developing, and implementing large-scale data processing systems using cloud-based technologies.Key Responsibilities:Data Ingestion and Processing:Design and develop data ingestion pipelines using structured, semi-structured, and unstructured data...


  • Lahore, Punjab, Pakistan NorthBay Solutions Full time

    Transform Data into Insights with Our Data Engineering TeamWe are seeking a highly skilled Data Engineer to join our team at NorthBay Solutions.Responsibilities:Data Architecture: Design and implement scalable data architecture to handle large volumes of data.Real-time Data Processing: Develop real-time data processing capabilities to handle streaming data...

  • Big Data Architect

    3 days ago


    Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    Job OverviewAvanceon Middle East & South Asia seeks a seasoned Big Data Architect to spearhead the design and implementation of cloud-based data solutions. As a key member of our engineering team, you will be responsible for leading large-scale data projects from concept to delivery.


  • Lahore, Punjab, Pakistan Devsinc Full time

    About the RoleWe're looking for a highly skilled Data Pipeline Architect to oversee the development of scalable and robust data pipelines that can handle high volumes of real-time data. You'll work closely with engineering, analytics, and operations teams to ensure consistent and accurate map experiences. A strong understanding of real-time data pipelines...


  • Lahore, Punjab, Pakistan Devsinc Full time

    We are searching for an experienced Data Pipeline Manager to join our team at Devsinc.This role involves overseeing the development of scalable and robust data pipelines that can handle high volumes of real-time data, ensuring consistent and accurate map experiences.The ideal candidate will have a strong understanding of map data ingestion, curation, and...


  • Lahore, Punjab, Pakistan Arbisoft Full time

    As a Big Data Engineer at Arbisoft, you will be responsible for designing and developing robust data pipelines and platforms using big data technologies. You will also be expected to build and deploy cloud-based data solutions using cloud-native technologies.Key Responsibilities:Design and develop data pipelines and platforms using big data...


  • Lahore, Punjab, Pakistan ACE Money Transfer Full time

    Job DescriptionThe Senior Data Engineer at ACE Money Transfer will lead the development and implementation of advanced data solutions, including big data platforms, data warehousing, and business intelligence tools. As a key member of our data team, you will collaborate with cross-functional teams to design, develop, and deploy scalable and high-performance...


  • Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    Azure Data Engineer at Avanceon Middle East & South Asia: The Ideal CandidateWe are looking for an experienced Azure Data Engineer to join our team. This individual will be responsible for designing and developing data processing pipelines and maintaining scalable data storage solutions.Responsibilities:Implement data processing pipelines using Azure Data...

  • Software Architect

    3 days ago


    Lahore, Punjab, Pakistan Big Data Inc Full time

    Technical RequirementsTo be successful in this role, you should possess:Minimum 3 years of IT experience.Minimum 2 years of experience with designing and developing Spring, J2EE based enterprise applications.Experience in one or more of the following development languages: Spring Boot-Java, J2EE, Angular2+.Experience with CI/CD pipeline GIT, Stash,...

  • Cloud Data Engineer

    6 days ago


    Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    Job DescriptionAs a Big Data Engineer at Avanceon Middle East & South Asia, you will play a key role in designing and developing large-scale data solutions that cater to the company's diverse applications. Your primary responsibility will be to manage massive datasets, ensuring seamless integration with cloud-based infrastructure.The ideal candidate should...