Senior Big Data Engineer

4 weeks ago


Lahore, Punjab, Pakistan Avanceon Limited Full time

Join to apply for the Senior Big Data Engineer (Azure) role at Avanceon Middle East & South Asia.

As a Big Data engineer, you will be responsible for data-related implementation tasks that include provisioning of data storage services, ingesting, streaming and batch data, transforming data, implementing security requirements, implementing data retention policies, identifying performance bottlenecks, and accessing external data. You will develop and implement data Lake solutions for company-wide applications and manage large sets of structured, semi-structured, and unstructured data. You will work in a modern cloud-based data lakehousing environment alongside a team of diverse, intense, and interesting co-workers.

Job Responsibilities
  • Performing analysis of large data stores and derive insights using Big Data querying.
  • Design and develop code, scripts, and data pipelines that leverage structured, semi-structured, and unstructured data.
  • Manage data ingestion pipelines and stream processing.
  • Perform computations using Azure functions.
  • Responsible for the documentation, design, and development of Hadoop applications.
  • Perform day-to-day tasks to accuracy with minimal directions while meeting deadlines.
Qualification & Skillset
  • Bachelor/master's degree in computer science.
  • At least 4-6 years of relevant working experience with Cloud Computation.
  • Hands-on experience on Azure/AWS.
  • Experience working with Apache Spark, Apache Hive, Apache HBase.
  • Hands-on experience on SQL, Scala, Node.js, JS, Java.
  • Hands-on experience with Python, Pandas, NumPy, TensorFlow.
  • Experience with ETL.
Preferred Requirement
  • Implementing Lambda Architecture with stream processing and batch processing.
  • Implement security standards and practices in the Architecture of Cloud Solution.
  • Training/Certification on Data Lake HDFS.
  • Training/Certification on Azure Data Lake Gen 2.
  • Training/Certification on Data Bricks.
  • Experience with Azure Data Explorer and Kusto query language would be a plus.
  • Experience with Azure Data Factory would be a plus.
Seniority level
  • Mid-Senior level
Employment type
  • Full-time
Job function
  • Engineering and Information Technology
Industries
  • Automation Machinery Manufacturing
#J-18808-Ljbffr

  • Lahore, Punjab, Pakistan Big Data Inc Full time

    Overview:Big Data Inc is a cutting-edge technology company that offers innovative solutions for big data processing and analysis. We are currently seeking an experienced Full Stack Java Developer to join our team in Lahore, Pakistan.About the Role:The incumbent will be responsible for designing, testing, and implementing large-scale enterprise applications...


  • Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, participate in a large variety of exciting and challenging projects,...


  • Lahore, Punjab, Pakistan NorthBay Solutions Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a...


  • Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    We are seeking an experienced Big Data Engineering Expert to join our team at NorthBay Solutions LLC. As a Senior Big Data Engineer, you will have the opportunity to design and develop cutting-edge Big Data technologies, build critical distributed systems from scratch, and lead complex ETL jobs and pipelines.About the Role:Design and develop using Big Data...

  • Big Data Engineer

    1 week ago


    Lahore, Punjab, Pakistan Gtradecenter Full time

    InvoZone is a fast-growing company that offers a range of services to clients worldwide. As a Big Data Engineer, you will play a key role in developing innovative business solutions using Apache Spark.ResponsibilitiesDevelop and implement big data solutions using Apache SparkWork with cross-functional teams to identify business needsDesign and implement data...


  • Lahore, Punjab, Pakistan NorthBay Solutions Full time

    NorthBay Solutions is seeking an experienced Big Data Engineering Manager to manage our big data engineering efforts. The ideal candidate will have 4 to 6+ years of experience in data engineering with a focus on big data technologies.About the Position:Main Responsibilities: Lead our data engineering efforts, design and develop large-scale ETL pipelines in...


  • Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    **About the Role:**We are seeking a highly skilled Senior Big Data Engineer to join our team at Avanceon Middle East & South Asia. As a Senior Big Data Engineer, you will be responsible for developing and implementing big data solutions using cloud-based technologies.Responsibilities:Big Data Solutions: Develop and implement big data solutions using...


  • Lahore, Punjab, Pakistan Avanceon Limited Full time

    We are looking for a seasoned Senior Big Data Solutions Architect to lead our team's data engineering efforts.The successful candidate will have extensive experience with big data technologies and a strong understanding of data lake architecture. Key responsibilities include implementing lambda architecture with stream processing and batch processing, as...


  • Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    NorthBay Solutions LLC is hiring a skilled Cloud Architect for Big Data Solutions. As a Senior Big Data Engineer, you will be responsible for designing and developing cloud-based Big Data solutions using AWS services such as EMR, Glue, Lambda, and Athena.About the Role:Design and develop cloud-based Big Data solutions using AWS services.Develop and manage...


  • Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    As a Big Data engineer, you will be responsible for data-related implementation tasks that include provisioning of data storage services, ingesting, streaming and batch data, transforming data, implementing security requirements, implementing data retention policies, identifying performance bottlenecks, and accessing external data. You will develop and...

  • Big Data Architect

    2 weeks ago


    Lahore, Punjab, Pakistan Devsinc Full time

    Job Title: Senior Data SpecialistDevsinc is seeking a highly skilled Senior Data Specialist to design, implement, and maintain efficient data pipelines. This position requires expertise in data modeling, ETL frameworks, big data tools, and programming languages.Key ResponsibilitiesData Pipeline Development: Design and implement scalable and secure data...


  • Lahore, Punjab, Pakistan Avanceon Limited Full time

    We are looking for a senior Big Data Technology Lead to drive our data engineering efforts forward.The successful candidate will have extensive experience with big data technologies and a strong understanding of data lake architecture. Key responsibilities include implementing lambda architecture with stream processing and batch processing, as well as...


  • Lahore, Punjab, Pakistan NorthBay Solutions Full time

    We are looking for a talented Cloud Big Data Specialist to join our team at NorthBay Solutions. The ideal candidate will have 4 to 6+ years of experience in cloud-based big data technologies.About the Role:Main Responsibilities: Design and develop cutting-edge big data technologies to build critical, highly complex distributed systems from scratch.Required...


  • Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    NorthBay Solutions LLC is hiring a skilled Senior Data Engineer for Cloud Platforms. As a Senior Big Data Engineer, you will be responsible for designing and developing cloud-based Big Data solutions using AWS services such as EMR, Glue, Lambda, and Athena.About the Job:Develop and manage complex ETL jobs and pipelines in AWS-based environments.Collaborate...

  • Big Data Specialist

    2 weeks ago


    Lahore, Punjab, Pakistan Devsinc, LLC Full time

    The Senior Data Engineer will play a vital role in shaping our data strategy. We seek an experienced professional who can design, develop, and maintain large-scale data pipelines and databases, as well as collaborate with cross-functional teams to ensure seamless data integration.Key Responsibilities:Data Pipeline Development: Design, develop, and maintain...


  • Lahore, Punjab, Pakistan Big Data Inc Full time

    Company Overview:Big Data Inc is a cutting-edge technology company that specializes in big data processing and analysis. Our mission is to provide innovative solutions to businesses and organizations.About the Role:The incumbent will be responsible for designing, testing, and implementing large-scale enterprise applications using Java back-end...


  • Lahore, Punjab, Pakistan Devsinc Full time

    As a Big Data Infrastructure Specialist, you will play a vital role in designing, developing, and maintaining our big data infrastructure. You will work closely with our cross-functional teams to ensure that our data systems are scalable, reliable, and efficient.Main Responsibilities:Designing and Developing Big Data Infrastructure for High Performance and...

  • Data Engineer

    2 weeks ago


    Lahore, Punjab, Pakistan Crypthon Technologies Full time

    We're looking for a Data Engineer who thrives on building scalable data solutions, designing robust pipelines, and transforming complex data into actionable insights.Responsibilities:Build and maintain data pipelines and architectureCollaborate with teams to define data requirementsOptimize systems for performance and scalabilityWork with big data tools and...


  • Lahore, Punjab, Pakistan Confiz Full time

    Big Data Engineer RoleAs a data engineer at Confiz, you will play a crucial role in designing and implementing big data pipelines that enable seamless data sharing across our systems.Key Responsibilities:Collaborate with solution architects to ensure data pipeline effectiveness.Develop ETL/ELT ingestion pipelines using cloud and on-premises...


  • Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    We are seeking an experienced Data Engineering Leader to join our team at NorthBay Solutions LLC. As a Senior Big Data Engineer, you will have the opportunity to design and develop cutting-edge Big Data technologies, build critical distributed systems from scratch, and lead complex ETL jobs and pipelines.About the Role:Design and develop using Big Data...