Cloud Data Engineer

7 days ago

Lahore, Punjab, Pakistan beBee Careers Full time
Cloud Data Integration Specialist

The role involves designing and implementing data pipelines, resolving technical issues, and ensuring data accuracy across various platforms such as Azure Synapse Analytics, Azure Data Lake Gen2, and SQL environments.

This position requires extensive expertise in SQL and a strong background in PySpark development. Key responsibilities include working with Azure Synapse Pipelines and PySpark for data transformation and pipeline management, performing data integration and schema updates in Delta Lake environments, and collaborating with the Azure DevOps team on CI/CD processes for deployment of Infrastructure as Code (IaC) and Workspace artifacts.

Data Engineering Tasks Include:
  • Designing and implementing ETL processes to integrate data from various sources.
  • Working with our Azure DevOps team on Continuous Integration and Continuous Deployment (CI/CD) processes for infrastructure as code.
  • Developing custom solutions for customers defined by our Data Architect and assisting in improving our data solution patterns over time.

Ticket Management

Key responsibilities also include monitoring the Jira ticket queue and responding to tickets as they are raised, understanding ticket issues using extensive SQL, Synapse Analytics, and other tools to troubleshoot them, and communicating effectively with customer users who raised the tickets and collaborating with other teams as needed to resolve issues.

Desired Skills & Qualifications

We are seeking a candidate with 5+ years of experience in Dynamics 365 ecosystem and a strong background in PySpark development. The ideal candidate will possess the following attributes and qualifications:
  • Extensive experience with SQL, including query writing and troubleshooting in Azure SQL, Synapse Analytics, and Delta Lake environments.
  • Strong understanding and experience in implementing and supporting ETL processes, Data Lakes, and data engineering solutions.
  • Proficiency in using Azure Synapse Analytics, including workspace management, pipeline creation, and data flow management.
  • Hands-on experience with PySpark for data processing and automation.
  • Ability to use VPNs, MFA, RDP, jump boxes/jump hosts, etc., to operate within secure environments.
  • Some experience with Azure DevOps CI/CD IaC and release pipelines.
  • Ability to communicate effectively both verbally and in writing, with strong problem-solving and analytical skills.
  • Understanding of the operation and underlying data structure of D365 Finance and Operations, Business Central, and Customer Engagement.
  • Experience with Data Engineering in Microsoft Fabric.
  • Experience with Delta Lake and Azure data engineering concepts (e.g., ADLS, ADF, Synapse, AAD, Databricks).
  • Certifications in Azure Data Fabric: DP-600 / DP-700

  • Cloud Data Engineer Leader

    2 weeks ago

    Lahore, Punjab, Pakistan Arbisoft Full time

    Arbisoft is seeking a highly skilled Cloud Data Engineer Leader to oversee the development and maintenance of scalable data systems.Responsibilities:Design, develop, and implement robust data pipelines and platforms utilizing cloud-based services.Manage the end-to-end development lifecycle, ensuring seamless deployment in production...

  • Cloud Data Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan beBee Careers Full time

    Azure Data Engineer PositionThe Azure Data Engineer will be responsible for architecting, building, and maintaining large-scale data systems using Azure Data Factory, Azure Functions, Data Lake Gen2, and Databricks.Key Skills Required:Strong understanding of Azure services and their applications.Experience with Python, SQL, Scala, or JavaScript programming...

  • Senior Cloud Data Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan beBee Careers Full time

    Senior Data EngineerA senior data engineer with expertise in building large-scale data pipelines and architecting cloud-based data solutions is sought after by a leading software development company. The ideal candidate will have experience designing, developing, and deploying data solutions that power business insights.Key Responsibilities:Data Pipeline...

  • Cloud-Native Data Architect

    2 weeks ago

    Lahore, Punjab, Pakistan beBee Careers Full time

    Cloud-Native Data EngineerWe are looking for a highly skilled Cloud-Native Data Engineer who has experience with cloud computing platforms like AWS, Azure, or GCP. The ideal candidate will be responsible for designing, developing, and maintaining robust data infrastructure to ensure the availability, scalability, and reliability of our data systems.Main...

  • Cloud Data Engineer Role

    2 weeks ago

    Lahore, Punjab, Pakistan Devsinc Full time

    Data Engineer Job SummaryAt Devsinc, we're looking for an experienced Senior Data Engineer to lead the design, development, and maintenance of high-performance data processing systems and data pipelines. This individual will collaborate with cross-functional teams to ensure seamless data access and reliability.Our Company CultureWe foster a dynamic and...

  • Cloud Data Engineering Expert

    1 week ago

    Lahore, Punjab, Pakistan ITC WORLDWIDE Full time

    This Cloud Data Engineering role at ITC Worldwide involves leveraging cutting-edge technologies to design and implement scalable data integration solutions. As a Cloud Data Engineer, you will be responsible for developing and maintaining data pipelines using Azure Synapse Pipelines and PySpark for data transformation and pipeline management.Data Engineering...

  • Cloud Data Architect

    1 week ago

    Lahore, Punjab, Pakistan beBee Careers Full time

    Job Title: Data Engineering Specialist">Data Engineers design and implement data pipelines, systems, and processes.ResponsibilitiesTechnical Presentations: Prepare and deliver detailed technical presentations to showcase product strengths, capabilities, and benefits.Solution Architecture Design: Architect and design solutions using cloud computing and...

  • Lead Data Engineer

    4 days ago

    Lahore, Punjab, Pakistan beBee Careers Full time

    Data Engineering ManagerThis leadership role requires a seasoned professional with extensive experience in data engineering, cloud platforms, and machine learning operations.About the Role:Data Engineering Leadership: Provide technical leadership and mentor junior engineers, fostering a collaborative environment.Cloud Data Infrastructure: Engineer and manage...

  • Cloud Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan beBee Careers Full time

    About the RoleThe Backend Engineer will be responsible for building high-performance services using microservices architecture, cloud-native technologies, and advanced data processing pipelines.You will work closely with front-end engineers to ensure seamless API consumption and implement security best practices for authentication, authorization, and data...

  • Data Engineering Manager

    6 days ago

    Lahore, Punjab, Pakistan beBee Careers Full time

    Data Engineering ManagerWe are looking for a seasoned Data Engineering Manager to lead the design, development, and management of data pipelines, orchestration, and cloud operations for all data initiatives.This role requires strong experience in data engineering, cloud platforms, and machine learning operations, as well as excellent leadership and technical...