Azure Data Integration Specialist

3 days ago

Gujrat, Punjab, Pakistan beBeeData Full time $80,000 - $150,000
Job Overview

The role of Data Integration Engineer at a leading organization involves designing and managing data pipelines, troubleshooting issues, and ensuring data accuracy across platforms such as Azure Synapse Analytics, Azure Data Lake Gen2, and SQL environments.

Responsibilities
  • Data Engineering: Collaborate with teams to work on Azure Synapse Pipelines and PySpark for data transformation and pipeline management.
  • Perform data integration and schema updates in Delta Lake environments, ensuring smooth data flow and accurate reporting.
  • Work with our DevOps team on CI/CD processes for deployment of Infrastructure as Code (IaC) and Workspace artifacts.
  • Develop custom solutions for customers defined by our Data Architect and assist in improving our data solution patterns over time.
Documentation
  • Document ticket resolutions, testing protocols, and data validation processes.
  • Collaborate with stakeholders to provide specifications and quotations for enhancements requested by customers.
Ticket Management
  • Monitor the ticket queue and respond to tickets as they are raised.
  • Understand ticket issues, utilizing extensive SQL, Synapse Analytics, and other tools to troubleshoot them.
  • Communicate effectively with customer users who raised the tickets and collaborate with other teams as needed to resolve issues.
Troubleshooting And Support
  • Handle issues related to ETL pipeline failures, Delta Lake processing, or data inconsistencies in Synapse Analytics.
  • Provide prompt resolution to data pipeline and validation issues, ensuring data integrity and performance.
Requirements
  • At least 5 years of experience in a similar field.
  • Knowledge of Azure Synapse Analytics, PySpark, and Delta Lake environments.
  • Strong problem-solving and analytical skills.
Qualifications
  • Extensive experience with SQL, including query writing and troubleshooting in Azure SQL, Synapse Analytics, and Delta Lake environments.
  • Strong understanding and experience in implementing and supporting ETL processes, Data Lakes, and data engineering solutions.
  • Proficiency in using Azure Synapse Analytics, including workspace management, pipeline creation, and data flow management.
  • Hands-on experience with PySpark for data processing and automation.
  • Ability to use VPNs, MFA, RDP, jump boxes/jump hosts, etc., to operate within customers' secure environments.
  • Some experience with Azure DevOps CI/CD IaC and release pipelines.
  • Ability to communicate effectively both verbally and in writing, with strong problem-solving and analytical skills.
  • Understanding of the operation and underlying data structure of D365 Finance and Operations, Business Central, and Customer Engagement.
  • Experience with Data Engineering in Microsoft Fabric
  • Experience with Delta Lake and Azure data engineering concepts (ADLS, ADF, Synapse, AAD, Databricks).
  • Certifications in Azure Data Fabric: DP-600 / DP-700

  • Data Systems Specialist

    3 days ago

    Gujrat, Punjab, Pakistan beBeeDataEngineer Full time 1,500,000 - 2,500,000

    Key Roles and ResponsibilitiesWe are seeking a skilled professional to contribute to our dynamic team.Design, develop, and optimize data pipelines in Azure Data Lake for efficient processing of large datasets.Create robust data pipeline solutions within Azure Data Lake.Integrate and transform data from various sources to enable comprehensive analytics and...

  • Cloud Data Solutions Specialist

    3 days ago

    Gujrat, Punjab, Pakistan beBeeDataEngineer Full time 900,000 - 1,200,000

    Job OpportunityWe are seeking a skilled Data Engineer to contribute to our team.Main ResponsibilitiesData Pipeline Architectures: Design and implement robust data pipeline solutions using Azure Data Lake.Data Integration and Transformation: Integrate and transform data from various sources to enable comprehensive analytics and reporting.Quality Assurance:...

  • Data Architect

    3 days ago

    Gujrat, Punjab, Pakistan beBeeData Full time 1,200,000 - 2,000,000

    Job Title: Senior Data ArchitectWe are seeking a highly skilled professional to lead our data architecture initiatives and contribute to the success of our organization.The ideal candidate will possess strong technical skills in Azure Data Services, including Azure Data Lake Storage, Azure Data Factory, and Azure Synapse Analytics. They will also have...

  • Data Infrastructure Specialist

    3 days ago

    Gujrat, Punjab, Pakistan beBeeData Full time 20,000,000 - 25,000,000

    Design and Develop Large-Scale Data ArchitecturesThis role involves the creation of high-performance data infrastructure, integrating various systems, and ensuring scalability, reliability, and optimal performance.Key Responsibilities Include:Architecting efficient data architectures for integration, storage, processing, orchestration, and...

  • Data Engineer

    3 days ago

    Gujrat, Punjab, Pakistan beBeeDataTransformation Full time 15,000,000 - 25,000,000

    Big Data Transformation ExpertWe are undergoing a comprehensive big data transformation to ensure accurate and consistent data is available precisely when and where it's needed.The goal is to deliver precise, reliable, and high-quality data across multiple systems.This involves ensuring seamless data flow and accuracy across multiple systems.We're looking...

  • Azure SQL Database Expert

    3 days ago

    Gujrat, Punjab, Pakistan beBeeDatabaseAdministrator Full time 800,000 - 1,500,000

    Database Administrator - Azure SQLWe seek a skilled Database Administrator to join our dynamic organization.Aquaint As-Is SQL Server DB structure on premises and design, implement, and maintain database systems on Azure SQL.Monitor database performance and troubleshoot issues for optimal results.Manage backup and recovery processes to ensure data integrity...

  • Senior Data Transformation Specialist

    3 days ago

    Gujrat, Punjab, Pakistan beBeeDataEngineer Full time 9,000,000 - 15,000,000

    Big Data Engineer and TesterWe are undergoing a significant transformation to ensure our data is accurate, consistent, and available when needed.Responsibilities:Write PySpark and SQL scripts to validate data pipelines, transformations, and integrations.Design and run tests for data validation, storage, and retrieval using Azure services like Data Lake,...

  • Chief Data Architect

    3 days ago

    Gujrat, Punjab, Pakistan beBeeDataEngineer Full time $90,000 - $120,000

    About the Job Opportunity This is a remote, contract position focused on designing, building, and maintaining infrastructure for data integration, storage, processing, and analytics. We are looking for an experienced Senior Data Engineer with a strong background in Python, SQL, PySpark, Azure, Databricks, Synapse, Azure Data Lake, DevOps, and cloud-based...

  • Enterprise Data Strategist

    3 days ago

    Gujrat, Punjab, Pakistan beBeeDataArchitect Full time $120,000 - $150,000

    About the RoleThis is a remote opportunity to lead the design, development, and maintenance of an enterprise data platform.The ideal candidate will collaborate with data engineers, BI engineers, and data business analysts across various platform layers, ensuring a scalable, secure, and high-performance data infrastructure. This role requires a deep...

  • High-Performance Data Architect

    3 days ago

    Gujrat, Punjab, Pakistan beBeeDataEngineer Full time 15,000,000 - 30,000,000

    Senior Data Engineer OpportunityThis is a remote position responsible for designing, building, and maintaining the infrastructure required for data integration, storage, processing, and analytics.You will develop in an Agile environment, contributing to the architecture, design, and implementation of Data products in the Aviation Industry, including...