Azure Big data Engineer | 4 to 8 Yrs | WFO | SOW 2779

24 hours ago

Bangalore preferred Hyderabad, Pakistan PradeepIT Consulting Services Full time ₹1,200,000 - ₹25,000,000 per year


Job Title: Azure Big Data Engineer

Years of Experience: 4 to 8 Years

Relevant Experience: 4+ Years

Headcount: 10

Job/Work type: WFO (5 days a week for First 3 months, later its Hybrid 3 days a week)

Location: Bangalore (preferred), Hyderabad

Time Zone: IST time zone (In IST 10:00 am to 7:00 pm)

Project Name: -

Client: Bosch

End Client: -

Other details:

Role & Responsibilities / Job Description:

Mandatory Skills:

Azure Skills Like:

  • Azure Databricks
  • Azure Synapse
  • Azure Data Factory (ADF)
  • Azure Integration Services
  • Azure Functions
  • Azure Data Lake
  • Azure API Management
  • CI/CD and PowerShell Scripts
  • ETL Pipelines
  • Unity Catalog
  • Azure SQL Database
  • SQL Coding

Responsibilities :

  • Overall 4 8 years of experience with a minimum 4+ years of relevant professional work experience with Azure Synapse and Azure Databricks
  • ETL Consultant - Databricks / ADF / Synapse, building common logging mechanism for all ETL processes, Build the ingestion process for fact data tables, Building the pipeline temporary pipeline to load data into on premise, Prepare the flow diagram covering it up all required ETL's Prepare folder design document Netezza Box
  • Design, implement, and maintain data pipelines for data ingestion, processing, and transformation in Azure.
  • Work together with data analysts to understand the needs for data and create effective data workflows.
  • Create and maintain data storage solutions including Azure Databricks, Azure Functions, Azure Data Lake, Azure Synapse, Azure Integration Services, Azure API Management, CI/CD and PowerShell Scripts, ADF, ETL Pipelines, Unity Catalog, Azure SQL Database, SQL Coding, Azure SQL Database, Azure Data Lake.
  • Utilizing Azure Data Factory to create and maintain ETL (Extract, Transform, Load) operations using ADF pipelines.
  • Hands-On Experience working on Data Bricks for implementing Transformations and Delta Lake.
  • Hands-On Experience working on Serverless SQL Pool, Dedicated SQL Pool.
  • Use ADF pipelines to orchestrate the end to end data transformation including the execution of DataBricks notebooks.
  • Should have experience working on Medallion Architecture
  • Experience working on CI/CD pipelines using Azure DevOps
  • Attaching both ADF and ADB to DevOps
  • Creating and managing Azure infrastructure across the landscape using Bicep
  • Implementing data validation and cleansing procedures will ensure the quality, integrity, and dependability of the data.
  • Improve the scalability, efficiency, and cost-effectiveness of data pipelines.
  • Monitoring and resolving data pipeline problems will guarantee consistency and availability of the data.
  • Good to have exposure on Power BI, Power Automate
  • Good to have knowledge on Azure fundamentals.
Desirable skills
  • Good ability to anticipate issues and formulate remedial actions
  • Sound interpersonal and team working skills
  • Sound Knowledge of unit testing methodologies and frameworks
  • Should be able to work independently
  • Good communication skills

  • DB2 Azure Migration SME

    24 hours ago

    Hyderabad,, Pakistan Techwave Consulting Full time $60,000 - $120,000 per year

    Techwave, we are always in an exercise to foster a culture of growth, and inclusivity. We ensure whoever is associated with the brand is being challenged at every step and is provided with all the necessary opportunities to excel in life. People are at the core of everything we do.Join us    Who are we? Techwave is a leading global IT and engineering...

  • Senior Data Engineer

    24 hours ago

    Hyderabad, Remote, Pakistan Coeo Full time 1,500,000 - 3,000,000 per year

    Senior Data EngineerLocation: Hyderabad - RemoteCoeo are trusted data management and analytics experts, delivering technology strategy and support for business. The team have deep technical and commercial experience working with Microsoft Data Services to help our clients optimise their costs and maximise the benefits from their investments in these...

  • Data Lead

    24 hours ago

    Hyderabad,, Pakistan Techwave Consulting Full time $120,000 - $150,000 per year

    Techwave, we are always in an exercise to foster a culture of growth, and inclusivity. We ensure whoever is associated with the brand is being challenged at every step and is provided with all the necessary opportunities to excel in life. People are at the core of everything we do.Join us    Who are we? Techwave is a leading global IT and engineering...

  • Data Engineer III

    24 hours ago

    Chennai / Hyderabad, Pakistan PDI Technologies Full time $60,000 - $120,000 per year

    At PDI Technologies, we empower some of the world's leading convenience retail and petroleum brands with cutting-edge technology solutions that drive growth and operational efficiency.   By "Connecting Convenience" across the globe, we empower businesses to increase productivity, make more informed decisions, and engage faster with customers through...

  • Azure Network Architect

    24 hours ago

    Hyderabad,, Pakistan Techwave Consulting Full time 1,200,000 - 2,400,000 per year

    Techwave, we are always in an exercise to foster a culture of growth, and inclusivity. We ensure whoever is associated with the brand is being challenged at every step and is provided with all the necessary opportunities to excel in life. People are at the core of everything we do.Join us    Who are we? Techwave is a leading global IT and engineering...

  • Cloud Engineer

    24 hours ago

    Hyderabad,, Pakistan Techwave Consulting Full time $80,000 - $120,000 per year

    Techwave, we are always in an exercise to foster a culture of growth, and inclusivity. We ensure whoever is associated with the brand is being challenged at every step and is provided with all the necessary opportunities to excel in life. People are at the core of everything we do.Join us    Who are we? Techwave is a leading global IT and engineering...

  • Senior Analog Design Engineer

    1 day ago

    Hyderabad / Bangalore, Pakistan Omni Design Technologies Full time 1,200,000 - 2,400,000 per year

    The Senior Analog IC Design Engineer will be working on one or two of the following: 1. high speed SerDes, 2. PLL, Oscillators, 3. PMU, 4. Data Converters Should be familiar with Analog design for 1 or two of the ares mentioned above Role and Responsibilities Work with chip/system architecture, physical design, and design teams to implement the...

  • Database Engineer III

    24 hours ago

    Hyderabad / Bangalore / Chennai, Pakistan PDI Technologies Full time $120,000 - $180,000 per year

    At PDI Technologies, we empower some of the world's leading convenience retail and petroleum brands with cutting-edge technology solutions that drive growth and operational efficiency.   By "Connecting Convenience" across the globe, we empower businesses to increase productivity, make more informed decisions, and engage faster with customers through...

  • Senior Data Engineer

    1 day ago

    Hyderabad, Hybrid, Pakistan Coeo Full time $90,000 - $120,000 per year

    About CoeoCoeo are trusted data management and analytics experts, delivering technology strategy and support for business. The team has deep technical and commercial experience working with Microsoft Data Services to help our clients optimise their costs and maximise the benefits from their investments in these technologies. Established over 15 years ago,...

  • Software Development Engineer II

    24 hours ago

    Hyderabad/Bangalore, Pakistan Zeta Full time $100,000 - $150,000 per year

    About Zeta Zeta is a Next-Gen Banking Tech company that empowers banks and fintechs to launch banking products for the future. It was founded by Bhavin Turakhia and Ramki Gaddipati in 2015.Our flagship processing platform - Zeta Tachyon - is the industry's first modern, cloud-native, and fully API-enabled stack that brings together issuance, processing,...