Data engineer- ADF- Business Req

3 days ago


Hyderabad City Taluka, Pakistan techcarrot Full time

Job Title: Data
Engineer

Job Summary:

Are you passionate about building scalable data pipelines, optimizing ETL processes, and designing efficient data models? We are looking for a Databricks Data Engineer to join our team and play a key role in managing and transforming data in Azure cloud environments.

In this role, you will work with Azure Data Factory (ADF), Databricks, Python, and SQL to develop robust data ingestion and transformation workflows. You'll also be responsible for integrating SAP IS-Auto data, optimizing performance, and ensuring data quality & governance.

If you have strong experience in big data processing, distributed computing (Spark), and data modeling, we'd love to hear from you

Key Responsibilities:

  1. Develop & Optimize ETL Pipelines:
    • Build robust and scalable data pipelines using ADF, Databricks, and Python for data ingestion, transformation, and loading.
  2. Data Modeling & Systematic Layer Modeling:
    • Design logical, physical, and systematic data models for structured and unstructured data.
  3. Integrate SAP IS-Auto:
    • Extract, transform, and load data from SAP IS-Auto into Azure-based data platforms.
  4. Database Management:
    • Develop and optimize SQL queries, stored procedures, and indexing strategies to enhance performance.
  5. Big Data Processing:
    • Work with Azure Databricks for distributed computing, Spark for large-scale processing, and Delta Lake for optimized storage.
  6. Data Quality & Governance:
    • Implement data validation, lineage tracking, and security measures for high-quality, compliant data.
  7. Collaboration:
    • Work closely with business analysts, data scientists, and DevOps teams to ensure data availability and usability.
  8. Testing and Debugging:
    • Write unit tests and perform debugging to ensure the implementation is robust and error-free.
    • Conduct performance optimization and security audits.

Required Skills and Qualifications:

  1. Azure Cloud Expertise:
    • Strong experience in Azure Data Factory (ADF), Databricks, and Azure Synapse.
  2. Programming:
    • Proficiency in Python for data processing, automation, and scripting.
  3. SQL & Database Skills:
    • Advanced knowledge of SQL, T-SQL, or PL/SQL for data manipulation.
  4. SAP IS-Auto Data Handling:
    • Experience integrating SAP IS-Auto as a data source into data pipelines.
  5. Data Modeling:
    • Hands-on experience in dimensional modeling, systematic layer modeling, and entity-relationship modeling.
  6. Big Data Frameworks:
    • Strong understanding of Apache Spark, Delta Lake, and distributed computing.
  7. Performance Optimization:
    • Expertise in query optimization, indexing, and performance tuning.
  8. Data Governance & Security:
    • Knowledge of RBAC, encryption, and data privacy standards.

Preferred Qualifications:

  1. Experience with CI/CD for data pipelines using Azure DevOps.
  2. Knowledge of Kafka/Event Hub for real-time data processing.
  3. Experience with Power BI/Tableau for data visualization (not mandatory but a plus).
#J-18808-Ljbffr

  • Hyderabad City Taluka, Pakistan techcarrot Full time

    Job Title: Business IntelligenceEngineerJob Summary:We are seeking a highly skilled andexperienced Business Intelligence (BI) Engineer to design, develop, andmaintain cutting-edge BI solutions that empower data-driven decision-makingacross the organization. This role is ideal for individuals with a strongtechnical background in data analytics, visualization,...


  • Hyderabad City Taluka, Pakistan techcarrot Full time

    Required Skills and QualificationsAzure Cloud Expertise: Strong experience in Azure Data Factory (ADF), Databricks, and Azure Synapse.Programming: Proficiency in Python for data processing, automation, and scripting.SQL and Database Skills: Advanced knowledge of SQL, T-SQL, or PL/SQL for data manipulation.SAP IS-Auto Data Handling: Experience integrating SAP...


  • Hyderabad City Taluka, Pakistan techcarrot Full time

    Key ResponsibilitiesData Ingestion and Transformation: Build robust and scalable data pipelines using ADF, Databricks, and Python for data ingestion, transformation, and loading.Data Modeling: Design logical, physical, and systematic data models for structured and unstructured data.SAP IS-Auto Integration: Extract, transform, and load data from SAP IS-Auto...


  • Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full time

    At Warner Bros. Discovery, we are looking for a skilled Data Engineering Manager to lead our analytics team in driving business growth through data-driven insights.About the Role:This Senior Analytics Lead position is responsible for designing, implementing, and maintaining large-scale analytics solutions that support various business functions within Warner...


  • Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full time

    Welcome to Warner Bros. Discovery, where data meets storytelling. Our Data & Analytics organization is seeking a skilled Senior Analytics Engineer to lead our efforts in data pipeline, strategy, and visualization.About the RoleAs a Senior Analytics Engineer, you will be responsible for designing and implementing data models that support flexible querying and...

  • Data Engineering Lead

    2 weeks ago


    Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full time

    Welcome to Warner Bros. Discovery, a global leader in entertainment and media. Our company is built on a foundation of creativity, innovation, and collaboration.We are seeking a highly skilled Principal Data Engineer to join our Data & Shared Services organization. This role will be responsible for leading the engineering and development of complex data...


  • Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full time

    Warner Bros. Discovery is a global leader in the entertainment and media industry, committed to creating innovative stories and experiences that captivate audiences worldwide.We are currently seeking a highly skilled Director, Data Platforms to join our team and lead the development of cutting-edge data platforms. As a key member of our data engineering...


  • Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full time

    Welcome to Warner Bros. Discovery, where innovation meets creativity.As a Staff Software Engineer in our Data Platform team, you will play a crucial role in designing and building high-volume data pipelines and systems that drive business growth and inform strategic decisions.We are seeking an experienced engineer with a passion for solving complex...


  • Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full time

    Welcome to Warner Bros. Discovery… the stuff dreams are made of.Who We Are…When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD's vast portfolio of iconic content and beloved brands, are the storytellers bringing our characters to life,...


  • Hyderabad City Taluka, Pakistan Backbase Full time

    About Us">Backbase is a leading provider of digital banking platforms, helping banks and financial institutions deliver exceptional customer experiences. Our Data as a Service platform enables customers to run analytical, AI, and ML workloads on data stored in our Engagement Banking Platform.">We're a fast-paced and innovative company, always looking for...

  • Senior Data Engineer

    4 weeks ago


    Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full time

    Welcome to Warner Bros. Discovery… the stuff dreams are made of.Who We Are…When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD's vast portfolio of iconic content and beloved brands, are the storytellers bringing our characters to life,...

  • Data Engineer Role

    5 days ago


    Hyderabad City Taluka, Pakistan Confiz Full time

    At Confiz, we're seeking an experienced and skilled Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing, building, and maintaining scalable data solutions to support critical business needs.The ideal candidate will have hands-on experience with distributed data platforms such as BigQuery,...

  • Engineering Director

    2 weeks ago


    Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full time

    Company Overview:Warner Bros. Discovery is a global media and entertainment company that combines Warner Media's premium entertainment, sports, and news assets with Discovery's leading non-fiction and international entertainment and sports businesses.Job Description:We are seeking an experienced Senior Engineering Manager to lead our data platform efforts...


  • Hyderabad City Taluka, Pakistan Qualcomm Technologies, Inc Full time

    Requirements:We are seeking a highly skilled Senior Big Data Engineer with 2+ years of academic or work experience with programming languages such as C, C++, Java, Python, etc. The ideal candidate will have a bachelor's degree in engineering, information systems, computer science, or a related field. Experience with data modeling, data warehousing, and...


  • Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full time

    Welcome to a world of limitless possibilities, where technology and creativity come together to shape the future. At Warner Bros. Discovery, we are on a mission to drive innovation and excellence in all aspects of our business.As a Senior Analytics Engineer, you will play a critical role in leading our data pipeline, strategy, and visualization efforts for...


  • Hyderabad City Taluka, Pakistan Astronomer Full time

    Astronomer designed Astro, an industry-leading, orchestration-first DataOps platform for data teams. Powered by Airflow, Astro accelerates building reliable data products that unlock insights, unleash AI value, and drive data-driven applications.We're a globally-distributed and rapidly growing venture-backed team of learners, innovators, and collaborators....


  • Hyderabad City Taluka, Pakistan HSBC Full time

    We are looking for a skilled Cloud Infrastructure Professional to join our Platform & Data Engineering Team. In this role, you will contribute to delivering critical analytics engineering requirements for our strategic cybersecurity data lake and analytics platform.Key Responsibilities:You will work closely with experienced engineers to develop data...


  • Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full time

    At Warner Bros. Discovery, we believe that our employees are our greatest asset. That's why we're committed to creating a workplace culture that supports their growth, well-being, and success.We're seeking a seasoned Principal Data Engineer to join our Data & Shared Services organization. In this role, you'll lead the development of complex data management...


  • Hyderabad City Taluka, Pakistan techcarrot Full time

    Data EngineerAs a Data Engineer in our team, you will be responsible for assessing complex new data sources and quickly turning these into business insights. You also will support the implementation and integration of these new data sources into our Azure Data platform.Responsibilities:You are detailed reviewing and analyzing structured, semi-structured and...


  • Hyderabad City Taluka, Pakistan Confiz Full time

    We are seeking an experienced and motivated Data Engineer to join our team. In this role, you will design, build, and maintain scalable data solutions to support critical business needs. You will work with distributed data platforms, cloud infrastructure, and modern data engineering tools to enable efficient data processing, storage, and analytics. The role...