Principal Azure Data Engineer

2 days ago


Lahore, Punjab, Pakistan Confiz Full time

We are looking for an experienced data engineer responsible for expanding and optimizing our big data pipelines. You will be working alongside other engineers and developers working on different layers of the infrastructure. Therefore, a commitment to collaborative problem solving, sophisticated design, and the creation of quality products is essential.

Responsibilities:

  • Collaborate with Big Data Solution Architects to design, prototype, implement, and optimize data ingestion pipelines so that data is shared effectively across various business systems.
  • Build ETL/ELT Ingestion pipelines and analytics solutions using cloud and on-premises technologies.
  • Ensure the design, code and procedural aspects of solution are production ready, in terms of operational, security and compliance standards.
  • Participate in day-to-day project and production delivery status meetings, and provide technical support for faster resolution of issues.

Requirements:

  • Bachelor's degree from an accredited college/university or equivalent work experience.
  • 5+ years of experience performing data engineering, warehousing, publishing and visualization throughout the full data lifecycle.
  • At least two (2) years of experience as an Azure Data Engineer, or in a similar role in a technology or IT consulting environment.
  • 3 years of experience Data profiling, cataloguing and mapping to enable the design and build of technical data flows.
  • 3 years of experience defining ELT/ETL architecture and process design.
  • 2 years of experience performing end-to-end implementation of data warehousing analytics solutions built on MS or Azure platforms.
  • 2 years of experience working with Azure.
  • Experience in creating data pipelines using ETL tools like Airflow, ADF.
  • Experienced in distributing computing technologies like Apache Spark, Databricks.
  • Experienced with stream-processing systems like Spark Streaming, Eventhub.
  • Proficient in one of the object-oriented and functional languages: Python, Java, Scala, etc.
  • Experience of writing effective and maintainable unit and integration tests for ingestion pipelines.
  • Experience of using static analysis and code quality tools.
  • Experience in using code version control repositories like Git.
  • Experienced in security best practices including encryption of sensitive data, encryption at rest and encryption in flight.
  • Experience of building CI/CD pipelines & DevOps orchestration tool like Jenkins.
  • Familiarity with Data Lakehouse, Delta Lake, Apache Hudi.
#J-18808-Ljbffr
  • Azure Data Engineer

    3 weeks ago


    Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    Azure Data EngineerAVANCEON is hiring an Azure Data Engineer for their Lahore office. You will work with a team to develop user-friendly software from start to finish in a fast-moving and collaborative environment.Must-Have Checklist:Practical Experience With:Azure Data FactoryAzure FunctionsData Lake Gen2 and DatabricksSkills in:Python, SQL, Scala, or...


  • Lahore, Punjab, Pakistan HR POD Careers Full time

    Requirements:Bachelors or master's degree in computer science, Engineering, Information Technology, or a related field.5 to 7 years of experience in data engineering with a strong focus on cloud-based solutions.Expertise in Azure Synapse Analytics, including data integration, management, and security.Proficient with SQL, Python, PySpark, and other scripting...


  • Lahore, Punjab, Pakistan Confiz Full time

    This role requires a professional with a deep background in data engineering and expertise in designing and implementing cloud-based data warehousing solutions using Azure Synapse. The ideal candidate will be a strategic thinker with a proven record of accomplishment of delivering scalable and efficient data solutions to meet complex business...


  • Lahore, Punjab, Pakistan HR POD Careers Full time

    About the RoleAs an Azure Data Engineering Expert, you will play a critical role in the design and deployment of large-scale data solutions in Azure environments. Your expertise will include cloud-based data integration, management, and security, as well as experience with Azure Data Factory and ELT/ETL processes.You will work closely with clients or...


  • Lahore, Punjab, Pakistan The Services Tree Full time

    Years of Experience: 5-8 yearsJob Overview:We are seeking a skilled Azure & Fabric Data Engineer to design, develop, and maintain data pipelines using Azure and Fabric data services. The ideal candidate will have experience with CI/CD deployment automation, data science service configuration, and seamless integration of data services. This role requires...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    Unlocking Business Potential through Cutting-Edge Data Engineering and AnalyticsData Pilot is dedicated to empowering businesses with the data-driven insights they need to drive growth and success. As an Associate Data Architect, you will play a critical role in designing and implementing innovative data architectures that support advanced data engineering...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    Empowering Businesses with Real-Time Insights through Advanced Data ArchitecturesData Pilot is committed to helping organizations unlock the full potential of their data assets and maximize business value through our expert engineering and business acumen. As an Associate Data Architect, you will play a crucial role in designing and implementing scalable,...


  • Lahore, Punjab, Pakistan Avanceon Limited Full time

    About Avanceon LimitedAvanceon Limited is a leading provider of innovative technology solutions.Job Description:Senior Big Data Engineer Role SummaryWe are seeking an experienced Senior Big Data Engineer to join our team in the Middle East & South Asia region. As a key member of our data engineering team, you will be responsible for designing, developing,...


  • Lahore, Punjab, Pakistan Confiz Full time

    Big Data Engineer RoleAs a data engineer at Confiz, you will play a crucial role in designing and implementing big data pipelines that enable seamless data sharing across our systems.Key Responsibilities:Collaborate with solution architects to ensure data pipeline effectiveness.Develop ETL/ELT ingestion pipelines using cloud and on-premises...


  • Lahore, Punjab, Pakistan The Services Tree Full time

    Job Overview:The Services Tree is seeking a seasoned Azure & Fabric Data Engineer to join our team. This role requires designing, developing, and maintaining data pipelines using Azure and Fabric data services.Key Responsibilities:Design and develop data pipelines using Azure and Fabric data services.Implement CI/CD deployment automation using Azure DevOps...

  • Azure Engineer Role

    3 days ago


    Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    About the Job:As an Azure Data Engineer at Avanceon Middle East & South Asia, you will play a crucial role in developing and maintaining large-scale data systems. You will be working closely with the data engineering team to design and implement data solutions that meet the needs of our clients.Key Skills and Qualifications:Practical experience with Azure...


  • Lahore, Punjab, Pakistan Systems Limited Full time

    Senior Data Architect Position:Sytems Limited is looking for an experienced senior data architect with expertise in Azure data services, ETL pipelines, big data processing, and data governance. The ideal candidate will have a strong understanding of the telecom industry and experience working with large datasets.Responsibilities:Design and implement data...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    Transforming Data into Business Insights through Innovative SolutionsAs an Associate Data Architect at Data Pilot, you will have the opportunity to work on cutting-edge data engineering and analytics projects that drive business growth and success. Our team is passionate about harnessing the power of data to deliver exceptional results and drive business...


  • Lahore, Punjab, Pakistan Creative Chaos Full time

    Job SummaryA Principal Software Engineer for Python is an associate at Creative Chaos who is mainly responsible for developing server applications of various software platforms including but not limited to writing Restful APIs to be consumed by client applications. A Principal Software Engineer makes sure that the functionality, security, and performance of...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    Associate Data Architect - Analytics & AICompany Overview:At Data Pilot, we're more than just a data product development and consulting services company – we're your strategic partner on the journey from data to decisions. Our mission is simple: to help businesses unlock the full potential of their data assets and maximize business value through our expert...

  • Data Architect Lead

    1 week ago


    Lahore, Punjab, Pakistan Data Pilot Full time

    Data Pilot is a cutting-edge data product development and consulting services company that empowers businesses to unlock the full potential of their data assets. As an Associate Data Architect, you will play a pivotal role in designing and implementing scalable, efficient data architectures that support advanced data engineering and analytics projects.Key...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    Data Pilot: Unlocking Business Value through Scalable Data ArchitecturesAt Data Pilot, we're more than just a data product development and consulting services company – we're your strategic partner on the journey from data to decisions. Our mission is simple: to help businesses unlock the full potential of their data assets and maximize business value...

  • Data Engineer

    3 days ago


    Lahore, Punjab, Pakistan Prime System Solutions Full time

    Education Requirements, Skills, and KnowledgeGood problem solving and decision-making skills; ability to understand and analyze complex issuesSelf-motivated, detail orientated, highly organized and able to handle a variety of tasks and responsibilities in an efficient manner with a high level of qualityExceptional analytical and conceptual thinking...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    At Data Pilot, we're more than just a data product development and consulting services company – we're your strategic partner on the journey from data to decisions. Our mission is simple: to help businesses unlock the full potential of their data assets and maximize business value through our expert engineering and business acumen.In the dynamic landscape...


  • Lahore, Punjab, Pakistan Azure Games Full time

    Company OverviewWe are a technology-driven company, Azure Games, specializing in creating engaging gaming experiences. Our team is passionate about innovation and excellence.Job DescriptionWe are seeking a skilled Senior PHP Developer/Team Lead to lead our PHP team in Lahore. The ideal candidate should possess excellent communication, team management,...