Data Engineering Team Lead at Fusemachines

5 days ago

Islamabad, Islamabad, Pakistan FuseMachines Full time

About Fusemachines

Fusemachines is a pioneer in AI strategy, talent, and education services, providing innovative solutions to businesses worldwide. With operations in 4 countries and over 400 employees, Fusemachines aims to democratize AI and drive growth.

About the Role

This is a contract position responsible for designing, developing, and maintaining large-scale data infrastructure for integration, storage, processing, and analytics (BI, visualization, and Advanced Analytics).

We seek a skilled Senior Data Engineer with a strong background in Python, SQL, PySpark, Azure, Databricks, Synapse, Azure Data Lake, DevOps, and cloud-based large-scale data applications. The ideal candidate will have a passion for data quality, performance, and cost optimization, with experience in Agile development environments.

Qualifications & Experience

  • A full-time Bachelor's degree in Computer Science or similar.
  • At least 5 years of experience as a data engineer with expertise in Databricks, Azure, DevOps, or other hyperscalers.
  • Experience with Azure DevOps, GitHub, and proven experience delivering large-scale projects and products for Data and Analytics, including migrations.
  • The following certifications:
    • Databricks Certified Associate Developer for Apache Spark
    • Databricks Certified Data Engineer Associate
    • Microsoft Certified: Azure Fundamentals
    • Microsoft Certified: Azure Data Engineer Associate
    • Microsoft Exam: Designing and Implementing Microsoft DevOps Solutions (nice to have)

Required Skills/Competencies

  • Strong programming skills in one or more languages such as Python, Scala, and proficiency in writing efficient and optimized code for data integration, migration, storage, processing, and manipulation.
  • Experience with SQL and writing advanced SQL queries.
  • Thorough understanding of big data principles, techniques, and best practices.
  • Experience with scalable and distributed Data Processing Technologies such as Spark/PySpark, DBT, and Kafka to handle large volumes of data.
  • Solid Databricks development experience with significant Python, PySpark, Spark SQL, Pandas, NumPy in Azure environment.
  • Experience in designing and implementing efficient ELT/ETL processes in Azure and Databricks and using open-source solutions to develop custom integration solutions as needed.
  • Skilled in Data Integration from different sources such as APIs, databases, flat files, event streaming.
  • Expertise in data cleansing, transformation, and validation.
  • Proficiency with Relational Databases (Oracle, SQL Server, MySQL, Postgres, or similar) and NonSQL Databases (MongoDB or Table).
  • Good understanding of Data Modeling and Database Design Principles.
  • Experience in designing and implementing Data Warehousing, data lake, and data lake house solutions in Azure and Databricks.
  • Experience with Delta Lake, Unity Catalog, Delta Sharing, Delta Live Tables (DLT).
  • Strong understanding of the software development lifecycle (SDLC), especially Agile methodologies.
  • Knowledge of SDLC tools and technologies Azure DevOps and GitHub.
  • Understanding of DevOps principles, including continuous integration, continuous delivery (CI/CD), infrastructure as code (IaC), configuration management, automated testing, performance tuning, and cost management and optimization.
  • Knowledge in cloud computing specifically in Microsoft Azure services related to data and analytics.
  • Experience in Orchestration using technologies like Databricks workflows and Apache Airflow.
  • Knowledge of data structures and algorithms and good software engineering practices.
  • Proven experience migrating from Azure Synapse to Azure Data Lake, or other technologies.
  • Strong analytical skills to identify and address technical issues, performance bottlenecks, and system failures.
  • Proficiency in debugging and troubleshooting issues in complex data and analytics environments and pipelines.
  • Understanding of Data Quality and Governance.
  • Experience with BI solutions including PowerBI is a plus.
  • Strong written and verbal communication skills.
  • Ability to document processes, procedures, and deployment configurations.
  • Understanding of security practices, including network security groups, Azure Active Directory, encryption, and compliance standards.
  • Ability to implement security controls and best practices within data and analytics solutions.
  • Self-motivated with the ability to work well in a team, and experienced in mentoring and coaching different members of the team.
  • A willingness to stay updated with the latest services, Data Engineering trends, and best practices in the field.
  • Comfortable with picking up new technologies independently and working in a rapidly changing environment with ambiguous requirements.
  • Care about architecture, observability, testing, and building reliable infrastructure and data pipelines.

Responsibilities

  • Architect, design, develop, test, and maintain high-performance, large-scale, complex data architectures.
  • Contribute to detailed design, architectural discussions, and customer requirements sessions.
  • Actively participate in the design, development, and testing of big data products.
  • Construct and fine-tune Apache Spark jobs and clusters within the Databricks platform.
  • Migrate out of Azure Synapse to Azure Data Lake or other technologies.
  • Assess best practices and design schemas that match business needs for delivering a modern analytics solution.
  • Design and implement data models and schemas that support efficient data processing and analytics.
  • Design and develop clear, maintainable code with automated testing.
  • Collaborate with cross-functional teams to understand data requirements and develop data solutions.
  • Evaluate and implement new technologies and tools to improve data integration, data processing, storage, and analysis.
  • Evaluate, design, implement, and maintain data governance solutions.
  • Continuously monitor and fine-tune workloads and clusters to achieve optimal performance.
  • Provide guidance and mentorship to junior team members.
  • Maintain clear and comprehensive documentation of the solutions.
  • Promote and enforce best practices in data engineering, data governance, and data quality.
  • Ensure data quality and accuracy.
  • Design, implement, and maintain data security and privacy measures.
  • Be an active member of an Agile team.

  • Senior Data Engineer Azure Databricks Expert

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines, a leader in AI strategy, talent, and education services, empowers businesses to unlock their potential through democratized AI. With a presence in 4 countries and over 400 employees, Fusemachines seeks to drive innovation globally.About the RoleThis role involves designing, developing, and maintaining large-scale data...

  • Big Data Infrastructure Specialist

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines, a leader in AI strategy, talent, and education services, drives innovation through democratized AI. With a presence in 4 countries and over 400 employees, Fusemachines seeks to empower businesses worldwide.About the RoleThis role involves designing, developing, and maintaining large-scale data infrastructure for integration,...

  • Cloud Data Architect

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines is a leading AI strategy, talent, and education services provider with a mission to democratize AI. With operations in 4 countries and over 400 employees, Fusemachines aims to bring its global expertise in AI to transform companies worldwide.About the RoleThis is a remote contract position responsible for designing, building,...

  • Data-Driven Business Transformation Specialist

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines is a leading AI provider, dedicated to delivering cutting-edge AI products and solutions that transform industries worldwide.Our mission, led by Sameer Maskey, Ph.D., an Adjunct Associate Professor at Columbia University, is to democratize AI and unlock the potential of the global talent pool from underserved communities.We...

  • Java Software Engineer

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    The Role:Job DescriptionAs a Full Stack Developer, you will collaborate with cross-functional groups to identify areas of enhancement. You will troubleshoot, debug issues, and propose technical solutions with estimates. Your responsibilities will include designing and building solutions, enhancing the core platform by improving APIs and SQL queries to be...

  • Microsoft Power Platform Engineer

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines is a pioneering AI company, dedicated to delivering cutting-edge AI products and solutions that transform industries worldwide.Our mission, led by Sameer Maskey, Ph.D., an Adjunct Associate Professor at Columbia University, is to democratize AI and unlock the potential of the global talent pool from underserved communities.We...

  • Mid-Level Cloud Software Engineer

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    Job TitleNet Cloud DeveloperAbout UsFusemachines is a leading AI company founded by Dr. Sameer Maskey with a mission to democratize AI and provide opportunities to individuals from underprivileged communities. Our company has extensive experience in delivering high-quality AI solutions across various sectors.Job DescriptionOur organization seeks a skilled...

  • Senior Fullstack Developer

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About Fusemachines:OverviewFusemachines is a renowned AI company with over 10 years of experience in delivering cutting-edge AI products and solutions to diverse industries. Founded by Sameer Maskey, Ph.D., an Adjunct Associate Professor at Columbia University, our mission is to democratize AI and harness the power of global AI talent from underserved...

  • AI Solutions Developer

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines is a leading provider of cutting-edge AI products and solutions, dedicated to delivering innovative technological advancements to various industries.Founded by Sameer Maskey, Ph.D., an Adjunct Associate Professor at Columbia University, our mission is to democratize AI and leverage the global talent pool from underserved...

  • AI Software Engineer

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines is a pioneering AI company that specializes in delivering innovative AI solutions to diverse industries. Founded by Sameer Maskey, Ph.D., our mission is to leverage AI technology to empower businesses and create opportunities for underrepresented communities.We have a significant presence in multiple countries with a talented...

  • Senior Cloud Application Developer

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About UsFusemachines is a pioneering AI company driven by the vision of democratizing AI and empowering global talent from underserved communities. With over a decade of experience in delivering innovative AI solutions, we continue to push the boundaries of technology.Job DescriptionAt Fusemachines, we are seeking a highly skilled Mid-level Cloud Developer...

  • Full-Stack Azure Cloud Expert

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    Job OverviewAt Fusemachines, we are on a mission to revolutionize the AI industry by leveraging global talent from underserved communities. As a leading AI company, we deliver cutting-edge AI solutions across various sectors and foster a collaborative and inclusive work environment.Job DescriptionOur organization is seeking a skilled Mid-level Developer to...

  • Cloud Developer Specialist

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    Company OverviewFusemachines is a leading AI company with over a decade of experience delivering cutting-edge AI solutions to various industries. Founded by Sameer Maskey, Ph.D., our mission is to democratize AI and leverage global talent from underserved communities.Job Description:We are seeking an experienced Mid-level Developer to join our team. The...

  • Cloud Enterprise Application Developer

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines is a cutting-edge AI company, dedicated to delivering innovative AI products and solutions that transform industries worldwide.Our mission, led by Sameer Maskey, Ph.D., an Adjunct Associate Professor at Columbia University, is to democratize AI and unlock the potential of the global talent pool from underserved communities.We...

  • Backend Developer

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines is a leading provider of state-of-the-art AI solutions to various industries. Founded by Sameer Maskey, Ph.D., an Adjunct Associate Professor at Columbia University, we aim to harness global AI talent from underserved communities and promote AI-driven innovation.We have a significant presence in multiple countries with a...

  • Senior Python Developer

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines is a leading AI company that delivers cutting-edge AI products and solutions to various industries. Our mission, founded by Sameer Maskey, Ph.D., an Adjunct Associate Professor at Columbia University, is to democratize AI and tap into global AI talent from underrepresented communities.We have a strong presence in multiple...

  • Software Architect

    5 days ago

    Islamabad, Islamabad, Pakistan FuseMachines Full time

    About FusemachinesFusemachines is a renowned AI company that specializes in providing cutting-edge AI solutions to diverse industries. Founded by Sameer Maskey, Ph.D., we aim to foster AI innovation and promote diversity in the tech industry.We have a strong presence in multiple countries with a talented team of over 400 full-time employees dedicated to...

  • Software Engineer Team Leader

    5 days ago

    Islamabad, Islamabad, Pakistan AT Technology Services Ltd Full time

    Senior Back-End Software Engineer and Team LeadWe are seeking a highly skilled Senior Back-End Software Engineer and Team Lead to join our team. The ideal candidate will have proven experience working with Linux and various web servers, strong understanding of cloud architecture principles, and proficiency in popular frameworks like Symfony, Django, and...

  • Data Engineering Lead

    5 days ago

    Islamabad, Islamabad, Pakistan VisionX Technologies, Inc. Full time

    Job OverviewWe are seeking an experienced Senior Data Engineer to join our thriving Data & Analytics team and contribute to the development of high-quality Data Lakehouse products and solutions.As a Senior Data Engineer, you will leverage your expertise in data engineering, software development, data modeling, integration, and business acumen, combined with...

  • Data Engineering Lead

    5 days ago

    Islamabad, Islamabad, Pakistan VisionX Technologies, Inc. Full time

    About UsVisionX Technologies, Inc. is a leading innovation partner to Fortune 1000 brands, providing product strategy and custom application development leveraging agile methodologies, technology accelerators, and creating Intellectual Property.We have been recognized as one of the Top 10 Most Innovative Companies of 2020 by Fast Company, alongside Microsoft...