Big Data Infrastructure Specialist

4 days ago


Islamabad, Islamabad, Pakistan FuseMachines Full time

About Fusemachines

Fusemachines, a leader in AI strategy, talent, and education services, drives innovation through democratized AI. With a presence in 4 countries and over 400 employees, Fusemachines seeks to empower businesses worldwide.

About the Role

This role involves designing, developing, and maintaining large-scale data infrastructure for integration, storage, processing, and analytics. As a Senior Data Engineer, you will contribute to Agile development environments, focusing on data quality, performance, and cost optimization.

Qualifications & Experience

  • A Bachelor's degree in Computer Science or similar.
  • At least 5 years of experience as a data engineer with expertise in Databricks, Azure, DevOps, or other hyperscalers.
  • Experience with Azure DevOps, GitHub, and delivering large-scale projects and products for Data and Analytics, including migrations.
  • The following certifications:
    • Databricks Certified Associate Developer for Apache Spark
    • Databricks Certified Data Engineer Associate
    • Microsoft Certified: Azure Fundamentals
    • Microsoft Certified: Azure Data Engineer Associate
    • Microsoft Exam: Designing and Implementing Microsoft DevOps Solutions (nice to have)

Required Skills/Competencies

  • Strong programming skills in Python, Scala, and SQL, with proficiency in writing efficient and optimized code.
  • Experience with big data principles, techniques, and best practices.
  • Knowledge of scalable and distributed Data Processing Technologies such as Spark/PySpark, DBT, and Kafka.
  • Experience in designing and implementing efficient ELT/ETL processes in Azure and Databricks.
  • Skilled in Data Integration, data cleansing, transformation, and validation.
  • Proficiency with Relational Databases and NonSQL Databases.
  • Understanding of Data Modeling and Database Design Principles.
  • Experience in designing and implementing Data Warehousing, data lake, and data lake house solutions.
  • Understanding of DevOps principles, including CI/CD, IaC, configuration management, automated testing, and cost management and optimization.
  • Knowledge in cloud computing specifically in Microsoft Azure services related to data and analytics.
  • Experience in Orchestration using technologies like Databricks workflows and Apache Airflow.
  • Knowledge of data structures and algorithms and good software engineering practices.
  • Proven experience migrating from Azure Synapse to Azure Data Lake, or other technologies.
  • Strong analytical skills to identify and address technical issues, performance bottlenecks, and system failures.
  • Proficiency in debugging and troubleshooting issues in complex data and analytics environments and pipelines.
  • Understanding of Data Quality and Governance.
  • Experience with BI solutions including PowerBI is a plus.
  • Strong written and verbal communication skills.
  • Ability to document processes, procedures, and deployment configurations.
  • Understanding of security practices, including network security groups, Azure Active Directory, encryption, and compliance standards.
  • Ability to implement security controls and best practices within data and analytics solutions.
  • Self-motivated with the ability to work well in a team, and experienced in mentoring and coaching different members of the team.
  • A willingness to stay updated with the latest services, Data Engineering trends, and best practices in the field.
  • Comfortable with picking up new technologies independently and working in a rapidly changing environment with ambiguous requirements.
  • Care about architecture, observability, testing, and building reliable infrastructure and data pipelines.

Responsibilities

  • Architect, design, develop, test, and maintain high-performance, large-scale, complex data architectures.
  • Contribute to detailed design, architectural discussions, and customer requirements sessions.
  • Actively participate in the design, development, and testing of big data products.
  • Construct and fine-tune Apache Spark jobs and clusters within the Databricks platform.
  • Migrate out of Azure Synapse to Azure Data Lake or other technologies.
  • Assess best practices and design schemas that match business needs for delivering a modern analytics solution.
  • Design and implement data models and schemas that support efficient data processing and analytics.
  • Design and develop clear, maintainable code with automated testing.
  • Collaborate with cross-functional teams to understand data requirements and develop data solutions.
  • Evaluate and implement new technologies and tools to improve data integration, data processing, storage, and analysis.
  • Evaluate, design, implement, and maintain data governance solutions.
  • Continuously monitor and fine-tune workloads and clusters to achieve optimal performance.
  • Provide guidance and mentorship to junior team members.
  • Maintain clear and comprehensive documentation of the solutions.
  • Promote and enforce best practices in data engineering, data governance, and data quality.
  • Ensure data quality and accuracy.
  • Design, implement, and maintain data security and privacy measures.
  • Be an active member of an Agile team.


  • Islamabad, Islamabad, Pakistan Aricrypta Full time

    Role DescriptionWe are looking for a Data Engineer to lead the development of our big data infrastructure, focusing on scalability, reliability, and efficiency.Key DutiesArchitect and implement data pipelines utilizing Azure Data Factory and Azure Logic Apps.Develop and maintain cloud-based data solutions using AWS, Google Cloud Platform, or Azure.Implement...


  • Islamabad, Islamabad, Pakistan Evamp & Saanga Full time

    About This RoleWe are seeking a highly skilled Big Data Engineer to join our team at Evamp & Saanga. As a key member of our data engineering team, you will design, develop, and implement large-scale data processing systems that meet the needs of our business.Responsibilities include:Designing and building applications and frameworks for big data processing,...


  • Islamabad, Islamabad, Pakistan Confiz Full time

    Confiz is seeking a skilled Big Data QA Automation professional to join our team.Key ResponsibilitiesDevelop and automate processes for gathering expected results from data sources and comparing data results from testing.Assist with the development and maintenance of smoke, performance, functional, and regression tests to ensure code is functioning as...

  • Big Data Specialist

    5 days ago


    Islamabad, Islamabad, Pakistan VisionX Technologies, Inc. Full time

    About VisionXVisionX is a leading innovation partner to Fortune 1000 brands, providing product strategy and custom application development leveraging agile methodologies, technology accelerators, and creating Intellectual Property.We have been recognized as one of the Top 10 Most Innovative Companies of 2020 by Fast Company, alongside Microsoft and Snap...


  • Islamabad, Islamabad, Pakistan Evamp & Saanga Full time

    Job DescriptionAs a Big Data Engineer at Evamp & Saanga, you will play a crucial role in designing, developing, and implementing large-scale data processing systems. Your primary focus will be on leveraging big data technologies to build robust and efficient solutions for managing and analyzing vast amounts of data.Key Responsibilities:Designing and...


  • Islamabad, Islamabad, Pakistan Confiz Full time

    Are you a motivated and experienced professional looking for a new challenge? Do you have a passion for Big Data QA Automation?Key ResponsibilitiesDevelop and execute comprehensive test plans to ensure high-quality data and accurate results.Collaborate with the development team to integrate automated tests into the CI/CD pipeline.Use industry-standard tools...


  • Islamabad, Islamabad, Pakistan Confiz Full time

    Confiz is committed to delivering exceptional results through our talented team of professionals.Key ResponsibilitiesDevelop and implement effective test automation strategies to ensure high-quality data and accurate results.Collaborate with the development team to integrate automated tests into the CI/CD pipeline.Use industry-standard tools to write,...


  • Islamabad, Islamabad, Pakistan NorthBay Solutions Full time

    NorthBay Solutions Job DescriptionWe are looking for a skilled Data Engineer to join our team at NorthBay Solutions. In this role, you will be responsible for designing, developing, and implementing robust data pipelines using Microsoft, Databricks Stack.About the RoleYou will assess complex new data sources, turning them into business insights, and support...


  • Islamabad, Islamabad, Pakistan Arpatech (Pvt) Ltd Full time

    Job DescriptionAs a Data Engineer at Arpatech (Pvt) Ltd, you will be responsible for designing, building, and maintaining the infrastructure required to manage and process large volumes of data.You will work closely with cross-functional teams to ensure efficient flow, security, and availability of data, enabling analytics reporting and AI-driven...


  • Islamabad, Islamabad, Pakistan Touchstone Communications Full time

    Role SummaryThis is a challenging role that requires a skilled SQL DBA to lead our data infrastructure efforts. The ideal candidate will have a proven track record of designing and implementing complex database systems, collaborating with developers, and maintaining high levels of database performance.What You Will DoDesign and implement SQL clusters to...


  • Islamabad, Islamabad, Pakistan HR Ways Full time

    We're actively hiring for a talented Monitoring Infrastructure Specialist to join our team at HR Ways in Islamabad.Job DescriptionThe successful candidate will be responsible for designing, implementing, and configuring monitoring solutions for applications, infrastructure, and networks. This includes setting up monitoring agents, data collection, and...


  • Islamabad, Islamabad, Pakistan Aricrypta Full time

    Job OverviewAricrypta is seeking a skilled Data Engineer to design, build, and maintain infrastructure enabling our organization to store, process, and analyze large volumes of data.Main ResponsibilitiesDesign and implement data pipelines moving data from various sources into a central repository or data lake.Develop and maintain data warehouses and data...

  • Big Data Specialist

    5 days ago


    Islamabad, Islamabad, Pakistan 10Pearls Full time

    We are looking for an experienced Data Engineer to join our team at 10Pearls. Your primary responsibility will be to develop, construct, test, and maintain optimal data pipeline architecture.Key Responsibilities:Develop, construct, test, and maintain optimal data pipeline architecture.Assemble large, complex data sets that meet functional / non-functional...


  • Islamabad, Islamabad, Pakistan ieng Group Full time

    The ieng Group is committed to providing equal opportunities in all aspects of our business, including recruitment. We strive to create an inclusive environment where everyone feels valued and respected.Job DescriptionWe are seeking a highly skilled Data Center Engineer to join our team. The successful candidate will be responsible for designing,...


  • Islamabad, Islamabad, Pakistan EWS Group Full time

    Cloud Infrastructure SpecialistJob Summary:The EWS Group is seeking a Cloud Infrastructure Specialist to oversee the deployment and management of cloud-based infrastructure. This role will focus on ensuring the efficient operation of cloud services, including the Salesforce Lightning platform.


  • Islamabad, Islamabad, Pakistan NexGen-X Full time

    Wireless Infrastructure Specialist Job DescriptionNexGen-X is a digital transformation powerhouse working in the computer networking industry. We are looking for a skilled Wireless Infrastructure Specialist to join our team.About Company:NexGen-X is building tomorrow's solutions today. As a Wireless Infrastructure Specialist, you will be responsible for...


  • Islamabad, Islamabad, Pakistan NADRA Technologies Ltd Full time

    About the RoleWe are looking for an experienced IT professional to join our team as an IT Infrastructure Specialist. The successful candidate will be responsible for designing, implementing, and maintaining our IT infrastructure to ensure high availability and performance.Key Responsibilities:Database AdministrationNetwork ManagementSystem...


  • Islamabad, Islamabad, Pakistan EWS Group Full time

    Job Overview:We are seeking a highly skilled Cloud Infrastructure Specialist to join our team at EWS Group. The successful candidate will be responsible for designing, implementing, and maintaining scalable cloud infrastructure solutions for our clients.


  • Islamabad, Islamabad, Pakistan NADRA Technologies Ltd Full time

    NADRA Technologies Ltd: Where Innovation Meets ExcellenceWe are seeking a highly skilled Infrastructure Specialist to join our team at NADRA Technologies Ltd. As a key member of our consulting services, you will be responsible for designing, implementing, and managing IT infrastructure solutions for our clients.About the Position:You will work closely with...


  • Islamabad, Islamabad, Pakistan Nysonian Inc. Full time

    Job Title: IT ManagerWe are seeking an accomplished IT Manager to lead our IT operations, overseeing the management of hardware, software, networks, and IT systems to ensure seamless performance and security.Key Responsibilities:Infrastructure Management: Install, maintain, and upgrade hardware, software, networks, and IT systems to optimize performance and...