Cloud Infrastructure and Data Engineering Expert

6 days ago

Karachi, Sindh, Pakistan beBee Careers Full time

Job Description:

The Senior Azure Administrator will be responsible for designing, deploying, managing, and optimizing cloud resources and Databricks environments. They will play a crucial role in ensuring seamless integration of data pipelines, analytics platforms, and cloud-based applications while maintaining high levels of security, performance, and scalability.

Main Responsibilities:

  1. Oversee Azure services such as Virtual Machines, Storage, Networking, Azure Kubernetes Service (AKS), and Azure Synapse Analytics.
  2. Manage Azure resources using tools like Azure Portal, PowerShell, and CLI.
  3. Identity and Security:
  4. Administer Azure Active Directory, role-based access control (RBAC), and conditional access policies.
  5. Implement security best practices, including firewall configurations, Key Vault management, and encryption.
  6. Monitoring and Cost Optimization:
  7. Use Azure Monitor, Log Analytics, and Cost Management tools to track performance and resource usage.
  8. Optimize Azure environments for scalability and cost-efficiency.
  9. Backup and Disaster Recovery:
  10. Configure Azure Backup and Site Recovery to ensure high availability.
  11. Databricks Administration:
  12. Platform Management:
  13. Deploy and configure Databricks clusters for performance and cost-efficiency.
  14. Monitor and manage Databricks jobs, pipelines, and workspace configurations.
  15. Integration and Automation:
  16. Ensure seamless integration of Databricks with Azure services like Data Lake, Synapse Analytics, and Azure ML.
  17. Automate workflows using tools like Azure Data Factory (ADF) or Azure Event Grid.
  18. Data Security and Compliance:
  19. Implement fine-grained data access controls using Databricks SQL Analytics and Unity Catalog.
  20. Ensure compliance with GDPR, HIPAA, or other relevant standards.
  21. Performance Optimization:
  22. Monitor cluster performance and troubleshoot bottlenecks in data pipelines or ML workloads.
  23. Optimize Spark job execution and storage formats (e.g., Delta Lake).
  24. Cross-Functional Collaboration:
  25. Work closely with data engineers, analysts, and DevOps teams to ensure end-to-end platform reliability.
  26. Provide guidance on best practices for deploying scalable and efficient data pipelines.

Requirements:

  • Technical Skills:
  • Advanced knowledge of Azure infrastructure, networking, and services like Data Lake, Synapse, AKS, and Key Vault.
  • Hands-on experience with Azure DevOps for CI/CD pipelines.
  • Strong understanding of Spark architecture and Databricks-specific features like Delta Lake and Unity Catalog.
  • Experience with Python, SQL, and Scala for data engineering workflows.
  • Automation and Scripting:
  • Proficiency in scripting with PowerShell, Python, or Bash for automation.
  • Experience with Infrastructure as Code (IaC) using ARM templates, Terraform, or Bicep.
  • Monitoring and Troubleshooting:
  • Familiarity with tools like Azure Monitor, Databricks CLI, and third-party monitoring solutions.
  • Certifications:
  • Microsoft Certified: Azure Solutions Architect Expert (required)
  • Databricks Certified Data Engineer Professional (required)
  • Databricks Certified Associate Developer for Apache Spark 3.0 or higher (preferred)
  • Experience:
  • 10+ years of overall experience.
  • 5+ years of experience in Azure administration.
  • 3+ years of experience managing Databricks environments.
  • Soft Skills:
  • Excellent problem-solving and analytical skills.
  • Strong communication and collaboration abilities.
  • Proactive approach to learning and implementing emerging technologies.

  • Cloud Infrastructure Engineer

    6 days ago

    Karachi, Sindh, Pakistan beBee Careers Full time

    Job Description">We are seeking a highly skilled Cloud Infrastructure Engineer to design, deploy, and manage Azure cloud resources. The ideal candidate will have advanced knowledge of Azure infrastructure, networking, and services like Data Lake, Synapse, AKS, and Key Vault.">Responsibilities">Design and deploy scalable Azure cloud resources to meet business...

  • Cloud Infrastructure Engineer

    2 weeks ago

    Karachi, Sindh, Pakistan UNAVAILABLE Full time

    About UsWe are UNAVAILABLE, a company that provides innovative solutions to meet the needs of our clients.We are seeking a highly skilled Cloud Infrastructure Engineer to join our team and contribute to the design and implementation of our cloud infrastructure.Job DescriptionDesign and implement cloud infrastructure solutions using Linux.Troubleshoot complex...

  • Cloud Engineering Expert

    6 days ago

    Karachi, Sindh, Pakistan beBee Careers Full time

    About the RoleWe are seeking a highly experienced Cloud Engineer to join our team. As a Staff DevOps Engineer, you will be responsible for designing, implementing, and maintaining scalable cloud-based infrastructure.Key ResponsibilitiesEvaluate and ensure adherence to Service Level Agreements (SLAs) outlined in contractsDocument system configurations,...

  • Senior Data Infrastructure Engineer

    4 days ago

    Karachi, Sindh, Pakistan beBee Careers Full time

    Job DescriptionThe Senior Data Infrastructure Engineer will be responsible for designing, developing, and maintaining large-scale data systems and pipelines to meet the needs of our organization.Main ResponsibilitiesDevelop and maintain scalable data pipelines using ETL/ELT processesImplement data quality checks, validation processes, and monitoring...

  • Cloud Data Architect Position

    2 weeks ago

    Karachi, Sindh, Pakistan Minhas Technologies Full time

    We are Minhas Technologies, a leading IT solutions provider. As a mid-senior level expert in data engineering, you will play a critical role in designing and optimizing our cloud-based data solutions.**Job Summary:**We are seeking an experienced Python core engineer with expertise in Azure data engineering tools and services. The ideal candidate should have...

  • Cloud Infrastructure Manager

    4 days ago

    Karachi, Sindh, Pakistan beBee Careers Full time

    In this Cloud Infrastructure Manager role, you will be responsible for designing, implementing, and maintaining cloud-based infrastructure solutions using OLVM. Key responsibilities include designing and provisioning compute, networking, and storage resources, developing scripts and automation routines for installation and daily operations, and ensuring high...

  • Senior Data Infrastructure Engineer

    2 days ago

    Karachi, Sindh, Pakistan beBee Careers Full time

    About the RoleWe are looking for a highly skilled Senior Data Infrastructure Engineer to join our team. As a key member of our infrastructure team, you will be responsible for designing, implementing, and managing large-scale data infrastructure systems in the cloud.The ideal candidate will have extensive experience with database administration, data...

  • Azure Cloud Engineer

    2 weeks ago

    Karachi, Sindh, Pakistan 10Pearls, LLC Full time

    Welcome to 10Pearls, where technology meets innovation and partnership. We're a digital technology services partner dedicated to helping businesses harness the power of technology to stay ahead of the competition. From cloud architecture to software development, data insights to cybersecurity, our experts bring together a wealth of knowledge and experience...

  • Cloud Infrastructure Expert

    5 days ago

    Karachi, Sindh, Pakistan beBee Careers Full time

    We are seeking a skilled Oracle Infrastructure Technology Consultant with strong experience in implementing and managing Oracle Cloud Infrastructure (OCI) environments end-to-end. The ideal candidate will play a critical role in designing, deploying, and supporting scalable, secure, and high-performance OCI solutions to meet business and technical...

  • Cloud Infrastructure Specialist

    2 weeks ago

    Karachi, Sindh, Pakistan 10Pearls, LLC Full time

    10Pearls is a digital technology services partner that helps businesses leverage technology for competitive advantage. We assist our customers in digitalizing their existing operations, developing innovative products, and augmenting their teams with high-performance members. Our broad expertise encompasses product management, user experience/design, cloud...