Distributed Inference Architect

7 days ago

Sukkur, Sindh, Pakistan beBeeInnovation Full time $200,000 - $350,000
Digital Innovation Leader

Embark on a challenging role where you will have the opportunity to design and implement cutting-edge distributed inference platforms for serving large language models. Your work will directly impact our clients, driving their digital transformation and growth.

Main Responsibilities:
  • Develop scalable and efficient distributed systems using Kubernetes, gRPC, Go, Python, and GPU programming using CUDA, ROCm.
  • Build native cross-platform inference support across NVIDIA and AMD GPUs for various model architectures, ensuring seamless performance.
  • Collaborate with our team to improve open-source inference engines, focusing on their performance on our cloud infrastructure.
  • Create tooling and observability to monitor system health and develop auto-tuning capabilities, optimizing runtime and infrastructure layers for best model performance.
  • Design benchmarking frameworks to test model-serving performance and guide system and infrastructure tuning efforts.
  • Mentor engineers on inference systems, GPU infrastructure, and distributed inference best practices, sharing your expertise and knowledge.
Required Qualifications:
  • Bachelor's or Master's degree in Computer Science, Electrical Engineering, or related field.
  • Proven experience building distributed systems, including experience with Kubernetes, gRPC, Go, Python, and GPU programming using CUDA, ROCm.
  • Familiarity with L3-L7 network protocols, block storage, object storage, and multi-tenant systems - identity management, tenant isolation.
  • Strong track record of defining and achieving performance KPIs (latency, throughput, cost).
Preferred Qualifications:
  • Experience with one or more inference engines: vLLM, SGLang, Modular.
  • Familiarity with one or more distributed inference serving frameworks - llm-d, NVIDIA dynamo, Ray Serve.
  • Knowledge of distributed inference optimization techniques - tensor/data parallelism, KV cache optimizations, smart routing.
  • Familiarity with common LLM architectures and inference optimization techniques (e.g., continuous batching, quantization).
  • Experience with GPU Interconnect technologies like NVlink, XGMI, RoCE.
  • Open-source contributions to inference libraries, frameworks, model kernels.
Why You'll Thrive at Our Organization:
  • We innovate to drive business success. As a member of our team, you'll contribute to cutting-edge technology that simplifies cloud and AI, empowering developers to create software that transforms industries.
  • We prioritize career development, offering resources to help you grow professionally, including reimbursement for relevant conferences, training, and education, as well as LinkedIn Learning courses.
  • We value your well-being, providing a competitive array of benefits to support your physical and mental health, including an Employee Assistance Program and flexible time off policy.
  • We recognize individual contributions, rewarding our employees based on market data, relevant years of experience, and skills.
  • We celebrate diversity and inclusion, recognizing that diverse perspectives and backgrounds foster stronger teams and products that serve our clients better.

  • Innovative Distributed System Developer

    7 days ago

    Sukkur, Sindh, Pakistan beBeeSoftware Full time $150,000 - $250,000

    Build Scalable Distributed SystemsAs a Software Engineer on our ML Platform team, you will lead efforts to build systems enabling AI R&D teams to deliver secure, highly scalable, distributed services.You will leverage technology such as Kubeflow, Kubernetes, Helm, Terraform, and Python to build inference systems, infrastructure, training pipelines, and...

  • Lead Research Innovation Architect

    6 days ago

    Sukkur, Sindh, Pakistan beBeeMachineLearning Full time $120,000 - $150,000

    **Research Innovation Engineer Position**">At Tether Operations Limited, we drive innovation in architecture development for cutting-edge models of various scales. Your work will enhance intelligence, improve efficiency, and introduce new capabilities to advance the field.About the RoleYou will have a deep expertise in video generation model architectures...

  • Highly Experienced Infrastructure Engineer Wanted

    7 days ago

    Sukkur, Sindh, Pakistan beBeeSoftwareDeveloper Full time 600,000 - 7,200,000

    Infrastructure Software DeveloperThis is a challenging role that requires an experienced software engineer to join our team. As a member of the distributed team, you will be responsible for designing object-oriented software at scale and leveraging your experience in traditional web technologies to launch products.With a minimum of 3 years of work...

  • AI Infrastructure Engineer

    6 days ago

    Sukkur, Sindh, Pakistan beBeeSoftware Full time $104,590 - $179,468

    Empowering the people who run physical operations, we at Motive provide tools to make their work safer, more productive, and profitable.We serve over 100,000 customers across various industries, including transportation, construction, energy, field service, manufacturing, agriculture, food and beverage, retail, and the public sector.About the Role:You will...

  • Full-Stack Software Solutions Developer

    7 days ago

    Sukkur, Sindh, Pakistan beBeeSoftware Full time 15,000,000 - 20,000,000

    Software Solutions ArchitectWe are seeking a skilled software architect to design, develop and maintain scalable software solutions for Supply Chain and warehouse management.Key Responsibilities:Develop full-stack software systems to support Supply Chain services and warehouse management.Design, architect and implement frontend and backend solutions for...

  • Senior Technical Architect

    7 days ago

    Sukkur, Sindh, Pakistan beBeeSoftware Full time $160,000 - $230,000

    Job OverviewWe are seeking a seasoned software engineer to join our development team. As a key member of this team, you will be responsible for designing components and services for new feature developments.You will work closely with cross-functional teams to design and document scalable solutions, implement secure and maintainable code that powers our...

  • Chief Technology Architect

    7 days ago

    Sukkur, Sindh, Pakistan beBeeDevelopment Full time 8,000,000 - 12,000,000

    Job DescriptionWe are seeking an experienced and skilled Senior Java Developer to fill a key role in our development team.As a critical member of our technical team, you will be responsible for designing, developing, and maintaining robust web-based applications using Java.Designing, developing, and maintaining scalable microservices using Spring Boot and...

  • Technology Innovation Leader

    7 days ago

    Sukkur, Sindh, Pakistan beBeeSoftware Full time 15,024,000 - 21,636,000

    Software ArchitectThe role involves empowering teams with technical tools to make their work safer, more efficient and profitable.This position empowers a cross-functional team with technological solutions that automate tasks, reduce manual workloads, and enhance productivity.Motive has a strong presence across various industries including transportation,...

  • Senior Blockchain Developer

    6 days ago

    Sukkur, Sindh, Pakistan beBeeBlockchain Full time $180,000 - $240,000

    Job TitleWe are seeking a highly skilled Senior Blockchain Developer with ~5 years of professional experience to design and build scalable, production-ready Web3 applications.The ideal candidate has strong expertise in smart contract development, distributed systems, and cloud-native architectures, combined with hands-on experience in modern web technologies...

  • Highly Experienced Linux System Architect

    7 days ago

    Sukkur, Sindh, Pakistan beBeeLinux Full time $120,000 - $150,000

    We're seeking a highly skilled and experienced Senior Linux System Engineer to join our team. This role requires expertise in managing, optimizing, and securing Linux-based hosting environments.Job OverviewThis position involves working at the forefront of our operations, ensuring that our hosting platforms are always online and secure. The successful...