Software Development Engineer in Test

5 days ago

Lahore, Punjab, Pakistan Brio Full time 104,000 - 130,878 per year

Role Overview

This position is within a project with one of the foundational LLM companies. The goal is to evaluate and enhance Large Language Models by testing their performance on real-world software engineering challenges.

For this role, you will contribute by analyzing, configuring, and running open-source repositories. Your work will focus on assessing LLMs' ability to identify, understand, and resolve complex software issues, ensuring the development of high-quality evaluation datasets for cutting-edge AI research.

What does day-to-day look like:

● Analyze and triage GitHub issues across trending open-source libraries.

● Set up and configure code repositories, including Dockerization and environment setup.

● Evaluate unit test coverage and quality to ensure robustness of test suites.

● Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

● Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.

● Review outcomes and refine testing methodologies to improve consistency and reliability.

● (Optional) Lead a small team of junior engineers in collaborative evaluation projects.

Requirements:

● 4+ years of experience with software engineering, ideally with Python and modern development workflows.

● Familiarity with GitHub workflows, Docker, and environment setup.

● Strong debugging and analytical skills with the ability to run and modify complex codebases.

● Clear written communication skills in English.

● Prior experience in software testing, QA, or open-source contributions is a plus.

● Comfort with precision-focused evaluation work and collaborative research settings.

What success looks like:

● Delivering consistent, accurate evaluations of LLM performance across repositories.

● Effectively triaging and setting up diverse open-source projects.

● Contributing to a gold-standard dataset that supports research in AI code understanding and bug fixing.

● Driving insights that help advance the state of AI for software engineering.

Perks of Working With Brio:

● Work in a hybrid environment.

● Opportunity to work on cutting-edge AI projects with leading LLM companies.

Offer Details:

● Commitments Required: At least 8 hours per day and a minimum of 40 hours per week with overlap of 4 hours with PST.

● Engagement Type: Contractor assignment (no medical/paid leave).

● Duration of Contract: 5 months (adjustable based on engagement).

● Market Leading Salary

Interview Process
(approximately 30 mins)
:

● Interview to assess programming, debugging, and critical thinking skills.

● Questions about Software Engineering, Machine Learning, and AI.

Evaluation Process:

Candidates will be asked to complete a small technical task (offline after the interview), such as setting up a repository and running/debugging unit tests. The combined time to complete this task is expected to be around half a day.

  • Software Test Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan Clustox Full time $40,000 - $80,000 per year

    About Clustox:Clustox is a leading software development company based in Lahore, Pakistan. We specialize in providing cutting-edge solutions to our clients in various industries, including healthcare, fintech and retail. Our team is constantly growing and we are currently looking for a highly skilled Software Test Engineer to join our team and help us...

  • Software Testing Engineer

    7 days ago

    Lahore, Punjab, Pakistan beBeeTesting Full time $100,000 - $150,000

    Job OverviewWe are seeking a skilled Software Testing Engineer to lead quality assurance testing for large-scale web and mobile applications in an agile environment.ResponsibilitiesDevelop and execute comprehensive test plans to ensure high-quality software delivery.Implement automation frameworks using open-source tools and drive test automation...

  • Software Test Engineer

    7 days ago

    Lahore, Punjab, Pakistan PakWheels Full time

    As a Software Test Engineer at PakWheels, you will be responsible for ensuring the quality and reliability of products by collaborating closely with engineering, production, marketing, and sales teams throughout the product life cycle. You will translate product requirements into detailed test plans, conduct rigorous testing, and analyze market data to...

  • Software Test Engineer

    7 days ago

    Lahore, Punjab, Pakistan beBeeQuality Full time 200,000 - 400,000

    About This RoleThis internship is an excellent opportunity for individuals looking to launch a career in quality assurance. You will work under the guidance of experienced QA engineers, testing applications, identifying bugs, and ensuring product quality.Key ResponsibilitiesAssist in writing and executing test cases to ensure high-quality results.Identify,...

  • Software Test Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan OpenDrawing Full time $60,000 - $80,000 per year

    Location:Lahore, Pakistan (Full-Time, On-Site)Start Date:12 Oct 2025Team:EngineeringReports to:CTOAbout OpenDrawingOpenDrawing is building construction cost estimating software that will change the way our world is built. We're automating one of the most complex, manual parts of the construction process — and that means building software that's reliable,...

  • Lab Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan Test Confedential Full time $40,000 - $80,000 per year

    Role DescriptionThis is a full-time on-site role for a Lab Engineer (Mechanical) located in Lahore. The Lab Engineer will be responsible for designing, developing, and testing mechanical systems and prototypes. Day-to-day tasks will include working with a team to analyze mechanical and thermal devices, troubleshooting malfunctions, and implementing...

  • Internship - Software Test Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan Howmuch Full time 900,000 - 1,200,000 per year

    OverviewWe are seeking talented Test Automation Engineer Intern with strong dedication towards shipping high quality software, and has a strong drive to foster a high performance agile test culture using manual and automated testing.Major Duties & ResponsibilitiesDesign and develop high quality test plans and test cases with emphasis on automation using...

  • Senior Software Test Engineer

    1 week ago

    Lahore, Punjab, Pakistan NETSOL Technologies Inc. Full time $90,000 - $120,000 per year

    Education & QualificationsBachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, Math, Information Technology, or equivalent degree is preferred.Skill SetSenior Test Automation Engineer should have working experience in:Proficient in one of the programming language like JavaScript, VB Script, C#, Python.Hands on automation...

  • Test Engineer- Fresh

    2 weeks ago

    Lahore, Punjab, Pakistan Clustox Full time 800,000 - 1,200,000 per year

    About the job Test Engineer- Fresh Job Overview: Join Clustox as a Fresh Test Engineer If you enjoy identifying bugs and want to grow in automation testing, this role is ideal for you. Key Responsibilities: - Collaborate with the development team to understand the project requirements and design effective test plans - Develop and execute manual and...

  • Software Development Engineer in Test Apprentice

    7 days ago

    Lahore, Punjab, Pakistan beBeeQuality Full time 500,000 - 1,000,000

    The role of an SDET Trainee is to join a QA team in order to contribute to the software development lifecycle through quality assurance and testing activities.Key Responsibilities:Conduct thorough testing of software systemsDevelop automated test cases using various tools and technologiesIdentify system weaknesses and propose enhancements to improve...