Software Development Engineer in Test

7 days ago

Lahore, Punjab, Pakistan Brio Full time 104,000 - 130,878 per year

Role Overview

This position is within a project with one of the foundational LLM companies. The goal is to evaluate and enhance Large Language Models by testing their performance on real-world software engineering challenges.

For this role, you will contribute by analyzing, configuring, and running open-source repositories. Your work will focus on assessing LLMs' ability to identify, understand, and resolve complex software issues, ensuring the development of high-quality evaluation datasets for cutting-edge AI research.

What does day-to-day look like:

● Analyze and triage GitHub issues across trending open-source libraries.

● Set up and configure code repositories, including Dockerization and environment setup.

● Evaluate unit test coverage and quality to ensure robustness of test suites.

● Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

● Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs.

● Review outcomes and refine testing methodologies to improve consistency and reliability.

● (Optional) Lead a small team of junior engineers in collaborative evaluation projects.

Requirements:

● 4+ years of experience with software engineering, ideally with Python and modern development workflows.

● Familiarity with GitHub workflows, Docker, and environment setup.

● Strong debugging and analytical skills with the ability to run and modify complex codebases.

● Clear written communication skills in English.

● Prior experience in software testing, QA, or open-source contributions is a plus.

● Comfort with precision-focused evaluation work and collaborative research settings.

What success looks like:

● Delivering consistent, accurate evaluations of LLM performance across repositories.

● Effectively triaging and setting up diverse open-source projects.

● Contributing to a gold-standard dataset that supports research in AI code understanding and bug fixing.

● Driving insights that help advance the state of AI for software engineering.

Perks of Working With Brio:

● Work in a hybrid environment.

● Opportunity to work on cutting-edge AI projects with leading LLM companies.

Offer Details:

● Commitments Required: At least 8 hours per day and a minimum of 40 hours per week with overlap of 4 hours with PST.

● Engagement Type: Contractor assignment (no medical/paid leave).

● Duration of Contract: 5 months (adjustable based on engagement).

● Market Leading Salary

Interview Process
(approximately 30 mins)
:

● Interview to assess programming, debugging, and critical thinking skills.

● Questions about Software Engineering, Machine Learning, and AI.

Evaluation Process:

Candidates will be asked to complete a small technical task (offline after the interview), such as setting up a repository and running/debugging unit tests. The combined time to complete this task is expected to be around half a day.

  • Software Engineer In Test III

    5 days ago

    Lahore, Punjab, Pakistan TCP Software Full time 1,200,000 - 3,600,000 per year

    TCP is committed to cultivating a diverse and inclusive team. However, we are not able to sponsor visas for this role.About TCP (TimeClock Plus):  For more than 30 years, TCP has helped organizations engage their people by providing flexible, mobile timekeeping and workforce management solutions. Trusted by tens of thousands of customers and millions of...

  • Software Engineer In Test III

    5 days ago

    Lahore, Punjab, Pakistan TCP Software Full time 900,000 - 1,200,000 per year

    TCP is committed to cultivating a diverse and inclusive team. However, we are not able to sponsor visas for this role. About TCP (TimeClock Plus):  For more than 30 years, TCP has helped organizations engage their people by providing flexible, mobile timekeeping and workforce management solutions. Trusted by tens of thousands of customers and millions of...

  • Software Engineer in Test I

    5 days ago

    Lahore, Punjab, Pakistan TCP Software Full time 900,000 - 1,200,000 per year

    TCP is committed to cultivating a diverse and inclusive team. However, we are not able to sponsor visas for this role. About TCP (TimeClock Plus):  For more than 30 years, TCP has helped organizations engage their people by providing flexible, mobile timekeeping and workforce management solutions. Trusted by tens of thousands of customers and millions of...

  • Software Engineer in Test I

    5 days ago

    Lahore, Punjab, Pakistan TCP Software Full time 1,200,000 - 2,400,000 per year

    TCP is committed to cultivating a diverse and inclusive team. However, we are not able to sponsor visas for this role.About TCP (TimeClock Plus):  For more than 30 years, TCP has helped organizations engage their people by providing flexible, mobile timekeeping and workforce management solutions. Trusted by tens of thousands of customers and millions of...

  • Software Test Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan OpenDrawing Full time 900,000 - 1,200,000 per year

    Location:Lahore, Pakistan (Full-Time, On-Site)Start Date:12 Oct 2025Team:EngineeringReports to:CTOAbout OpenDrawingOpenDrawing is building construction cost estimating software that will change the way our world is built. We're automating one of the most complex, manual parts of the construction process — and that means building software that's reliable,...

  • Internship - Software Test Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan Howmuch Full time 900,000 - 1,200,000 per year

    OverviewWe are seeking talented Test Automation Engineer Intern with strong dedication towards shipping high quality software, and has a strong drive to foster a high performance agile test culture using manual and automated testing.Major Duties & ResponsibilitiesDesign and develop high quality test plans and test cases with emphasis on automation using...

  • Senior Software Test Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan NETSOL Technologies Inc. Full time 900,000 - 1,200,000 per year

    Education & QualificationsBachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, Math, Information Technology, or equivalent degree is preferred.Skill SetSenior Test Automation Engineer should have working experience in:Proficient in one of the programming language like JavaScript, VB Script, C#, Python.Hands on automation...

  • Software Engineer IV

    5 days ago

    Lahore, Punjab, Pakistan TCP Software Full time 1,200,000 - 2,400,000 per year

    TCP is committed to cultivating a diverse and inclusive team. However, we are not able to sponsor visas for this role. About TCP (TimeClock Plus):  For more than 30 years, TCP has helped organizations engage their people by providing flexible, mobile timekeeping and workforce management solutions. Trusted by tens of thousands of customers and millions of...

  • Software Engineer IV

    5 days ago

    Lahore, Punjab, Pakistan TCP Software Full time 1,200,000 - 2,400,000 per year

    TCP is committed to cultivating a diverse and inclusive team. However, we are not able to sponsor visas for this role.About TCP (TimeClock Plus):  For more than 30 years, TCP has helped organizations engage their people by providing flexible, mobile timekeeping and workforce management solutions. Trusted by tens of thousands of customers and millions of...

  • Software Engineer II

    5 days ago

    Lahore, Punjab, Pakistan TCP Software Full time 900,000 - 1,200,000 per year

    TCP is committed to cultivating a diverse and inclusive team. However, we are not able to sponsor visas for this role.About TCP (TimeClock Plus):  For more than 30 years, TCP has helped organizations engage their people by providing flexible, mobile timekeeping and workforce management solutions. Trusted by tens of thousands of customers and millions of...