QA Lead

6 hours ago

Quezon City, National Capital Region, Philippines Playbook Sports Full time ₱1,080,000 - ₱1,440,000 per year

Job Title: QA Lead (AI Agents)

Level: Lead

Location: Remote (Philippines)

Work Hours: Full-time, US EST (Night Shift)

Company Overview:Playbook is the best all-in-one software platform for sports, youth, and community businesses. We are at the forefront of integrating AI to build smarter, more efficient tools for our customers.

The Opportunity:We are seeking a pioneering QA Lead to establish and own the quality assurance strategy for our cutting-edge AI agents. This is not a traditional QA role. You will be responsible for ensuring our AI is reliable, accurate, and safe by designing and implementing sophisticated evaluation frameworks (evals). You will be the company's expert on AI quality, defining how we measure and benchmark the performance of our AI-powered features. This is a unique opportunity to shape a critical function at the intersection of QA and Artificial Intelligence.

Key Responsibilities:

  • AI Evaluation Framework: Design, develop, and maintain a robust framework for evaluating our AI agents. This includes setting up automated and manual evaluation processes.
  • Test Case & Dataset Management: Create and manage comprehensive datasets, test cases, and prompts to benchmark AI performance across key metrics like accuracy, relevance, tone, safety, and factuality.
  • AI QA Strategy: Establish and own the overall QA strategy for all AI-powered features, from initial development to post-release monitoring.
  • Analysis & Reporting: Analyze results from evaluations, identify failure modes, and provide clear, actionable feedback and reports to the engineering and product teams.
  • Collaboration: Work closely with developers and product managers to understand AI agent behavior, define quality criteria, and integrate testing early in the development lifecycle.
  • Tooling & Research: Stay up-to-date with the latest techniques, research, and tools in AI testing and evaluation, and implement best practices within the team.

Qualifications:

  • 4+ years of experience in software quality assurance, with at least 1-2 years of recent, direct focus on testing AI/ML models.
  • Crucial: Direct, hands-on experience designing and implementing evaluation frameworks (evals) for LLMs, chatbots, or other AI agents.
  • Proficiency in a scripting language (Python is strongly preferred) for creating test scripts and analyzing evaluation data.
  • Strong understanding of core machine learning concepts and the unique challenges of testing non-deterministic systems.
  • Experience with API testing tools (e.g., Postman) is highly valuable.
  • Exceptional analytical skills and a meticulous, data-driven approach to quality.
  • Excellent communication skills, with the ability to explain complex AI failures to both technical and non-technical stakeholders.

Job Type: Full-time

Pay: Php90, Php120,000.00 per month

Work Location: In person

  • QA Lead

    4 hours ago

    Quezon City, National Capital Region, Philippines Pointwest Full time ₱900,000 - ₱1,200,000 per year

    Pointwest is looking for a Manual QA Tester to join our Insurance Business Unit. This role is key to delivering high-quality software through detailed testing, clear defect reporting, and active collaboration with cross-functional teams. You will contribute to project success by ensuring product reliability while aligning with Pointwest's core values...

  • Lead QA

    1 week ago

    Makati City, National Capital Region, Philippines Onward Worldwide Management & Consultancy Services Inc. Full time ₱1,200,000 - ₱2,400,000 per year

    Job Description:Leading and managing the QA culture practices processes and documentations.Create, implement, and manage the functional and non-functional test plan.Communicating with other stakeholders such as Product Owner, Product Designers, Developers, and Infrastructure EngineersCollaborate to formulate and improve the QA Test planChecking the daily...

  • AML Systems Data QA Lead

    4 weeks ago

    Quezon City, National Capital Region, Philippines Manulife Financial Full time

    OverviewWe're looking for a AML Systems Data QA Lead to join our Integrated Compliance Services team at MBPS. In this role, you are responsible for overseeing Systems and Data Quality Assurance (QA) activities, spearheading the development, execution, and reporting of the annual risk-based Anti-Money Laundering (AML) Systems and Data QA testing plan. This...

  • AML Systems Data QA Lead

    4 weeks ago

    Quezon City, National Capital Region, Philippines Manulife Full time

    Are you looking for a supportive and collaborative workplace with great benefits and clear career development? You've come to the right place.Why choose Manulife?Competitive Salary packages and performance bonusesDay 1 HMO + FREE coverage for your dependents (inclusive of same-sex partners)Retirement savings benefitRewarding culture that values wellness and...

  • QA Test Lead

    2 weeks ago

    Makati City, National Capital Region, Philippines People Matter PH Full time ₱1,200,000 - ₱1,680,000 per year

    We are looking for a Quality Assurance (QA) Lead to help deliver an enterprise software platform used in Europe and Asia Pacific. You will work with distributed teams across regions, develop test plans, ensure QA best practices, and support successful platform deployment.ResponsibilitiesQA & DevelopmentDevelop and maintain test plans, test cases, and test...

  • QA Lead

    2 weeks ago

    Makati City, National Capital Region, Philippines Security Bank Corporation Full time ₱500,000 - ₱1,000,000 per year

    About the roleAs QA Lead of Underwriting Team, you are responsible to effectively manage the operations of the underwriting support unit. Oversees and takes responsibility for the overall performance of the quality assurance officers. Provides support in maintaining, publishing, and communicating underwriting policies and procedures to stakeholders. Devises...

  • Lead QA Engineer

    2 weeks ago

    Makati City, National Capital Region, Philippines TASQ Staffing Solutions Full time ₱900,000 - ₱1,200,000 per year

    TASQ is urgently looking for a lead QA engineer to work remotely. The pre-screening interview will be conducted over the phone, so please ensure your contact line is accessible. **Position:** Lead QA Engineer | REMOTE **Job Highlights:** - Focus on Azure technology provides excellent opportunities to enhance your Microsoft and Azure expertise within a...

  • QA Engineer Lead

    2 weeks ago

    Makati City, National Capital Region, Philippines TASQ Staffing Solutions Full time ₱900,000 - ₱1,200,000 per year

    TASQ is looking for a Philippines-based lead QA engineer to start working ASAP. **Job Title: Lead QA Engineer | REMOTE** **Job Highlights:** Our technology is centered around Azure, providing numerous exciting opportunities to enhance your skills in Microsoft and Azure in an environment dedicated to technical excellence and client satisfaction within a...

  • Lead IT QA Engineer

    2 weeks ago

    Makati City, National Capital Region, Philippines TASQ Staffing Solutions Full time ₱1,200,000 - ₱2,400,000 per year

    TASQ is looking to fill a position for a lead QA engineer ASAP. The pre-screening interview will take place over the phone. About the job Lead QA Engineer | REMOTE Job Highlights: Our technology focus is on Azure you'll have plenty of exciting opportunities to grow your skills in Microsoft and Azure in an environment committed to technical excellence and...

  • QA Lead

    2 weeks ago

    Makati City, National Capital Region, Philippines SLI Consulting Inc Full time ₱1,440,000 - ₱1,920,000 per year

    Knowledgeable in Software Development, Software TestingProficient in Testing Lifecycle ManagementPreferred with International Software Testing Qualifications BoardExperience in front-end web technologies such as HTML/5, HTL, CSS, JavaScript frameworks (eg: React/Angular)Experience in back-end technologies such as SOAP/REST, Microservices and SQLExperience in...