AI Agent Quality Specialist

3 weeks ago

Antipolo, Philippines Mindrift Full time

Overview

At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI. The Mindrift platform, launched and powered by Toloka, connects domain experts with cutting-edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real-world expertise from across the globe.

We are looking for curious and intellectually proactive contributors who double-check assumptions and play devil's advocate. This is a flexible, project-based opportunity well-suited for part-time, remote work.

What you’ll be doing
  • Reviewing evaluation tasks and scenarios for logic, completeness, and realism
  • Identifying inconsistencies, missing assumptions, or unclear decision points
  • Helping define clear expected behaviors (gold standards) for AI agents
  • Annotating cause–effect relationships, reasoning paths, and plausible alternatives
  • Thinking through complex systems and policies to ensure agents are tested properly
  • Working closely with QA, writers, or developers to suggest refinements or edge case coverage
Requirements
  • Excellent analytical thinking: can reason about complex systems, scenarios, and logical implications
  • Strong attention to detail: can spot contradictions, ambiguities, and vague requirements
  • Familiarity with structured data formats: can read, not necessarily write JSON/YAML
  • Ability to assess scenarios holistically: identify what’s missing, what's unrealistic, what might break
  • Good written and verbal communication in English to document findings
Preferred background
  • Experience with policy evaluation, logic puzzles, case studies, or structured scenario design
  • Background in consulting, academia, Olympiads (logic/math/informatics), or research
  • Exposure to LLMs, prompt engineering, or AI-generated content
  • Familiarity with QA or test-case thinking (edge cases, failure modes)
  • Understanding of scoring/evaluation in agent testing (precision, coverage)
Benefits
  • Get paid for your expertise, with rates that can go up to $38/hour depending on skills, experience, and project needs
  • Flexible, remote, freelance project that fits around your commitments
  • Gain valuable experience on an advanced AI project to enhance your portfolio
  • Influence how future AI models understand and communicate in your field
Job details
  • Seniority level: Internship
  • Employment type: Part-time
  • Job function: Other
  • Industries: IT Services and IT Consulting
#J-18808-Ljbffr

  • Quality Assurance Specialist

    2 days ago

    Antipolo, Philippines Intelegencia Full time

    Urgent Hiring: Quality Assurance Specialist Job Description: We are seeking a highly detail-oriented and experienced Quality Assurance Specialist & Quality Control Specialist to join our team. These roles involve evaluating customer interaction quality (QA) and CRM data accuracy (QC) to ensure the delivery of high-quality service and reliable client...

  • AI Data Quality

    2 weeks ago

    Antipolo, Philippines TaskUs Full time

    Overview About TaskUs: TaskUs is a provider of outsourced digital services and next-generation customer experience to fast-growing technology companies, helping its clients represent, protect and grow their brands. Leveraging a cloud-based infrastructure, TaskUs serves clients in the fastest-growing sectors, including social media, e-commerce, gaming,...

  • AI Data Quality

    4 weeks ago

    Antipolo, Philippines TaskUs Full time

    About TaskUs: TaskUs is a provider of outsourced digital services and next-generation customer experience to fast-growing technology companies, helping its clients represent, protect and grow their brands. Leveraging a cloud-based infrastructure, TaskUs serves clients in the fastest-growing sectors, including social media, e-commerce, gaming, streaming...

  • AI & Automation Specialist

    3 weeks ago

    Antipolo, Philippines BruntWork Full time

    Direct message the job poster from BruntWork Schedule Mondays to Fridays, 9:00 am to 5:00 pm, with 1-hour unpaid leave Client time zone: UK Role Overview We’re looking for a tech-savvy AI & Automation Specialist to build a personalised AI assistant system for a busy company director. This role blends coding, automation, and workflow design to...

  • Quality Assurance Specialist

    5 days ago

    Antipolo City, Calabarzon, Philippines Intelegencia Full time ₱900,000 - ₱1,200,000 per year

    Job Description:We are seeking a highly detail-oriented and experiencedQuality Assurance Specialist&Quality Control Specialistto join our team. These roles involve evaluating customer interaction quality (QA) and CRM data accuracy (QC) to ensure the delivery of high-quality service and reliable client information. You will play a vital part in maintaining...

  • Quality Assurance Specialist

    3 days ago

    Antipolo City, Calabarzon, Philippines Intelegencia Full time ₱300,000 - ₱360,000 per year

    About the RoleWe are looking for highly motivated and detail-oriented Quality Assurance Specialists to join our growing team in OTC - Antipolo. This full-time, permanent position plays a key role in upholding service excellence by evaluating customer interactions and ensuring compliance with internal and industry quality standards.Key ResponsibilitiesMonitor...

  • Quality Analyst

    7 days ago

    Antipolo, Philippines Movate Technologies Services Limited Full time

    Audit voice and digital interactions based on established QA rubrics. Ensure timely completion of daily/weekly QA evaluations per agent. Provide constructive feedback to agents and Team Leads. Calibration & Coaching Support Participate in bi-weekly Willow syncs and QA calibration sessions. Align scoring standards with Team Leads and SMEs. Support coaching...

  • TaskUs Antipolo | Pioneer

    6 days ago

    Antipolo, Philippines TaskUs Full time

    Overview TaskUs is a provider of outsourced digital services and next-generation customer experience to fast-growing technology companies, helping its clients represent, protect and grow their brands. As a Teammate, you will receive inbound and/or place outbound calls. You will provide accurate and timely responses for inquiries coming through email and live...

  • AI Video Editor

    2 weeks ago

    Antipolo, Philippines Activate Talent Full time

    Job Title: Location: RemoteType: Full-TimeIndustry: Affiliate Marketing / DTC / Performance Creative About Us We’re a high-growth team driving top-performing affiliate marketing and DTC offers through conversion-driven content. Fueled by data, creativity, and AI innovation, we’re scaling fast — and we’re looking for talent that can keep pace. ...

  • Customer Service Training Specialist

    1 hour ago

    Antipolo, Philippines TalentPop App Full time

    3 days ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features. Empower Growth as Our Next Customer Service Training Specialist Are you passionate about helping others succeed? At TalentPop , we are looking for a Customer Service Training Specialist who will play a vital role in equipping our agents and candidates...