AI Agent Quality Specialist

22 hours ago

Antipolo, Philippines Mindrift Full time

Overview

At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI. The Mindrift platform, launched and powered by Toloka, connects domain experts with cutting-edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real-world expertise from across the globe.

We are looking for curious and intellectually proactive contributors who double-check assumptions and play devil's advocate. This is a flexible, project-based opportunity well-suited for part-time, remote work.

What you’ll be doing
  • Reviewing evaluation tasks and scenarios for logic, completeness, and realism
  • Identifying inconsistencies, missing assumptions, or unclear decision points
  • Helping define clear expected behaviors (gold standards) for AI agents
  • Annotating cause–effect relationships, reasoning paths, and plausible alternatives
  • Thinking through complex systems and policies to ensure agents are tested properly
  • Working closely with QA, writers, or developers to suggest refinements or edge case coverage
Requirements
  • Excellent analytical thinking: can reason about complex systems, scenarios, and logical implications
  • Strong attention to detail: can spot contradictions, ambiguities, and vague requirements
  • Familiarity with structured data formats: can read, not necessarily write JSON/YAML
  • Ability to assess scenarios holistically: identify what’s missing, what's unrealistic, what might break
  • Good written and verbal communication in English to document findings
Preferred background
  • Experience with policy evaluation, logic puzzles, case studies, or structured scenario design
  • Background in consulting, academia, Olympiads (logic/math/informatics), or research
  • Exposure to LLMs, prompt engineering, or AI-generated content
  • Familiarity with QA or test-case thinking (edge cases, failure modes)
  • Understanding of scoring/evaluation in agent testing (precision, coverage)
Benefits
  • Get paid for your expertise, with rates that can go up to $38/hour depending on skills, experience, and project needs
  • Flexible, remote, freelance project that fits around your commitments
  • Gain valuable experience on an advanced AI project to enhance your portfolio
  • Influence how future AI models understand and communicate in your field
Job details
  • Seniority level: Internship
  • Employment type: Part-time
  • Job function: Other
  • Industries: IT Services and IT Consulting
#J-18808-Ljbffr

  • AI Data Quality Specialist

    2 weeks ago

    Antipolo City, Calabarzon, Philippines beBeeData Full time ₱600,000 - ₱1,000,000

    Job Title: AI Data Quality SpecialistAbout This Role:We are seeking a diligent and detail-focused professional to maintain annotation standards and drive continuous improvements in our AI data.Key Responsibilities:Quality Audits: Perform quality audits on annotated datasets to ensure they meet established guidelines and quality benchmarks.Statistical...

  • AI Systems Designer

    2 weeks ago

    Antipolo City, Calabarzon, Philippines beBeeSpecialist Full time ₱28,000 - ₱35,500

    Software Infrastructure SpecialistThe ideal candidate will create robust software infrastructure, developing and scaling systems that power online communities. They will work closely with product and growth teams to automate core workflows, deploy AI agents, and maintain clean, reliable data pipelines.

  • Human-AI Collaboration Specialist

    2 weeks ago

    Antipolo City, Calabarzon, Philippines beBeeExpertise Full time $60,000 - $150,000

    Unlock the Potential of Human-AI CollaborationThis opportunity is ideal for professionals who want to leverage their expertise in research, editing, analysis, or related fields to shape the future of AI. You'll collaborate with large language models (LLMs) that handle repetitive tasks, while you bring the nuance, judgment, and creativity required to deliver...

  • AI Development Director

    2 weeks ago

    Antipolo City, Calabarzon, Philippines beBeeArchitect Full time $150,000 - $250,000

    Role OverviewWe are seeking an experienced AI Architect to lead the design, development, and deployment of advanced AI platforms.This is a remote/hybrid role open to candidates based in Asia-Pacific, with a focus on designing and delivering next-generation AI systems that drive real impact.Key Responsibilities:Develop and implement scalable AI models using...

  • AI Data Specialist

    2 weeks ago

    Antipolo City, Calabarzon, Philippines beBeeArtificialIntelligence Full time ₱4,500 - ₱7,500

    AI Data Specialist OpportunityWe are seeking an experienced AI Data Specialist to contribute to the enhancement of artificial intelligence by meticulously annotating and labeling data.This role involves carefully reviewing provided datasets (text, images, or videos) and accurately labelling or classifying content based on project guidelines.Carefully review...

  • Senior AI System Architect

    2 weeks ago

    Antipolo City, Calabarzon, Philippines beBeeBackend Full time $120,000 - $140,000

    Backend DeveloperWe are seeking a skilled Backend Engineer to power our AI system. As a key member of our team, you will be responsible for architecting and implementing core services that interface with LangGraph-based workflows, manage data access from structured and unstructured sources, and support seamless orchestration between our AI layers and data...

  • AI Video Creator and Automation Specialist

    2 weeks ago

    Antipolo City, Calabarzon, Philippines beBeeVideoCreator Full time ₱600,000 - ₱800,000

    We are seeking a highly skilled AI Video Creator and Automation Engineer to help implement AI solutions, automate workflows, and manage complex projects.As a key member of our team, you will work closely with the CEO, assisting with tasks ranging from AI experimentation to data analysis and market research.You will lead AI automation projects using tools...

  • AI Data Quality

    7 days ago

    Antipolo, Philippines TaskUs Full time

    About TaskUs: TaskUs is a provider of outsourced digital services and next-generation customer experience to fast-growing technology companies, helping its clients represent, protect and grow their brands. Leveraging a cloud-based infrastructure, TaskUs serves clients in the fastest-growing sectors, including social media, e-commerce, gaming, streaming...

  • AI Text Expert

    2 weeks ago

    Antipolo City, Calabarzon, Philippines beBeeExpertise Full time $45,000 - $60,000

    About The RoleThis is a unique opportunity to leverage your text expertise and contribute to the development of AI models that understand and generate human-like text.As an expert in written communication, you will be crafting original responses based on project guidelines, ensuring quality and consistency through collaboration with Quality Assurance...

  • Chief AI Application Developer

    2 weeks ago

    Antipolo City, Calabarzon, Philippines beBeeSoftware Full time ₱6,000 - ₱8,000

    AI-Powered Software EngineerWe're seeking a seasoned AI-first software engineer to develop and deploy innovative solutions that propel our business forward.As a key member of our engineering team, you'll collaborate with our CTO to identify opportunities for growth and optimize workflows using cutting-edge AI models. Your expertise in building and shipping...