AI Rater Program Specialist

2 days ago

Las Piñas, National Capital Region, Philippines beBeeLinguistic Full time ₱960,000 - ₱1,280,000
Job Overview

We are seeking a skilled Linguistic Quality Specialist to play a pivotal role in our AI Rater Program. This position will ensure that rater guidelines are accurate, usable, and aligned with client objectives.


This role blends linguistic expertise, quality management, and client collaboration. You will partner with program managers, data scientists, and raters to translate client objectives into clear evaluation standards and ensure consistent, high-quality outputs.



Key Responsibilities
  • Translate client objectives into clear, actionable rating guidelines.
  • Ensure guidelines reflect what "good quality" means, balancing linguistic rigor and client business needs.
  • Collaborate with clients during pilot phases to refine guidelines and make adjustments based on test outcomes.
  • Create and manage initial test sets to validate guidelines.
  • Review rater performance during pilots, identify error patterns, and adjust standards accordingly.
  • Partner with data scientists to ensure pilots are measurable and data-backed.
  • Act as the quality authority for rater performance, ensuring alignment with guidelines.


Requirements
  • Expertise in semantics, pragmatics, discourse, and applied linguistic analysis.
  • Demonstrated ability to develop and refine annotation/rating guidelines.
  • Skilled in Google Sheets (advanced formulas, data validation, quality tracking).
  • Experience delivering rater training, coaching, and feedback.


Preferred Qualifications
  • Experience with rater evaluation metrics (e.g., Krippendorff's alpha, agreement rates).
  • Familiarity with annotation tools and quality monitoring systems.
  • Oral and written comprehension
  • Appropriate use of language


The ideal candidate will have 3-5 years experience in linguistics, applied linguistics, or annotation/evaluation programs and a Bachelor's or Master's degree in Linguistics, Applied Linguistics, or related field, or equivalent practical experience.

This is a remote position with competitive salary and benefits package, growth potential as the program scales, and direct client exposure while maintaining a hands-on, specialist role.

  • Language Logic Specialist

    2 days ago

    Las Piñas, National Capital Region, Philippines beBeeRater Full time ₱800,000 - ₱1,250,000

    About This Job OpportunityWe're seeking a highly skilled rater to contribute to the development and testing of AI rating guidelines during early-stage pilot projects. As an Experimental Rater, you'll play a critical role in shaping the systems that train large language models by identifying gray areas in guidelines, flagging edge cases, and offering...

  • Data Insights Analyst

    2 days ago

    Las Piñas, National Capital Region, Philippines beBeeDataScientist Full time ₱3,500,000 - ₱5,000,000

    Job Description:As a key player in our organization, you will be responsible for uncovering valuable insights from data and driving business growth through data-driven decision making.Key responsibilities include developing and implementing program and rater performance metrics, designing and analyzing experiments to uncover drivers of performance, and...

  • AI-Assisted Medication Specialist

    2 days ago

    Las Piñas, National Capital Region, Philippines beBeeMedication Full time ₱2,000,000 - ₱2,500,000

    Job Title: AI-Assisted Medication SpecialistAbout the Role:The AI-Assisted Medication Specialist will utilize cutting-edge technology to ensure the accuracy of medications packed into dose-ready sachets, cross-verifying AI-identified medications against prescriptions and flagging missing or duplicate tablets.Key Responsibilities:AI-Driven Medication...

  • AI Content Specialist

    2 days ago

    Las Piñas, National Capital Region, Philippines beBeeContent Full time $60,000 - $1,000,000

    Job DescriptionThe role of an AI Content Specialist involves creating, refining, and optimizing content generated with the assistance of AI tools. This position requires a unique blend of creativity and technical proficiency to produce high-quality, engaging, and brand-consistent content for diverse platforms.This specialist will leverage AI-driven tools to...

  • Content Evaluation Specialist

    2 days ago

    Las Piñas, National Capital Region, Philippines beBeeEvaluation Full time ₱375,000 - ₱750,000

    Evaluate content in local languages using proprietary rating systems and guidelines.Job RoleAs a Content Evaluation Specialist, you will assess online content based on provided guidelines and instructions. This role involves evaluating information, logging into online rating systems, and utilizing your device to complete tasks (5-20...

  • Program Management Specialist

    2 days ago

    Las Piñas, National Capital Region, Philippines beBeeProject Full time ₱53,200

    Program Management SpecialistThe role of a Program Management Specialist is to oversee and coordinate various projects across the organization. This position requires strong organizational skills, excellent communication skills, and the ability to work independently.Coordinate projects from initiation to completionDevelop project schedules and manage...

  • AI Video Automation Specialist

    2 days ago

    Las Piñas, National Capital Region, Philippines beBeeAutomation Full time ₱900,000 - ₱1,200,000

    Job Title: AI Video Automation SpecialistWe are looking for a skilled professional to lead our AI video automation efforts. As an AI Video Automation Specialist, you will work with our team to implement AI solutions, automate workflows, and manage complex projects.The role requires strong analytical and problem-solving skills, as well as experience with AI...

  • AI/LLM Content Specialist

    2 days ago

    Las Piñas, National Capital Region, Philippines beBeeEvaluator Full time ₱1,044,000 - ₱1,246,000

    Job OpportunityWe are seeking a skilled professional to join our team as an AI/LLM Evaluator. This role involves working with generative AI models, analyzing and improving their performance, and collaborating with cross-functional teams.Key Responsibilities:Creating high-quality training data sets for AI modelsAnnotating data to ensure AI models align with...

  • AI Training Data Specialist

    2 days ago

    Las Piñas, National Capital Region, Philippines beBeeExpertise Full time ₱800,000 - ₱1,200,000

    Job OverviewWe're seeking a skilled AI Training Data Specialist to join our team. This role involves creating high-quality training data sets for AI models, ensuring they align with human values and improve accuracy, fluency, coherence, and voice.About the RoleCreate customized training data sets for AI models to enhance their performanceAnnotate data to...

  • AI Data Quality Specialist

    1 day ago

    Las Piñas, National Capital Region, Philippines beBeeDataQuality Full time ₱4,000 - ₱8,000

    Job Title: AI Data Quality SpecialistAchieve high-quality data standards with precision and attention to detail. As an AI Data Quality Specialist, you will be responsible for maintaining annotation standards and driving continuous improvements to ensure the accuracy and reliability of AI datasets.Responsibilities:Quality Audits: Conduct quality audits on...