AI Performance Evaluator

18 hours ago

San Fernando, Central Luzon, Philippines beBeePerformance Full time ₱1,200,000 - ₱1,600,000

AI Performance Evaluator

  • This role involves designing and implementing AI performance evaluation pipelines to measure the capabilities of AI systems in real-world enterprise use cases.

  • The successful candidate will have experience in systems, infrastructure, or performance engineering with strong ownership of metrics and benchmarking.

Responsibilities:

  • Design agent evaluation pipelines to benchmark AI capabilities

  • Build domain-specific benchmarks for product support, engineering operations, and other verticals

  • Develop performance benchmarks for latency, safety, cost-efficiency, and user-perceived quality

  • Create search- and retrieval-oriented benchmarks

  • Partner with AI and infrastructure teams to instrument models and agents with detailed telemetry

  • Drive human-in-the-loop and programmatic testing methodologies

Requirements:

  • 3–7 years of experience in systems, infrastructure, or performance engineering roles

  • Fluency in Python and full-stack development

  • Experience building or using large language models and vector-based search

  • Familiarity with model serving infrastructure and tuning workflows

  • Deep appreciation for measuring what matters in AI output quality

Impact and Culture:

This role offers the opportunity to design the KPIs of the AI-native enterprise and shape how intelligence is measured in SaaS 2.0.

Our culture values hunger, humility, honesty, and heart, and our mission is to empower engineers to embrace their customers through design, data engineering, and machine intelligence.

  • AI-Powered Data Preparation Specialist

    1 day ago

    San Fernando, Central Luzon, Philippines beBeeDataAnnotation Full time ₱25,000 - ₱35,000

    Job OverviewThe role of a Data Annotation Specialist involves working on AI-powered projects that require high-quality data preparation, annotation, and evaluation.Key responsibilities include performing data annotation tasks across text and multimodal datasets to ensure accuracy and consistency.Conduct prompt–response evaluations to assess AI outputs and...

  • AI Code Review Specialist

    1 day ago

    San Fernando, Central Luzon, Philippines beBeeSoftware Full time $80,000 - $120,000

    AI Code Reviewer PositionThis role entails the meticulous examination of AI-generated code to ensure it meets quality and accuracy standards.The ideal candidate will possess fluency in English, proficiency with the Ruby programming language, and experience condensing complex coding issues into clear, concise evaluations.A collaborative team environment...

  • Freelance Manuscript Evaluator

    1 day ago

    San Fernando, Central Luzon, Philippines beBeeRheumatologist Full time ₱900,000 - ₱1,200,000

    Job OpportunityWe are seeking a skilled Freelance Peer Reviewer to conduct high-quality manuscript evaluations.Required QualificationsA PhD or equivalent advanced degree in Rheumatology or a related field is essential for this role.Publication records in reputable academic journals within the last two years are mandatory.Expertise in research methodologies,...

  • AI Voice

    15 hours ago

    San Fernando, Central Luzon, Philippines Kwanii Full time $70,000 - $120,000 per year

    We are on the lookout for a highly skilled AI Voice and Workflow Specialist to join our innovative client in the telecommunications industry. As an AI Voice and Workflow Specialist, you will design, build, and manage intelligent voice agents while developing automation workflows that enhance business operations. Your expertise in real-time conversational AI...

  • AI Content Review Specialist

    1 day ago

    San Fernando, Central Luzon, Philippines beBeeDeveloper Full time $60,000 - $90,000

    Job TitleSoftware Developer for AI Content Review and DevelopmentJob DescriptionAs a key member of our team, you will play a vital role in reviewing and evaluating content for generative AI models to ensure accuracy and relevance across various topics. This opportunity is part of a broader initiative that brings together subject matter experts from different...

  • AI Product Manager

    1 week ago

    San Fernando, Central Luzon, Philippines MCI Full time

    POSITION OVERVIEWMCI is a leading Business Process Outsourcing (BPO) company specializing in tailored solutions to meet diverse client needs. Committed to excellence and innovation, we are a trusted industry partner.We are seeking a strategic and technically fluent AI Product Manager to lead the planning, development, and launch of AI-powered solutions. This...

  • AI Model Improvement Specialist

    2 days ago

    San Fernando, Central Luzon, Philippines beBeeEvaluator Full time ₱3,500,000 - ₱4,500,000

    About the RoleWe're seeking a skilled expert to join our team as an AI/LLM Evaluator. In this role, you will play a crucial part in enhancing the accuracy, fluency, coherence, and voice of AI models.Key Responsibilities:Create high-quality training datasets for AI models to improve their performance.Annotate data to align AI models with human values and...

  • AI Engineering Specialist

    2 days ago

    San Fernando, Central Luzon, Philippines beBeeEngineering Full time $8 - $65

    Artificial intelligence (AI) is shaping the future of engineering. We seek experts in various fields to contribute to our mission.Role OverviewWe're looking for specialists with a strong background in mechanical, civil, electrical, aerospace, chemical, software, industrial, systems, structural, and materials science disciplines.Key ResponsibilitiesEngage in...

  • Automotive AI Development Specialist

    2 days ago

    San Fernando, Central Luzon, Philippines beBeeEngineering Full time $15,000 - $20,000

    Job OverviewWe're seeking an Automotive AI Development Specialist to join our team. This exciting opportunity is for candidates with a background in Auto Engineering.Mindrift uses collective intelligence to shape the future of AI, unlocking its potential through real-world expertise.The RoleAs an AI Tutor in Auto Engineering, you will collaborate on projects...

  • AI Project Specialist

    2 days ago

    San Fernando, Central Luzon, Philippines beBeeArtificialIntelligence Full time ₱1,200,000 - ₱2,400,000

    Job OpportunityWe connect ambitious individuals with freelance online tasks that enhance artificial intelligence capabilities.Toloka unlocks the full potential of AI by involving real people in the development process.About The RoleAnnotation is crucial for AI to make sense of the world. As an annotator, you may be invited to take part in online projects...