AI Performance Engineer

13 hours ago

San Juan, National Capital Region, Philippines beBeeExpert Full time ₱60,000 - ₱90,000

Job Overview

This role involves designing and implementing evaluation pipelines to benchmark AI capabilities across real-world enterprise use cases.

The ideal candidate will have 3-7 years of experience in systems, infrastructure or performance engineering roles with strong ownership of metrics and benchmarking. They should be fluent in Python and comfortable working across full-stack and backend services.

Responsibilities
  • Design and implement agent evaluation pipelines that benchmark AI capabilities across real-world enterprise use cases.
  • Build domain-specific benchmarks for product support, engineering operations, GTM insights, and other verticals relevant to modern SaaS.
  • Develop performance benchmarks that measure and optimize for latency, safety, cost-efficiency, and user-perceived quality.
  • Create search- and retrieval-oriented benchmarks, including multilingual query handling, annotation-aware scoring, and context relevance.
  • Partner with AI and infra teams to instrument models and agents with detailed telemetry for outcome-based evaluation.
  • Drive human-in-the-loop and programmatic testing methodologies for fuzzy metrics like helpfulness, intent alignment, and resolution effectiveness.
  • Contribute to DevRev's open evaluation tooling and benchmarking frameworks, shaping how the broader ecosystem thinks about SaaS AI performance.
What We Offer
  • A chance to design the KPIs of the AI-native enterprise - to bring rigor to systems that reason, and structure to software that thinks.
  • The opportunity to shape how intelligence is measured in SaaS 2.0.
  • A collaborative culture committed to those who are hungry, humble, honest, and who act with heart.

  • AI Systems Engineer

    2 days ago

    San Juan, National Capital Region, Philippines beBeeBackend Full time ₱28,000 - ₱35,500

    Key Job DetailsAwarding-winning company seeks a seasoned backend engineer with exceptional technical skills to spearhead the development and maintenance of data infrastructure, AI operations, and backend systems. This pivotal role involves automating core workflows, deploying AI agents, and maintaining reliable data pipelines in collaboration with product...

  • AI Quality Engineering Specialist

    2 days ago

    San Juan, National Capital Region, Philippines beBeeData Full time ₱6,000,000 - ₱8,000,000

    Job Description:OverviewWe are seeking a proactive leader to architect and uphold high-quality annotation workflows so that AI models are built and evaluated on reliable data without compromising on speed or efficiency.The Role:Lead and mentor a team of Data Quality Analysts, setting clear quality goals, delivering feedback, and fostering a culture of...

  • AI Development Specialist

    2 days ago

    San Juan, National Capital Region, Philippines beBeeOpportunity Full time ₱1,200,000 - ₱1,800,000

    Job Opportunity: AI EngineerWe are seeking a talented AI engineer to join our team in building the next generation of intelligent credit risk and collections systems. This is a remote-first role, with the opportunity to collaborate in hybrid mode at our Athens offices alongside cross-functional teams shaping AI-powered features for real-world impact.As an AI...

  • Lead AI Architect

    1 day ago

    San Juan, National Capital Region, Philippines beBeeMachineLearning Full time $80,000 - $120,000

    Job Title: Lead AI ArchitectWe are seeking a visionary AI Engineering Lead to spearhead the development of our cutting-edge AI systems. As a key member of our team, you will play a pivotal role in architecting our AI-first future.Main Responsibilities:Design and build innovative AI systems that drive business growth.Create a model-agnostic platform allowing...

  • AI Conversational Dialogue Specialist

    1 day ago

    San Juan, National Capital Region, Philippines beBeePrompt Full time ₱40,000 - ₱60,000

    Conversational AI Prompt EngineerWe are seeking an experienced Conversational AI Prompt Engineer to join our team. The ideal candidate will have a strong background in linguistics, communications, and AI/ML or computer science. They will design, test, and refine prompts for conversational AI systems, ensuring natural, effective, and compliant interactions...

  • Unlock Your Potential as an AI Capacity Expert

    2 days ago

    San Juan, National Capital Region, Philippines beBeeEngineer Full time $90,000 - $120,000

    Job Title: AI Capacity EngineerWe are seeking a highly skilled AI Capacity Engineer to join our team. As an AI Capacity Engineer, you will be responsible for ensuring the smooth operation of our AI systems.Key Responsibilities:Create, update, and delete model pools.Provision and decommission Dev, Test, and Production environments.Implement and maintain...

  • AI Solutions Developer

    1 day ago

    San Juan, National Capital Region, Philippines beBeeData Full time $9,600 - $18,000

    We are seeking an experienced AI Engineer to join our organization. The ideal candidate will have a strong background in artificial intelligence and machine learning, with the ability to design and implement AI-powered solutions.About the PositionThis role is responsible for supporting the development of AI-driven processes that replace manual back-and-forth...

  • Senior Crypto and AI Technologist

    2 days ago

    San Juan, National Capital Region, Philippines beBeeDeveloper Full time $100,000 - $120,000

    Lead Blockchain and AI AdvocateWe are seeking a skilled Blockchain and AI Advocate to join our team. In this role, you will be the primary point of contact between our company and the developer community, fostering strong relationships and driving adoption of our platform.Key Responsibilities:Serve as the primary liaison between our company and the developer...

  • AI Model Assurance Specialist

    13 hours ago

    San Juan, National Capital Region, Philippines beBeeDataQuality Full time $50,000 - $80,000

    About the RoleEnsuring the accuracy and reliability of AI models is a top priority for our organization. The role of an AI Data Quality Analyst plays a vital part in this process by guaranteeing that training and evaluation datasets meet the highest standards.Key Responsibilities:Perform quality audits on annotated datasets to ensure they adhere to...

  • AI Safety Advocate

    1 day ago

    San Juan, National Capital Region, Philippines beBeeJob Full time $15,000 - $25,000

    Unlock Opportunities in AI SafetyThis role is your chance to pioneer the dialogue between businesses and innovators in AI.We're looking for a Business Development Representative who can consistently hit targets, balance inbound and outbound efforts, and inform our product roadmap with prospect insights.Key Responsibilities:Qualify Inbound Leads: Leverage...