AI Performance Engineer

2 days ago

Pasay, National Capital Region, Philippines beBeeAI Full time ₱1,800,000 - ₱2,500,000
Design and Implement AI Performance Solutions

You will be responsible for designing and implementing agent evaluation pipelines that benchmark AI capabilities across real-world enterprise use cases.

Benchmarking involves creating domain-specific benchmarks for product support, engineering operations, GTM insights, and other relevant verticals. You will also develop performance benchmarks that measure and optimize for latency, safety, cost-efficiency, and user-perceived quality.

Our team is looking for a skilled professional to create search- and retrieval-oriented benchmarks, including multilingual query handling, annotation-aware scoring, and context relevance.

As a key member of our team, you will partner with AI and infrastructure teams to instrument models and agents with detailed telemetry for outcome-based evaluation.

We are seeking an individual who can drive human-in-the-loop and programmatic testing methodologies for fuzzy metrics like helpfulness, intent alignment, and resolution effectiveness.

Contributions to DevRev's open evaluation tooling and benchmarking frameworks are highly valued, as we strive to shape the broader ecosystem's understanding of SaaS AI performance.

  • Design and implement agent evaluation pipelines.
  • Build domain-specific benchmarks.
  • Develop performance benchmarks.
  • Create search- and retrieval-oriented benchmarks.
  • Instrument models and agents with detailed telemetry.
  • Drive human-in-the-loop and programmatic testing methodologies.
  • Contribute to DevRev's open evaluation tooling.
Requirements:
  • 3–7 years of experience in systems, infra, or performance engineering roles.
  • Fluency in Python and comfort working across full-stack and backend services.
  • Experience building or using LLMs, vector-based search, or agentic frameworks in production environments.
  • Familiarity with LLM model serving infrastructure.
  • Experience working with model tuning workflows.
  • Deep appreciation for measuring what matters.
  • Familiarity with evaluation techniques in NLP, information retrieval, or human-centered AI.
  • Strong product and user intuition.

  • Engineering Specialist

    2 days ago

    Pasay, National Capital Region, Philippines beBeeArtificialIntelligence Full time $8 - $65

    Transform the Future of AIAbout the OpportunityWe are seeking engineering specialists who possess a profound understanding of mechanical, civil, electrical, aerospace, chemical, software, industrial, systems, structural, and materials science. Your expertise will be instrumental in propelling the next generation of AI forward.Your ResponsibilitiesEngage in...

  • Senior Software Engineer – Generative AI

    1 week ago

    Pasay, National Capital Region, Philippines Reliasourcing Full time $90,000 - $120,000 per year

    Job Title: Senior Software Engineer – Generative AI & Backend SystemsLocation: Onsite/HybridType: Full-timeAbout the Role: We're seeking Senior Software Engineers with a strong background in Backend Engineering and hands-on experience working with Generative AI (GenAI) technologies. In this role, you'll design, build, and scale AI-powered applications and...

  • AI Technology Specialist

    3 days ago

    Pasay, National Capital Region, Philippines beBeeAI Full time ₱2,000,000 - ₱3,000,000

    Transform Your Career as a Software Development ExpertThis position involves leading the design, development, and maintenance of software applications using AI technologies. The ideal candidate will be responsible for crafting solutions for complex projects with minimal supervision, mentoring junior engineers, and influencing best practices across teams.You...

  • Senior AI Strategist

    2 days ago

    Pasay, National Capital Region, Philippines beBeeArtificial Full time $80,000 - $120,000

    AI Engineering LeadWe are seeking a visionary leader to spearhead the development of AI systems across our organization.Design and implement a model-agnostic AI platform to optimize performance and reduce costs.Drive AI adoption in clinical operating systems, automate manual processes, and create intelligent assistants that enhance clinical team...

  • AI Solutions Developer

    3 days ago

    Pasay, National Capital Region, Philippines beBeeAutomation Full time $9,600 - $18,000

    Job Description:We are seeking a highly skilled AI Engineer to join our team. This is a full-time remote opportunity that requires expertise in designing and implementing AI and automation solutions.The ideal candidate will have advanced knowledge of Microsoft tools and ecosystem, as well as hands-on experience with AI transcription tools, file and data...

  • AI Model Evaluator Specialist

    3 days ago

    Pasay, National Capital Region, Philippines beBeeEvaluator Full time ₱350,000 - ₱500,000

    Job OverviewEvaluate and refine AI models by creating high-quality training data sets, annotating data to align with human values, and collaborating with cross-functional teams to ensure accuracy and fluency.Key ResponsibilitiesCreate comprehensive data sets for AI models to improve their performance and effectiveness.Annotate data to help AI models...

  • Senior AI Platform Developer

    2 days ago

    Pasay, National Capital Region, Philippines beBeeAI Full time $180,000 - $250,000

    About the RoleWe are seeking a seasoned Senior AI Platform Developer to power our platform of autonomous AI agents. This individual will architect and implement core services that interface with AI workflows, manage data access from structured and unstructured sources, and support seamless orchestration between AI layers and data infrastructure.] Key...

  • AI Model Trainer

    3 days ago

    Pasay, National Capital Region, Philippines beBeeArtificialintelligence Full time ₱550,000 - ₱850,000

    AI Training Data SpecialistDescription:Create comprehensive training datasets to enhance AI model accuracy, fluency, coherence, and voice.Annotate data to ensure AI models align with human values.Develop expertise in grammatical analysis and sentence structure refinement for both expert-generated and AI-generated text.Collaborate closely with...

  • AI Software Developer

    2 days ago

    Pasay, National Capital Region, Philippines beBeeArtificialIntelligence Full time $100,000 - $120,000

    Job OverviewThis opportunity is part of a strategic initiative to leverage expertise from multiple fields and drive the development of generative AI.The platform supports teams building AI-powered applications, enabling faster and more effective outcomes through tools that streamline labor-intensive processes and fine-tune large language models.As a skilled...

  • AI & Blockchain Development Specialist

    3 days ago

    Pasay, National Capital Region, Philippines beBeeExpertise Full time $120,000 - $150,000

    Blockchain & AI Expert WantedWe are seeking an experienced Blockchain & AI expert to join our dynamic team. The ideal candidate will have a strong background in blockchain technology and artificial intelligence, with a proven track record of creating engaging developer-focused content.Job Description:This is an exciting opportunity for a highly skilled and...