Gen AI Associate

5 days ago

Mandaue City, Central Visayas, Philippines Innodata Knowledge Services, Inc. Full time ₱900,000 - ₱1,200,000 per year

Job Title: Gen AI Associate - AI/LLM Evaluator (Orchestration)

Task:

· Evaluate AI/LLM conversations at step, turn, and task levels using structured metrics.

· Conduct Diagnostic evaluations to check tool usage, slot filling, and adherence to system/maker instructions.

· Perform Task Success evaluations to determine if user requests were completed accurately and efficiently.

· Execute Profiling evaluations to capture session- and turn-level insights such as query categorization, complexity, domain alignment, and customer satisfaction signals.

· Provide clear, concise rationales for all judgments, ensuring transparency and traceability.

· Identify errors, inconsistencies, and improvement opportunities in agent interactions.

· Ensure judgment consistency by rigorously applying evaluation guidelines.

· Contribute feedback on trends, guideline refinement, and evaluation process improvements.

· Collaborate with peers and managers to maintain high evaluation standards and meet project timelines.

Skills & Competencies:

· Strong analytical and critical thinking skills.

· Ability to edit and improve technical communication across a wide variety of fields.

· Creative thinker with problem-solving orientation.

· Strong attention to detail and consistency in applying evaluation criteria.

· Well-honed internet research skills to validate information and context.

· Ability to embrace collaboration with diverse, global teams.

· Adaptability to thrive in a dynamic environment with evolving guidelines and priorities.

· Strong written communication skills for documenting rationales.

· Comfortable working with structured evaluation tools and platforms.

· Ability to work independently while meeting deadlines.

Education

· Required: Bachelor's degree (any field with strong analytical or research components: Computer Science, Linguistics, Cognitive Science, Business, Social Sciences, etc.)

· Preferred: Advanced degree or coursework in AI/ML, data science, or domain-relevant fields.

Experience:

· Domain-Specific Roles: Candidates with professional expertise in the relevant domain (e.g., healthcare, finance, telecom, education) are strongly encouraged to apply. For these roles, prior annotation experience is not required but preferred.

· Preferred: Prior experience in AI/LLM evaluation, linguistic annotation, customer support QA, or usability testing.

Nice to have:

· Familiarity with LLM systems, AI agents, or conversational AI.

· Experience in BPO/Customer Support QA (useful for profiling and task success evaluations).

· Exposure to multi-domain environments (healthcare, finance, telecom, etc.).

· Ability to role-play end-user perspectives when evaluating AI interactions.

Employment information:

  • This role may be offered on a project-based, freelance, or full-time basis, depending on business needs.
  • Candidates should be available to work within a consistent time window between 8:00 AM to 8:00 PM

  • AI Journey Manager

    5 days ago

    Cebu City, Central Visayas, Philippines LeadAI Pty Ltd Full time ₱420,000 - ₱600,000 per year

    AI Journey Manager (UI/UX Specialist) – Design Next-Gen Human-AI Experiences at LeadAIIf You're Obsessed with Perfecting the Customer Journey Beyond Just Mapping it in Figma—This is Your Dream Role..Step into the next generation of AI applications where you don't just design screens—you shape how humans and AI work together. What to expect:→ Design...

  • Sales and Engagement Officer

    1 week ago

    Cebu City, Central Visayas, Philippines First Gen Corporation Full time ₱104,000 - ₱130,878 per year

    At First Gen, we take pride in our work of energizing the Philippines through clean power generation technologies. As we grow to become the best in the industry, we are continuously searching for talent who are driven, empowered, and passionate about making a positive difference in the world.We provide employees with ample opportunities and resources that...

  • Customer Service

    5 days ago

    Mandaue City, Central Visayas, Philippines RH-Studio Full time ₱240,000 per year

    About the RoleRH-Studio is looking for a Customer Service & Admin Associate to help with client communications, invoicing, and scheduling. This role mixes customer interaction with simple admin support—perfect for someone who's organized, service-oriented, and eager to learn.Shift ScheduleMid-shift: 3:00 PM – 12:00 AM (Monday–Friday)Onsite only (no...

  • Operation Manager

    1 day ago

    Cebu City, Central Visayas, Philippines Lifewood Data Technology Ltd. Full time ₱1 - ₱2 per year

    TheOperation Managerwill oversee the day-to-day operations of the company, ensuring that the processes supporting AI development, product delivery, and client engagement run efficiently and effectively. This role requires a balance of operational expertise, technical understanding of AI, and strategic thinking to support business growth and...

  • Business Planning and Analysis Specialist

    6 days ago

    Mandaue City, Central Visayas, Philippines Sandler Group Call Center Services Full time ₱960,000 - ₱1,320,000 per year

    Job Title: Business Planning & Analysis SpecialistSALARY OFFER : 80K-110K (open for negotiation based on experience , skill set and qualifications)Job Type: Onsite | Full time | Night Shift Schedule- 11pm to 8amDepartment: Financial OperationsReporting To: VP of Financial Operations and HRGeneral SummaryReporting to the VP of Financial Operations and HR. The...

  • Financial Planning

    5 days ago

    Mandaue City, Central Visayas, Philippines Sandler Group Call Center Services Full time ₱80,000 - ₱150,000 per year

    Job Title: Business Planning & Analysis SpecialistSALARY OFFER : 80K-110K (open for negotiation based on experience , skill set and qualifications)Job Type: Onsite | Full time | Night Shift Schedule- 11pm to 8amDepartment: Financial OperationsReporting To: VP of Financial Operations and HRGeneral SummaryReporting to the VP of Financial Operations and HR. The...

  • Executive Partner

    5 days ago

    Cebu City, Central Visayas, Philippines Athena Full time ₱30,000 - ₱460,000 per year

    About AthenaAthena is on a mission to improve how people work and live. Our membership helps growth-motivated leaders work smarter, live better, and achieve more with a personalized support team backed by the latest in AI tools. Join Athena to grow, take ownership of your career, and redefine the future of executive partnership and support.*The Role*Athena...

  • Executive Partner

    2 weeks ago

    Cebu City, Central Visayas, Philippines Athena Full time ₱720,000 - ₱1,080,000 per year

    About AthenaAthena is on a mission to improve how people work and live. Our membership helps growth-motivated leaders work smarter, live better, and achieve more with a personalized support team backed by the latest in AI tools. Join Athena to grow, take ownership of your career, and redefine the future of executive partnership and support.The RoleAthena...

  • Associate Manager

    3 days ago

    Angeles City, Central Luzon, Philippines Sutherland Full time ₱1,200,000 - ₱2,400,000 per year

    Company Description About SutherlandArtificial Intelligence. Automation. Cloud engineering. Advanced analytics. For business leaders, these are key factors of success. For us, they're our core expertise.We work with iconic brands worldwide. We bring them a unique value proposition through market-leading technology and business process excellence.We've...

  • Associate Manager

    3 days ago

    Angeles City, Central Luzon, Philippines Sutherland Full time ₱1,200,000 - ₱2,400,000 per year

    Company Description About SutherlandArtificial Intelligence. Automation. Cloud engineering. Advanced analytics. For business leaders, these are key factors of success. For us, they're our core expertise.We work with iconic brands worldwide. We bring them a unique value proposition through market-leading technology and business process excellence.We've...