Gen AI Associate

Innodata Knowledge Services, Inc., a data-engineering company delivering AI solutions to global clients, is currently hiring Gen AI Associate - AI/LLM Evaluator to join our dynamic team. We retrieved your application from our file, and we would like to invite you to explore this awesome opportunity.

If you are interested, please see below the details of our upcoming project.

Employment details:

• This role may be offered on a project-based, freelance, or full-time basis, depending on business needs.
• Work Schedule: Day Shift (9.19 hours/day between 8:00 AM and 8:00 PM, MONDAY-FRIDAY)
• Requirement: Stable internet connection
• CAN START IMMEDIATELY

Task:

· Evaluate AI/LLM conversations at step, turn, and task levels using structured metrics.

· Conduct Diagnostic evaluations to check tool usage, slot filling, and adherence to system/maker instructions.

· Perform Task Success evaluations to determine if user requests were completed accurately and efficiently.

· Execute Profiling evaluations to capture session- and turn-level insights such as query categorization, complexity, domain alignment, and customer satisfaction signals.

· Provide clear, concise rationales for all judgments, ensuring transparency and traceability.

· Identify errors, inconsistencies, and improvement opportunities in agent interactions.

· Ensure judgment consistency by rigorously applying evaluation guidelines.

· Contribute feedback on trends, guideline refinement, and evaluation process improvements.

· Collaborate with peers and managers to maintain high evaluation standards and meet project timelines.

Skills & Competencies:

· Strong analytical and critical thinking skills.

· Ability to edit and improve technical communication across a wide variety of fields.

· Creative thinker with problem-solving orientation.

· Strong attention to detail and consistency in applying evaluation criteria.

· Well-honed internet research skills to validate information and context.

· Ability to embrace collaboration with diverse, global teams.

· Adaptability to thrive in a dynamic environment with evolving guidelines and priorities.

· Strong written communication skills for documenting rationales.

· Comfortable working with structured evaluation tools and platforms.

· Ability to work independently while meeting deadlines.

Job Qualifications:

· Required: Bachelor's degree (any field with strong analytical or research components: Computer Science, Linguistics, Cognitive Science, Business, Social Sciences, etc.)

· Preferred: Advanced degree or coursework in AI/ML, data science, or domain-relevant fields.

· Domain-Specific Roles: Candidates with professional expertise in the relevant domain (e.g., healthcare, finance, telecom, education) are strongly encouraged to apply. For these roles, prior annotation experience is not required but preferred.

· Preferred: Prior experience in AI/LLM evaluation, linguistic annotation, customer support QA, or usability testing.

Nice to have:

· Familiarity with LLM systems, AI agents, or conversational AI.

· Experience in BPO/Customer Support QA (useful for profiling and task success evaluations).

· Exposure to multi-domain environments (healthcare, finance, telecom, etc.).

· Ability to role-play end-user perspectives when evaluating AI interactions.

DEIB STATEMENT

At Innodata, Diversity, Equity, Inclusion, and Belonging (DEIB) are at the heart of who we are. With over 35 years of expertise, Innodata fosters an inclusive workplace that celebrates the rich diversity of backgrounds, experiences, and perspectives our employees bring. This diversity fuels our innovation in AI and data engineering, supporting our mission as a global leader in the industry.

We encourage individuals from all walks of life to apply, particularly those who may feel they don't meet every qualification or hesitate due to their identity. Our inclusive programs, such as comprehensive learning initiatives, Employee Resource Groups (ERGs), and flexible work arrangements, create a sense of belonging for every team member, whether they work remotely or on-site.

By embedding these values in everything we do, Innodata is committed to delivering innovation while building a workplace where everyone feels empowered to succeed and valued.

Job Type: Full-time

Pay: Php30, Php40,000.00 per month

Benefits:

• Flexible schedule
• Flextime
• Health insurance
• Life insurance
• Opportunities for promotion
• Paid training
• Work from home

Work Location: Remote