AI Agent Evaluation Analyst
9 hours ago
Get AI‑powered advice on this job and more exclusive features. This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency. What We Do The Mindrift platform, launched and powered by Toloka, connects domain experts with cutting‑edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real‑world expertise from across the globe. Who We're Looking For We’re looking for curious and intellectually proactive contributors—individuals who double‑check assumptions and play devil’s advocate. Comfortable with ambiguity and complexity? Prefer an async, remote, flexible opportunity? Interested in learning how modern AI systems are tested and evaluated? This role is ideal. Flexible Project‑Based Opportunity Analysts, researchers, or consultants with strong critical thinking skills Students (senior undergrads / grad students) looking for an intellectually interesting gig People open to a part‑time and non‑permanent opportunity About the Project We are hiring QAs for autonomous AI agents for a new project focused on validating and improving complex task structures, policy logic, and agent evaluation frameworks. Throughout the project, you will balance quality assurance, research, and logical problem‑solving. This opportunity is ideal for those who enjoy looking at systems holistically and thinking through scenarios, implications, and edge cases. What You'll Be Doing Review evaluation tasks and scenarios for logic, completeness, and realism Identify inconsistencies, missing assumptions, or unclear decision points Help define clear expected behaviors (gold standards) for AI agents Annotate cause‑effect relationships, reasoning paths, and plausible alternatives Think through complex systems and policies as a human would to ensure agents are tested properly Work closely with QA, writers, or developers to suggest refinements or edge‑case coverage How to Get Started Apply to this post, qualify, and get the chance to contribute to a project aligned with your skills on your own schedule. Shape the future of AI while building tools that benefit everyone. Requirements Excellent analytical thinking: reason about complex systems, scenarios, and logical implications Strong attention to detail: spot contradictions, ambiguities, and vague requirements Familiarity with structured data formats: read JSON/YAML Ability to assess scenarios holistically: identify missing or unrealistic elements that might break Good communication and clear writing in English to document findings We also value applicants who have: Experience with policy evaluation, logic puzzles, case studies, or structured scenario design Background in consulting, academia, olympiads (logic/math/informatics), or research Exposure to LLMs, prompt engineering, or AI‑generated content Familiarity with QA or test‑case thinking (edge cases, failure modes) Understanding of how scoring or evaluation works in agent testing (precision, coverage, etc.) Benefits Get paid for your expertise, with rates up to $47/hour depending on skills, experience, and project needs Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments Participate in an advanced AI project and gain valuable experience to enhance your portfolio Influence how future AI models understand and communicate in your field of expertise #J-18808-Ljbffr
-
Ilocano/Iloko Linguistic Projects
1 week ago
Biñan, Philippines Sigma AI Full timeIlocano/Iloko Linguistic Projects (Remote) Join the Ilocano/Iloko Linguistic Projects (Remote) role at Sigma AI. Sigma is a leading global technology company specializing in data collection and annotation for Artificial Intelligence. With over 30 years of experience, offices in Spain, the US, and the UK, and operations in more than 200 languages, we support...
-
PH - AI Automation Specialist
9 hours ago
Biñan, Philippines Moxie Full timeThis range is provided by Moxie. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more. Base pay range $11.00/hr - $15.00/hr At Moxie, we empower ambitious aesthetic entrepreneurs to build profitable, independent practices—without burnout, overwhelm, or guesswork. In just a few years, we’ve grown from an...
-
AI Data Quality
4 weeks ago
Biñan, Philippines TaskUs Full timeAI Data Quality & Engineering Lead at TaskUs About TaskUs: TaskUs is a provider of outsourced digital services and next-generation customer experience to fast-growing technology companies, helping its clients represent, protect and grow their brands. Leveraging a cloud-based infrastructure, TaskUs serves clients in fast-growing sectors, including social...
-
Biñan, Philippines Remotasks Full timeVideo Content Description Specialist Expertise Sought for AI Training Join to apply for the Video Content Description Specialist Expertise Sought for AI Training role at Remotasks Video Content Description Specialist Expertise Sought for AI Training Join to apply for the Video Content Description Specialist Expertise Sought for AI Training role at Remotasks...
-
Freelance Civil Engineering Expert
6 days ago
Biñan, Philippines Mindrift Full timeFreelance Civil Engineering Expert - AI Trainer 4 days ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features. This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English....
-
Business Development Representative, AI Safety
4 weeks ago
Biñan, Philippines TaskUs Full timeBusiness Development Representative, AI Safety Join or sign in to find your next job Join to apply for the Business Development Representative, AI Safety role at TaskUs Business Development Representative, AI Safety 1 day ago Be among the first 25 applicants Join to apply for the Business Development Representative, AI Safety role at TaskUs Get...
-
Financial Analyst
1 week ago
Biñan, Philippines Crown Worldwide Group Full timeOverview Join to apply for the Financial Analyst role at Crown Worldwide Group . Crown Worldwide Group is a privately owned, global logistics company founded in 1965 and headquartered in Hong Kong. We are an extraordinary and purposeful business committed to making it simpler to live, work and do business anywhere in the world, delivered through our broad...
-
Data Scientist I, Data Science
1 day ago
Biñan, Calabarzon, Philippines Western Digital Full time ₱360,000 - ₱720,000 per yearCompany Description At Western Digital, our vision is to power global innovation and push the boundaries of technology to make what you thought was once impossible, possible. At our core, Western Digital is a company of problem solvers. People achieve extraordinary things given the right technology. For decades, we've been doing just that—our technology...
-
Data Scientist I, Data Science
3 days ago
Biñan, Calabarzon, Philippines Western Digital Full time ₱900,000 - ₱1,200,000 per yearCompany Description At Western Digital, our vision is to power global innovation and push the boundaries of technology to make what you thought was once impossible, possible.At our core, Western Digital is a company of problem solvers. People achieve extraordinary things given the right technology. For decades, we've been doing just that—our technology...
-
Quality Assurance Laboratory Analyst
9 hours ago
Biñan, Philippines Emperador Distillers, Inc. Full timeQuality Assurance Laboratory Analyst Job Description a. Directly reports to QA Lab/Line Supervisor b. Performs routine laboratory sampling, analysis and evaluation of raw materials, in-process and finished goods based on the current and standard laboratory procedure. c. Analyzes and interprets all laboratory tests results. d. Supports by ensuring the...
