AI Data Quality Analyst

Overview TaskUs Quezon City, National Capital Region, Philippines AI Data Quality Analyst AI Data Quality Analyst role at TaskUs applies to data quality and annotation standards for AI/ML data. This position focuses on ensuring the fidelity of training and evaluation datasets, maintaining data integrity, and driving continuous improvements in annotation quality. Responsibilities Data Analysis: Perform quality audits on annotated datasets to ensure they meet established guidelines and quality benchmarks. Statistical Reporting: Use quality metrics such as F1 score and inter-annotator agreement to evaluate data quality. Root Cause Analysis: Analyze annotation errors, trends, and project documentation to identify root causes and propose remediation strategies. Edge-Case Management: Resolve and analyze edge-case annotations to ensure quality and identify improvement areas. Tooling: Proficiently use annotation and quality control tools to perform reviews and track quality metrics. Guidelines: Master project-specific guidelines and provide feedback for clarifications or improvements. Continuous Improvement: Identify automation opportunities to enhance analytics, provide deeper insights, and improve efficiency. Documentation: Maintain up-to-date documentation on quality standards, annotation guidelines, and QC procedures. Feedback: Regularly provide feedback that identifies areas for improvement across the annotation pipeline. Collaboration & Communication: Work with project stakeholders and clients to understand requirements and improve annotation pipelines. Training: Assist with training annotators, providing guidance and feedback to maintain data quality. Reporting: Provide regular updates on data quality metrics, findings, and actionable insights for continuous process improvements. What You'll Bring 1+ years of experience as a data analyst with exposure to data quality and/or data annotation, ideally in an AI/ML context. Familiarity with basic AI/ML concepts, pipelines, and data. Core Skills Strong analytical and problem-solving skills with exceptional attention to detail. Excellent written and verbal communication, ability to clearly articulate quality issues and collaborate with diverse teams. Ability to work independently and manage time to meet deadlines. Critical thinking, problem-solving, and a drive for continuous optimization. Fast learner who can work independently in a fast-paced environment. Detail-oriented with ability to balance with strategic priorities. Positive can-do attitude and adaptability to new environments. Not afraid to speak up. Nice To Have Experience with data annotation tools (e.g., Labelbox, Dataloop, LabelStudio). Experience with multi-modal AI/ML datasets (images, videos, text, audio). Experience in agile or fast-paced tech environments with exposure to AI/ML pipelines. Programming knowledge (e.g., Python). Understanding of data quality concepts for AI/ML and their impact on model performance. Knowledge of common quality metrics and statistical methods used in AI/ML data quality. Experience with prompt engineering and leveraging LLMs in daily work. Why You’ll Love This Role Work remote: Work from anywhere you’re most productive. Unique datasets: Audit, analyze, and improve complex AI/ML datasets across projects. Visible impact: Your quality checks and automation ideas shape model performance. Growth opportunities: Build expertise in AI data quality and scale with project growth. Collaborative team: Work with analysts, engineers, and stakeholders who value accurate AI. The Impact You’ll Make Guard data integrity through audits against guidelines and benchmarks to uphold data reliability. Identify and resolve annotation issues; surface insights to elevate quality. Improve annotation processes through tooling proficiency, automation proposals, and clear documentation. Collaborate and coach: Clarify requirements and support skill development within the annotation pipeline. Track and communicate quality performance: Regularly report metrics and insights to drive improvements. DEI & Equal Opportunity : TaskUs is an equal opportunity workplace. If you need reasonable accommodations in any part of the hiring process, please let us know. We do not solicit money during the application process. Apply through the official TaskUs careers page: Req Id: R_2507_10226_5 Posted At: Wed Jul :00:00 GMT+0000 (Coordinated Universal Time) #J-18808-Ljbffr