AI Data Quality Analyst

AI Data Quality Analyst

Why this role exists: As AI-driven solutions expand across industries, ensuring the fidelity of training and evaluation datasets is essential for building reliable models. TaskUs needs a diligent, detail-focused Data Quality Analyst who can maintain annotation standards and drive continuous improvements so that essential AI data remains accurate, efficient, and scalable.

Responsibilities

• Quality Audits - Perform quality audits on annotated datasets to ensure that they meet established guidelines and quality benchmarks.
• Statistical Reporting - Leverage statistical based quality metrics such as F1 score and other quality benchmarks.
• Root Cause Analysis - Analyze annotation errors, trends, project processes, and project documentation to identify and understand the root cause of errors and propose remediation strategies.
• Edge-Case Management - Resolve and analyze edge-case annotations to ensure quality and identify areas for improvement.
• Tooling - Become proficient in using annotation and quality control tools to perform reviews and track quality metrics.
• Guidelines - Become an expert in the project specific guidelines and provide feedback for potential clarifications or improvements.
• Automation - Identify opportunities to use automation to help enhance analytics, provide deeper insights, and improve efficiency.
• Documentation - Develop and maintain up-to-date documentation on quality standards, annotation guidelines, and quality control procedures.
• Feedback - Provide regular feedback that identifies areas for improvement across the annotation pipeline.
• Cross-Functional Teamwork - Work closely with key project stakeholders and clients to understand project requirements and improve annotation pipelines.
• Training - Assist with training annotators, providing guidance, feedback, and support to improve data quality.
• Reporting - Provide regular updates that highlight data quality metrics, key findings, and actionable insights for continuous process improvements.
• Experience - 1+ years of experience as a data analyst with exposure to data quality and/or data annotation, ideally within an AI/ML context.
• Familiarity - Familiarity with the basic concepts of AI/ML pipelines and data.

Core skills

• Strong analytical and problem-solving skills with an exceptional eye for detail.
• Excellent written and verbal communication skills, with the ability to clearly articulate quality issues and collaborate with diverse teams.
• Ability to work independently and manage time effectively to meet deadlines.
• A strong problem-solver who thinks critically and drives innovation and continuous optimization.
• A quick learner with the ability to work independently in a fast-paced environment.
• A strong focus on detail, balanced against strategic priorities.
• A positive can-do attitude and the ability to easily adapt to new environments.
• Not afraid to speak up.

Nice to have

• Familiarity with data annotation tools (e.g. Labelbox, Dataloop, LabelStudio).
• Experience working with multi-modal AI/ML datasets (images, videos, text, audio).
• Prior experience in an agile or fast-paced tech environment with exposure to AI/ML pipelines.
• Knowledge of programming languages (e.g. Python).
• Knowledge of the concepts and principles of data quality for AI/ML models and the impacts it can have on model performance.
• Working understanding of common quality metrics and statistical methods used in AI/ML data quality.
• Knowledge of AI/ML concepts and experience with data for AI/ML models.
• Experience in prompt engineering and leveraging LLMs in day-to-day work.

Why you’ll love this role

• Work remote: Work from anywhere you’re most productive
• Every dataset is unique: Audit, analyze, and improve complex AI/ML datasets; no two projects are the same.
• Visible impact: Your quality checks, root cause analyses, and automation ideas will directly shape how AI models perform in real-world applications.
• Room to grow: Build deep expertise in AI data quality and scale your influence as our projects and capabilities expand.
• Mission-driven team: Collaborate with analysts, engineers, and stakeholders who share a passion for creating accurate, reliable AI.

The impact you’ll make

• Guard data integrity through audits: Conduct quality audits on annotated datasets against guidelines and benchmarks (e.g., F1 score, inter‑annotator agreement) to uphold data reliability.
• Pinpoint and resolve annotation issues: Analyse error patterns, edge-case annotations, and root causes to surface insights and elevate annotation quality.
• Optimise annotation processes: Proficiently use annotation and QA tools, propose automation, and maintain clear documentation of quality standards and procedures.
• Collaborate and coach: Work with stakeholders and annotators to clarify requirements, provide constructive feedback, and support skill development within the annotation pipeline.
• Track and communicate quality performance: Regularly report on data quality metrics, key findings, and actionable insights to drive continuous improvements.

How We Partner To Protect You

TaskUs will neither solicit money from you during your application process nor require any form of payment to proceed with your application. Ensure you communicate only with authorized TaskUs recruiters.

DEI

TaskUs is committed to equal opportunity and inclusive practices. If you need reasonable accommodations in any part of the hiring process, please let us know. We welcome applicants from diverse backgrounds and demographics.

We invite you to explore TaskUs career opportunities and apply through the provided URL at

#J-18808-Ljbffr

---