AI Data Quality Specialist

AI Data Quality Role

In this critical position, you will play a key role in ensuring the accuracy and quality of data for our clients. This involves working closely with cross-functional teams to understand project requirements and improve annotation pipelines.

• Quality Audits: Perform quality audits on annotated datasets to ensure they meet established guidelines and quality benchmarks.
• Statistical Reporting: Leverage statistical based quality metrics such as F1 score and inter-annotator agreement to evaluate data quality.
• Root Cause Analysis: Analyze annotation errors, trends, project processes, and project documentation to identify and understand the root cause of errors and propose remediation strategies.
• Edge-Case Management: Resolve and analyze edge-case annotations to ensure quality and identify areas for improvement.
• Tooling: Become proficient in using annotation and quality control tools to perform reviews and track quality metrics.
• Guidelines: Become an expert in the project-specific guidelines and provide feedback for potential clarifications or improvements.

Continuous Improvement

• Automation: Identify opportunities to use automation to enhance analytics, provide deeper insights, and improve efficiency.
• Documentation: Develop and maintain up-to-date documentation on quality standards, annotation guidelines, and quality control procedures.
• Feedback: Provide regular feedback that identifies areas for improvement across the annotation pipeline.

Cross-Functional Teamwork

• Collaborate with key project stakeholders and clients to understand project requirements and improve annotation pipelines.
• Assist with training annotators, providing guidance, feedback, and support to ensure data quality.
• Provide regular updates that highlight data quality metrics, key findings, and actionable insights for continuous process improvements.

To succeed in this AI Data Quality Analyst role, you should possess:

• 1+ years of experience as a data analyst with exposure to data quality and/or data annotation - ideally within an AI/ML context.
• Familiarity with the basic concepts of AI/ML pipelines and data.
• Strong analytical and problem-solving skills with an exceptional eye for detail.
• Excellent written and verbal communication skills, with the ability to clearly articulate quality issues and collaborate with diverse teams.
• Ability to work independently and manage time effectively to meet deadlines.
• A strong problem-solver who thinks critically and drives innovation and continuous optimization.
• A quick learner with the ability to work independently in a fast-paced environment.
• A strong focus on detail, balanced against strategic priorities.
• A positive can-do attitude and the ability to easily adapt to new environments.

We prioritize employee well-being with competitive salaries and comprehensive benefits packages.

PREFERRED QUALIFICATIONS

• Familiarity with data annotation tools (e.g., Labelbox, Dataloop, LabelStudio etc.).
• Experience working with multi-modal AI/ML datasets (images, videos, text, audio).
• Prior experience in an agile or fast-paced tech environment with exposure to AI/ML pipelines.
• Knowledge of programming languages (e.g., Python).
• Knowledge of the concepts and principles of data quality for AI/ML models and the impacts it can have on model performance.
• Working understanding of common quality metrics and statistical methods used in AI/ML data quality.
• Knowledge of AI/ML concepts and experience with data for AI/ML models.
• Experience in prompt engineering and leveraging LLMs in your day-to-day work.

Bachelor's/Master's degree in Statistic, Mathematics, Computer Science, Computer Engineering, or a directly related field.