Lead Linguist

We are seeking a skilled Linguistic Quality Lead to play a pivotal role in our AI Rater Incubation Program. This position will ensure that rater guidelines are accurate, usable, and aligned with client goals, focusing on ads relevance projects.

1. Translate client objectives into clear, actionable rating guidelines.
   
2. Ensure guidelines reflect what "good quality" means, balancing linguistic rigor and client business needs.
   
3. Collaborate with clients during pilot phases to refine guidelines and make adjustments based on test outcomes.
   
4. Create and manage initial test sets to validate guidelines.
   
5. Review rater performance during pilots, identify error patterns, and adjust standards accordingly.
   
6. Partner with data scientists to ensure pilots are measurable and data-backed.
   
7. Act as the quality authority for rater performance, ensuring alignment with guidelines.
   
8. Develop and deliver training sessions for raters, clarifying complex linguistic or evaluative concepts.
   
9. Monitor rater consistency and provide structured feedback to raise quality standards.
   
10. Work closely with program managers (client-facing coordination) and data scientists (insights/metrics support).
    
11. Maintain comprehensive documentation of guideline updates, quality frameworks, and training processes.
    
12. Perform mid-level reporting and analysis on rater performance (e.g., agreement rates, recurring errors).
    
13. Highlight risks, biases, or ambiguities in guidelines and propose solutions.
    
14. Continuously refine processes to improve consistency, efficiency, and client satisfaction.
    

• Expertise in semantics, pragmatics, discourse, and applied linguistic analysis.
  
• Demonstrated ability to develop and refine annotation/rating guidelines.
  
• Skilled in Google Sheets (advanced formulas, data validation, quality tracking).
  
• Experience delivering rater training, coaching, and feedback.
  
• Strong analytical skills and ability to connect linguistic quality with operational or client outcomes.