Data Quality and Labeling Specialist

About Collate

Collate is an AI document generation platform for life sciences. We automate paperwork with AI, helping our customers get life-saving innovations to patients years faster. Collate is an end-to-end solution, powering every step of drug, diagnostic, and medical device development—from concept to market.

Our CEO Surbhi Sarna is a former General Partner at Y Combinator. Surbhi founded nVision Medical, which developed a new method to detect ovarian cancer and was acquired by Boston Scientific. Our CTO Nate Smith is a former Visiting Partner at Y Combinator and founder of Lever. Our AI researchers, engineers, and designers have worked at Google, Nvidia, Meta, Netflix, Amazon, AirBnB, Hippocratic AI, and Grail, and 40% of our team are former founders.

We’re an elite team, with $125M in funding from top investors (Redpoint, First Round Capital, CRV, Conviction, Y Combinator) and leaders in healthcare and AI. This is a rare chance to join an early-stage company with world-changing potential, experienced founders, and resources to execute at scale.

About the Role We're looking for a Data Quality and Labeling Specialist to bring life-sciences expertise to the data behind our AI. Our models draft, extract from, and review regulated documents across drug, diagnostic, and medical device development, and this role makes sure the data they learn from and are measured against is accurate, consistent, and grounded in how regulated work is actually done. This is a hands-on role at the intersection of domain expertise and data quality. You'll work side by side with our AI researchers and engineers, and your judgment about what a correct document looks like will shape the product directly.

What You’ll Do

• Review and label regulatory, quality, and clinical documents and model-generated outputs for accuracy, completeness, and compliance.
• Build and maintain gold-standard datasets used to train and evaluate our models.
• Write and maintain labeling guidelines and evaluation rubrics as new document types and use cases are added.
• Run calibration with other reviewers to keep labels consistent and resolve disagreements.
• Audit labeled data before it reaches training and evaluation, and track label quality over time.
• Turn recurring model errors into clear, actionable feedback for the research and engineering teams.

What We’re Looking For

• Experience in a regulated life-sciences function such as regulatory affairs, quality, clinical, or CMC, at a pharma, biotech, diagnostics, or medical device company.
• Familiarity with the documents and standards that govern that work (e.g., FDA, EMA, ICH, or ISO requirements).
• Strong attention to detail: you notice an unsupported claim or inconsistent value and can explain why it matters.
• Ability to apply structured rubrics consistently and to flag when a rubric itself needs changing.
• Clear written communication.
• Prior experience with data annotation, model evaluation, or writing guidelines for other reviewers is a plus.
• Excitement to work in an early-stage environment on-site in San Francisco.

Why Join Collate? Impact: The documents you help get right are the ones that put drugs, diagnostics, and devices in front of patients sooner. Ownership: You define what "correct" means for a new category of AI product and own the data that enforces it, from day one. Learning: Work daily with AI researchers, engineers, and life-sciences leaders; see how your labels change model behavior within the week. Upside: Join early enough for meaningful equity and a role that grows with the data function—into leading a team of reviewers and contract SMEs as we scale.