Role overview

Scale AI is seeking a Subject Matter Expert to evaluate quality and establish standards for specialized projects. This part-time, hourly position offers flexible scheduling based on your availability, allowing you to work as many or as few hours as suit your needs. The role focuses entirely on quality assessment and continuous improvement rather than operational management or contributor support.

Key responsibilities

Conduct structured audits of task outputs to assess accuracy, consistency, and alignment with established project guidelines.
Apply specialized domain knowledge to evaluate complex work that requires expert-level assessment.
Identify patterns in quality issues and flag systemic gaps that affect overall output standards.
Define and refine quality benchmarks, scoring rubrics, and evaluation criteria tailored to project requirements.
Analyze audit findings to generate actionable insights and provide structured recommendations for quality improvements.
Support calibration efforts with internal stakeholders to ensure consistent quality expectations and scoring alignment.
Recommend enhancements to workflows, guidelines, and evaluation processes based on audit results.

Requirements

Deep expertise in a specialized field such as medicine, law, engineering, or a comparable domain.
Strong analytical capability to evaluate complex outputs against defined quality standards.
Experience in quality assurance, auditing, or evaluation frameworks is preferred.
Demonstrated ability to produce clear, structured, and actionable feedback.
High attention to detail with a strong commitment to accuracy and consistency.

What makes this role worth considering

This engagement offers genuine flexibility with no fixed schedule requirements—you control your hours based on availability. The role is designed for independent, expertise-driven contributors who prefer depth over breadth, with no administrative overhead, people management, or real-time support responsibilities. You will work directly on quality assessment and continuous improvement initiatives for projects supporting major organizations including Meta, Mayo Clinic, and government agencies.