Draft a job description for an evaluator role that is accurate, fair, and usable by hiring managers. Organization and role context: [ORGANIZATION AND CONTEXT] What the evaluator will assess: [EVALUATION SCOPE] Core responsibilities and workflow: [RESPONSIBILITIES AND WORKFLOW] Required capabilities and qualifications: [REQUIRED QUALIFICATIONS] Employment details and policy constraints: [EMPLOYMENT DETAILS] Write a 650-900 word job description with these sections: job title and reporting line; role purpose; essential responsibilities; evaluation process and decision authority; required qualifications; preferred qualifications; tools or subject-matter knowledge; success measures for the first 90 days; working conditions or schedule if relevant; compensation and location language only if supplied; and an equal-opportunity statement placeholder aligned to company policy. Describe evaluation work precisely. Specify what is being judged, the rubric or standards used, documentation expected, escalation paths for ambiguous cases, calibration with other evaluators, confidentiality expectations, and how conflicts of interest are disclosed. Distinguish essential job functions from preferences. Use observable capabilities rather than vague traits such as “rock star,” “digital native,” or “culture fit.” Do not add degree, years-of-experience, background-check, location, authorization, compensation, physical, or legal requirements unless my inputs support them and company policy permits them. After the description, add a short hiring-manager note with five structured interview topics tied to the required qualifications and one work-sample exercise that measures the actual evaluation task. Make clear that interviewers should use the same core questions and scoring criteria for comparable candidates. Before answering, check for requirements that could be discriminatory, unrelated to the essential work, or internally inconsistent. Also flag any policy, classification, pay-transparency, privacy, or legal language that must be reviewed by HR or counsel rather than presenting it as settled. Ask up to 3 clarifying questions only if a required input is missing.
Fill in
| Placeholder | What to enter | Example |
|---|---|---|
| [ORGANIZATION AND CONTEXT] | Describe the organization, department, reporting line, and why the role exists. | A remote learning company’s Content Quality team; reports to the Quality Operations Manager; created to review new lesson content before release. |
| [EVALUATION SCOPE] | Explain exactly what the evaluator will review and the standards or rubric they will apply. | Evaluate grade 6–8 science lessons against an internal rubric covering factual accuracy, instructional clarity, accessibility, source quality, and alignment to approved standards. |
| [RESPONSIBILITIES AND WORKFLOW] | List daily work, volume, stakeholders, decision rights, documentation, and escalation steps. | Review 20–25 lessons weekly, document rubric scores and evidence in Airtable, return revisions to editors, join weekly calibration, and escalate safety or factual disputes to the science lead. |
| [REQUIRED QUALIFICATIONS] | List only the capabilities, knowledge, credentials, and experience genuinely needed for the work. | Ability to apply a detailed rubric consistently; strong written evidence notes; familiarity with middle-school science content; comfort comparing sources; two years of editorial, education, QA, or evaluation work. |
| [EMPLOYMENT DETAILS] | Provide known location, schedule, classification, compensation, benefits, and company-policy constraints. | Full-time exempt role, U.S. remote, core collaboration hours 10 a.m.–3 p.m. Eastern. Salary range and benefits will be supplied by HR. Use the company’s approved equal-opportunity statement as a placeholder. |
How to use
- Gather the actual rubric, sample work, reporting line, and decision rights before drafting the description.
- Paste all five inputs and remove any requirement that is only a manager preference rather than necessary for the work.
- Check that the required qualifications map to a responsibility and that the work sample resembles the real task.
- Follow up with: “Create a structured interview scorecard with anchored ratings from 1 to 4 for the five interview topics.”
Variations
Contract evaluator
Use when defining a project-based external evaluator role.
Draft a statement of work and role description for a contract evaluator assessing [ITEMS TO EVALUATE] against [RUBRIC OR STANDARD]. Project context: [PROJECT CONTEXT]. Include deliverables, review volume, quality threshold, documentation format, calibration process, confidentiality, conflict-of-interest disclosure, acceptance process, and invoicing milestones if supplied. Clearly separate contractor outcomes from employee-style controls or benefits. Do not invent classification, tax, pay, or legal terms; flag them for HR, procurement, or counsel review. Ask up to 3 questions only if required inputs are missing.
Evaluator scorecard
Use when interviewers need a consistent way to assess candidates.
Create a structured interview scorecard for an evaluator role reviewing [EVALUATION SUBJECT]. Required capabilities are [REQUIRED CAPABILITIES], and the job workflow is [WORKFLOW]. Produce five competency areas, one behavioral question and one follow-up for each, evidence to listen for, red flags tied to the job, and anchored ratings from 1 to 4. Add a practical work-sample prompt and scoring rubric. Use consistent questions for comparable candidates and avoid protected-characteristic proxies. Ask up to 3 questions only if necessary.
Rubric calibration plan
Use when multiple evaluators need to score work consistently.
Build a rubric calibration plan for [EVALUATOR GROUP] who assess [WORK PRODUCT] using [RUBRIC]. Current inconsistency: [CURRENT PROBLEM]. Provide a kickoff process, blind double-scoring sample, disagreement-resolution method, acceptable agreement threshold if supplied, evidence log, recurring calibration agenda, and method for revising unclear rubric criteria. Do not claim reliability has improved without measurement. End with a dashboard definition for agreement, turnaround time, and escalation rate. Ask up to 3 questions only if required inputs are missing.
Tips
- Write essential functions as observable work, such as “document evidence for rubric ratings,” rather than personality labels or broad claims about fit.
- Require calibration because two qualified evaluators can interpret ambiguous standards differently without shared examples and a disagreement process.
- Use a work sample drawn from the actual task, with identifying information removed, instead of relying only on self-reported experience.
- Have HR or counsel review classification, pay transparency, background checks, work authorization, and equal-opportunity language for the jurisdictions involved.
FAQ
What qualifications should an evaluator job description include?
Include the knowledge and abilities needed to apply the actual rubric, document evidence, handle ambiguity, and maintain consistency. Avoid credentials that do not change performance on the essential work.
How do I avoid bias in evaluator hiring?
Use a shared scorecard, consistent core questions, and a job-relevant work sample with defined scoring criteria. Keep “culture fit” out of the selection standard.
Should an evaluator make final decisions?
State the decision authority clearly. Some roles score and recommend, while others approve, reject, or escalate cases; ambiguity creates inconsistent handling.