IT Specialist (AI) : NIST/CAISI

Mission
NIST’s Center for AI Standards and Innovation (CAISI) is seeking a candidate to develop scientific methods and tools to rigorously evaluate AI systems in real-world settings. You will support the AI Action Plan by identifying failure modes and areas of unreliability.

Key Responsibilities
Evaluation & Methodology: Design methods for scientifically sound evaluations of AI systems in application, post-deployment, and real-world settings.

Tooling & Standards: Develop tools and guidelines for rigorous evaluation of end-to-end AI systems (e.g., in sector-specific tasks) to identify functionality issues.

Analysis: Analyze and improve evaluation instruments themselves, including red teaming, benchmarks, and automated scoring techniques.

Collaboration: Work with cross-functional teams to invent solutions to AI evaluation problems and inform the development of technical best practices.

Communication: Communicate complex AI concepts to both technical and non-technical audiences.

Qualifications
Important Note: Education cannot be used to qualify for this position. You must meet specialized experience requirements.

Basic Requirements
Must demonstrate proficiency in: Attention to Detail, Customer Service, Oral Communication, and Problem Solving.

Specialized Experience
For ZP-IV (GS-12 Equivalent): 1 year of experience demonstrating technical understanding of Generative AI/ML algorithms, designing evaluation methods in real settings, and collaborating on AI projects.

For ZP-V (GS-14 Equivalent): 1 year of experience (at the GS-14 level) demonstrating advanced technical understanding of Generative AI/ML, leading cross-functional AI teams, and establishing best practices/workflows.

Security & Conditions
Citizenship: U.S. Citizens only.

Security Clearance: Sensitive Compartmented Information (SCI) / Special-Sensitive (High Risk).

Drug Testing: Required.

Financial Disclosure: OGE-450 form required annually.

Application Critical Details
This application has strict formatting and submission rules. Failure to follow them will result in disqualification.

Resume Limit: Your resume must NOT exceed two pages. If it is longer, only the first two pages will be reviewed.

Narrative Questions: You will be asked 4 narrative questions. You must certify that you did not use AI (like ChatGPT or Copilot) to write these responses.

Required Documents: Resume, vacancy question responses, and any eligibility documents (e.g., SF-50 for federal employees).