Shyft6

Quality Engineering & AI Validation Manager

Shyft6

United States Remote Other 19h ago via Himalayas
quality-engineering qa-management ai-validation software-engineering test-automation

About this role

This is a remote position.

Quality Engineering & AI Validation Manager

Reporting to Head of Engineering, leads Accordion's centralized Quality Engineering and AI Validation function across software delivery pods. Owns the quality operating model, release-readiness standards, test strategy, AI validation framework, and quality metrics used to support both conventional software and AI-enabled solutions. Ensures quality is designed into delivery from the start, not inspected in at the end.

Key Responsibilities

Quality Strategy & Governance

  • Define the firm-wide quality strategy, release gates, and validation standards across development pods.
  • Establish a risk-based quality model for traditional software and AI-enabled workflows.
  • Define when pod-level validation is sufficient and when independent QA validation is required.

AI Validation Leadership

  • Build and operationalize the QA approach for AI-led products, including benchmark datasets, scoring rubrics, regression comparisons, and grounded-output validation.
  • Establish testing expectations for instruction adherence, consistency, business correctness, control compliance, and hallucination risk.
  • Partner with Engineering and Product leaders on quality implications of prompt, model, workflow, and tooling changes.

Team Leadership

  • Lead and develop quality engineers, AI quality analysts, and domain-oriented QA resources.
  • Improve the maturity of automation, evaluation routines, test evidence standards, and release discipline.
  • Create a scalable central model that supports pods without becoming a bottleneck.

Finance & Risk Alignment

  • Partner with Finance and practice SMEs to ensure solutions are validated against real business use, materiality, and control expectations.
  • Ensure high-risk workflows receive the right level of domain review before production release.

Production Quality

  • Define the framework for production quality monitoring, including escaped defects, output-quality degradation, reviewer overrides, and control failures.
  • Create visibility into quality trends, readiness, and recurring failure patterns for Engineering and leadership.

Requirements

Required Qualifications
  • 8+ years of QA, software testing, or quality engineering experience, including team leadership.
  • Experience building test strategy, release-readiness processes, and automation programs in modern software environments.
  • Experience supporting AI-enabled applications, workflow automation, data products, or decision-support systems.
  • Strong understanding of functional, integration, regression, API, and data validation approaches.
  • Ability to translate business-critical finance workflows into quality controls and release criteria.
  • Strong communication skills and credibility with Engineering, Product, and business stakeholders.

·Bachelor's degree preferred.

You Are
  • Structured, pragmatic, and highly credible.
  • Comfortable asking tough questions about readiness, risk, and evidence.
  • A builder of quality systems and operating models.
  • Fluent in both engineering quality and business impact.
  • Focused on trust, traceability, and scalable execution.

Benefits

Salary plus a performance-based bonus
Actual compensation packages are determined by evaluating a wide array of factors unique to each candidate, including but not limited to skill set, years and depth of experience, education, certifications, cost of labor, and internal equity.

Originally posted on Himalayas

Apply for this job via Himalayas