MyComplianceOffice builds enterprise compliance and riskplatforms for global financial services clients. Our systems are distributed,highly configurable, and operate at scale—requiring quality engineeringapproaches that ensure reliability, trust, and regulatory confidence.
As our platform evolves to include AI-poweredcapabilities, quality becomes even more critical: AI systems must beexplainable, predictable within defined bounds, and safe to operate inregulated environments.
Role Summary
As a Senior QA Engineer – AI Products, you act as asenior technical authority responsible for defining and implementing qualitystrategies for AI-powered features and services built withinMyComplianceOffice.
In addition to governing automation frameworks at scale, youwill focus on validating AI system behaviour, including datadependencies, model outputs, non-deterministic behaviour, and production drift.You ensure that AI-driven functionality is reliable, trustworthy, andcompliant—both at release time and as models evolve in production.
You are accountable not just for writing automation, but forensuring confidence in AI systems at organisational scale.
Key Responsibilities
Core Automation & Platform Responsibilities
Design or govern modular automation frameworksadopted across multiple teams or platforms
Define and align automation goals across squadsor delivery streams
Solve complex automation challenges related todistributed systems, async flows, dynamic environments, or CI/CD scalability
Design, configure, and monitor risk- andsignal-based quality gates for high-volume pipelines
Build internal tools or integrations thatimprove execution speed, enablement, observability, and feedback loops
Define review standards, enforce qualitypractices, and enable safe test code reuse
Work closely with architects and seniorengineers to align test strategies with evolving system architecture
Drive automation and quality capability upliftthrough talks, training sessions, documentation, and pairing
AI & Machine Learning Quality Responsibilities
Define testing strategies for AI andML-powered systems, including model behaviour validation and data qualitychecks
Design approaches for testing non-deterministicand probabilistic outputs, where correctness is defined by behaviour,thresholds, or confidence ranges
Validate AI features for bias, fairness,explainability, and regulatory risk
Test AI behaviour across model versions, datachanges, and feature or prompt evolution
Partner with data scientists and ML engineers toembed quality gates into training, validation, and inference pipelines
Monitor production signals to detect modeldrift, degradation, or unexpected behaviour, and feed learnings back intotest strategy
Required Skills & Experience
5–8 years’ experience in quality engineering,automation, or SDET roles
Proven experience governing or scalingautomation frameworks across teams
Strong programming skills with an architecturalmindset
Deep understanding of CI/CD systems and testexecution at scale
Experience building internal test tooling orplatform integrations
Comfortable operating in ambiguous, evolvingtechnical environments
Strong cross-team influence and mentoringcapability
AI Systems Testing Experience (Required or StronglyPreferred)
Experience testing AI or ML-powered productsin production or pre-production environments
Understanding of the AI/ML lifecycle(training, validation, deployment, retraining)
Experience validating systems with non-deterministicoutputs
Familiarity with concepts such as falsepositives/negatives, confidence scoring, bias, and model drift
Ability to design tests where expected outcomesare behavioural, probabilistic, or risk-based, not purely binary
Desired Attributes
Systems thinker with strong judgement
Comfortable testing systems where correctnessis contextual rather than absolute
Pragmatic leader who balances standards withautonomy
Curious about how data, models, and userbehaviour interact in real-world systems
Strong ethical and risk awareness whenvalidating AI-driven decisions
Enjoys teaching, mentoring, and raisingorganisational capability
Data- and signal-driven, outcome-focused mindset
Why This Role Is Different
Opportunity to shape how AI quality isdefined and validated in a regulated enterprise platform
Real ownership of testing strategy for AIsystem — not just test execution
Space to influence architecture, model lifecycledecisions, and release confidence
Direct collaboration with senior engineering,architecture, and data leaders
#J-18808-Ljbffr