Overview
Join to apply for the
Staff AI QA Engineer
role at
Kaseya
.
Kaseya is the leading provider of complete IT infrastructure and security management solutions for Managed Service Providers (MSPs) and internal IT organizations worldwide powered by AI.
Founded in 2000, Kaseya serves customers in over 20 countries and manages over 15 million endpoints worldwide.
To learn more, visit .
Kaseya is not your typical company.
The thousands of people who succeed at Kaseya are prepared to go above and beyond for the betterment of our customers.
What you'll do
Join the dynamic team at Kaseya as a Staff AI QA Engineer.
Ensure the reliability, safety, and ethical integrity of our AI platform by crafting advanced testing strategies and automation frameworks tailored for adaptive, large-scale agentic systems and complex integrations.
You'll validate AI behavior, secure data pipelines, and enforce quality standards across distributed systems to ensure our intelligent agents operate with precision, fairness, and resilience in real-world environments.
What we are looking for
We're looking for a quality-focused engineer who can think critically about testing complex, adaptive systems.
You're a strong communicator who can clearly articulate quality risks and testing strategies to both technical and non-technical stakeholders.
You thrive in collaborative environments, working closely with developers, data scientists, and product managers, and you bring leadership in QA best practices, mentoring, and ethical testing.
Essential duties and responsibilities
Design and implement comprehensive test strategies for AI agents, platform services, and integrations
Develop and maintain automated test frameworks for functional, performance, and security testing
Validate AI model behavior, including prompt outputs, bias detection, and explainability
Conduct rigorous testing of distributed systems, microservices, and real-time data pipelines
Ensure data quality and integrity across training, inference, and integration workflows
Collaborate with engineering and research teams to define quality gates and ethical testing standards
Drive continuous testing practices in agile development cycles
What you'll bring
5–7+ years of experience in QA or Test Engineering
Mastery of test automation frameworks (e.g., PyTest, Playwright, Selenium, RestAssured)
Strong API testing skills (manual and automated)
Experience with performance, load, and high-volume stress testing (e.g., JMeter, k6, Gatling)
Deep understanding of AI/ML model evaluation metrics and methodologies
Experience testing distributed systems and microservices architectures
Proficiency in data quality validation for AI training and inference workflows
Experience with contract testing for microservices
Ability to develop test strategies for non-deterministic and adaptive behaviors in AI systems (e.g., prompt validation, guardrails, bias detection)
Familiarity with AI model testing tools (e.g., CheckList, Giskard, Deepchecks)
Experience with test planning, execution, and reporting in agile environments
Bachelor's degree in Computer Science, Software Engineering, or a related field
Preferred: Certifications in QA Automation, AI/ML Testing, or Ethical AI (e.g., ISTQB, Certified Ethical Emerging Technologist)
Additional training in AI safety, fairness, or model validation is a plus
Nice to have
Experience with AI red teaming and adversarial testing techniques
Explainability testing and validation for XAI methods (e.g., LIME, SHAP)
Usability testing for AI-driven UIs and agent interactions
Familiarity with data privacy testing techniques (e.g., differential privacy validation)
Security testing for AI models (e.g., model inversion, membership inference, poisoning attacks)
Experience testing AI ethics, fairness, and bias
Knowledge of formal verification methods for AI
Experience building synthetic data generation pipelines for testing
Additional Information
Kaseya provides equal employment opportunity to all employees and applicants without regard to race, religion, age, ancestry, gender, sex, sexual orientation, national origin, citizenship status, physical or mental disability, veteran status, marital status, or any other characteristic protected by applicable law.
Seniority level
Mid-Senior level
Employment type
Full-time
Job function
Engineering and Information Technology
Industries
Software Development
Referrals increase your chances of interviewing at Kaseya.
Get notified about new Staff Quality Assurance Engineer jobs in Dublin, Ireland.
#J-18808-Ljbffr