Job Overview
Kaseya is the leading provider of complete IT infrastructure and security management solutions for Managed Service Providers (MSPs) and internal IT organizations worldwide powered by AI.
We're looking for a quality-focused engineer who can think critically about testing complex, adaptive systems.
* Design and implement comprehensive test strategies for AI agents, platform services, and integrations
* Develop and maintain automated test frameworks for functional, performance, and security testing
* Validate AI model behavior, including prompt outputs, bias detection, and explainability
* Conduct rigorous testing of distributed systems, microservices, and real-time data pipelines
* Ensure data quality and integrity across training, inference, and integration workflows
* Collaborate with engineering and research teams to define quality gates and ethical testing standards
* Drive continuous testing practices in agile development cycles
Responsibilities and Requirements:
* 5–7+ years of experience in QA or Test Engineering
* Mastery of test automation frameworks (e.g., PyTest, Playwright, Selenium, RestAssured)
* Strong API testing skills (manual and automated)
* Experience with performance, load, and high-volume stress testing (e.g., JMeter, k6, Gatling)
* Deep understanding of AI/ML model evaluation metrics and methodologies
* Experience testing distributed systems and microservices architectures
* Proficiency in data quality validation for AI training and inference workflows
* Experience with contract testing for microservices
* Ability to develop test strategies for non-deterministic and adaptive behaviors in AI systems (e.g., prompt validation, guardrails, bias detection)
* Familiarity with AI model testing tools (e.g., CheckList, Giskard, Deepchecks)
* Experience with test planning, execution, and reporting in agile environments
* Bachelor's degree in Computer Science, Software Engineering, or a related field
* Preferred: Certifications in QA Automation, AI/ML Testing, or Ethical AI (e.g., ISTQB, Certified Ethical Emerging Technologist)
About Kaseya
Kaseya provides equal employment opportunity to all employees and applicants without regard to race, religion, age, ancestry, gender, sex, sexual orientation, national origin, citizenship status, physical or mental disability, veteran status, marital status, or any other characteristic protected by applicable law.