Job Summary
Lead- Data Automation and Excellence role based in New Jersey & Ireland (hybrid work model).
Senior individual contributor and technical leader responsible for leading a technology squad in the planning, design, monitoring, building and operation of high-quality, scalable, and AI-ready data assets.
Hands-on technical leadership, drives automation of data curation using AI agents, sets engineering standards, and ensures alignment with enterprise architecture and SDLC.
Major Duties & Responsibilities
Solution Architecture & Technical Enablement
– Drive solution architecture for data across ingestion, curation, and consumption layers.
Guide squad members in solving complex technical challenges and support implementation of sound, scalable solutions.
Shape technology standards and framework to ensure adherence to coding standards, architectural patterns, and data engineering best practices.
Mentor and coach engineers to raise overall technical maturity and delivery quality within the squad.
Lead the technical design of data assets using a medallion architecture (Bronze, Silver, Gold) with Technical Product owners within a federated data product squad model.
Recommend and introduce improved technical approaches to data architecture, modelling tools, and solution design to enhance scalability and maintainability.
Drive technical improvements in data curation practices using Change Data Capture (CDC) and event-driven curation patterns.
Design and guide implementation of event-driven and streaming architectures, including Kafka-based solutions integrated with batch pipelines.
Operational Excellence & SDLC
– Improve operational efficiency through optimization of CI/CD pipelines aligned with the enterprise SDLC toolchain.
Promote automation in partnership with information architects by introducing tools and AI-driven capabilities across testing, deployment, data quality validation, and observability to reduce manual effort and operational support risk.
Actively manage and prevent accumulation of technical debt while balancing delivery commitments.
AI Enablement & Data Readiness
– Apply data engineering best practices alongside strong AI and GenAI literacy: prompt engineering and AI-assisted development, large language models (LLMs) and GenAI concepts, data profiling and automated asset creation.
Collaborate with architects, platform teams, and product owners to ensure data products are designed with AI readiness and reuse in mind.
Other Duties
– Partner with Product Owners to refine technical requirements, define acceptance criteria, and support backlog shaping; collaborate with other Lead Engineers to align on standards, manage cross-squad dependencies, and resolve shared technical challenges; stay current with emerging data engineering, streaming, and AI technologies and assess their applicability within the platform ecosystem; support onboarding of new engineers by providing guidance on tools, frameworks, and engineering practices.
Required Minimum Education
Bachelor's Degree in Computing or similar discipline.
Required Knowledge, Skills and Abilities
Overall Experience: 7+ years in data engineering, including 3–5+ years in a senior or technical lead role.
Strong experience designing, building, and operating data pipelines across batch, streaming, and near-real-time use cases.
5+ years experience working with modern layered or medallion architectures, ideally within a federated data product model.
Proven ability to mentor engineers, lead by technical influence, and collaborate effectively across product, architecture, and platform teams.
3+ years hands-on experience with Kafka, event-driven architectures, and CDC-based ingestion and curation patterns.
4+ years experience implementing and optimizing CI/CD pipelines aligned with enterprise SDLC standards for data platforms.
Working knowledge of AI and GenAI concepts relevant to data platforms, including LLMs, prompt engineering, ontologies, metadata enrichment, and data profiling.
Strong understanding of data quality, lineage, governance, contextualization, and cataloging to support AI-ready data assets.
Preferred Skills
Agile Product Development, Analytical Reasoning, Coaching, Collaborating, Competitive Landscape Analysis, Critical Thinking, Customer Alignment, Demand Forecasting, Human-Computer Interaction (HCI), Organizing, Product Development, Product Improvements, Product Strategies, Requirements Analysis, Research and Development, Software Development Life Cycle (SDLC), Software Development Management, Stakeholder Management, Technical Credibility, Technical Writing, Technologically Savvy.
Compensation & Benefits
Base pay range: €63,****** – €101,******.
Benefits include an annual bonus with a set target (% of pay) depending on pay grade / location, vacation days, parental leave for a minimum of 12 weeks, bereavement leave, caregiver leave, volunteer leave, well-being reimbursement, programs for financial, physical and mental health, service anniversary and recognition awards, and several insurance plans.
#J-*****-Ljbffr