Join to apply for the Data Engineer with Apache NiFi role at Keyrus
Overview
Empowered by our motto, "Making Data Matter", Keyrus stands at the forefront of global consultancy in Data & Analytics, AI & Machine Learning, and Digital Transformation. With a presence in 27 countries across 4 continents, we're on a mission to elevate the impact of data. We are data experts, guiding our clients through their data journey regardless of their level of maturity in analytics. As an end-to-end partner, we address varied challenges that our clients face, from Advanced Analytics, Cloud, Data Governance, Visualisation to Enterprise Performance Management, from advisory to delivery.
With more than 3200 employees in all corners of the world, Keyrus is dedicated to helping businesses taking advantage of the Data and Digital paradigm to enhance their performance, facilitating, and accelerating their transformation, and generating new drivers of growth, competitiveness, and sustainability.
Job Summary
Keyrus is looking for a Senior Data Engineer. The ideal candidate embodies technical prowess in Apache NiFi, coupled with a fervent commitment to making every piece of data matter. We value individuals who are detail-oriented, independent contributors with exceptional communication and problem-solving skills, ensuring that together, we continue to transform possibilities into reality.
Note: All applications/CV’s must be submitted in English.
Key Responsibilities
* Develop and maintain data pipelines using Apache NiFi, ensuring efficient data flow and transformation.
* Integrate structured and unstructured data from multiple sources, including Salesforce, APIs, SFTP, SAP, and CRM, into cloud-based and on-premise databases.
* Ensure data quality, security, and governance across diverse sources and destinations.
* Optimise and troubleshoot NiFi workflows to enhance performance, reliability, and scalability.
* Work with APIs to fetch and send real-time data for external integrations (e.g., CustomerGauge).
* Manage and maintain NiFi architecture in a Kubernetes environment.
* Collaborate with data scientists, analysts, and software engineers to ensure seamless data integration and usability.
* Optimise ETL/ELT processes for big data processing.
* Use NiFi Registry and NiFi Toolkit to maintain version control and streamline deployments across environments.
* Develop Bash and Python scripts for automation, monitoring, and workflow execution.
* Implement CI/CD pipelines for data pipeline deployments and workflow automation.
* Analyse and resolve data pipeline issues, ensuring high availability and reliability.
Requirements for the role
* Strong expertise in Apache NiFi, including flow design, performance tuning, version control, and security configurations.
* Experience in data ingestion, transformation, and processing for both batch and real-time workflows.
* Proficiency in SQL and NoSQL databases, such as PostgreSQL, MySQL, and MongoDB.
* Experience with big data technologies, including Apache Kafka, Spark, Hadoop, and HDFS.
* Familiarity with cloud platforms (AWS, Azure, or GCP) and their data services (e.g., AWS Redshift, Azure Synapse, Google BigQuery).
* Proficiency in at least one programming language, such as Python or Java.
* Experience with CI/CD pipelines, version control tools (Git, Jenkins), and automation frameworks.
* Strong problem-solving skills and ability to work independently in a remote environment.
* Must be a Portuguese or European citizen or have a work permit for Portugal
Good to have
* Experience working with data warehousing solutions such as Snowflake, Redshift, or BigQuery.
* Understanding of containerization and orchestration tools, including Docker and Kubernetes.
* Knowledge of data security and compliance best practices (e.g., GDPR, HIPAA).
* Prior experience with data lake and real-time analytics solutions.
What we offer at Keyrus Portugal
* Competitive salary based on your skills & experience and according to the Data market practice
* Meal allowance of €8.32 per day
* Flexible benefits option
* Private Medical Insurance
* 22 days of annual leave (increasing every 3 years up to a maximum of 25 days)
* Training Program through KLX (Keyrus Learning Experience) Platform
Why Keyrus?
Joining Keyrus means joining a market leader in the Data Intelligence field and an (inter)national player in Management Consultancy and Digital Experience.
Keyrus Portugal operates on five guiding pillars that shape our approach to work and collaboration – Diversity; Trust; Flexibility; Ownership & Innovation - we're dedicated to cultivating a team as diverse as the challenges we take on.
We embrace and celebrate the rich variety of backgrounds, perspectives, and talents that each individual brings. We firmly believe that diversity fuels innovation, ignites creativity, and propels us towards greatness.
Regardless of race, ethnicity, gender, sexual orientation, disability, age, or any other dimension of diversity, everyone is not only welcome but valued here.
Join us in shaping a future where everyone can thrive!
Seniority level
* Not Applicable
Employment type
* Other
Job function
* Information Technology
#J-18808-Ljbffr