Join to apply for the Data Engineer role at myGwork - LGBTQ+ Business Community
Calling all innovators - find your future at Fiserv. We're Fiserv, a global leader in Fintech and payments, and we move money and information in a way that moves the world.
Job Title: Data Engineer
Data Engineers are Data enthusiasts who use their toolbox of technical talent to deliver data driven solutions. This person will be part of and support agile teams of data analytics domain in EMEA by designing and building cutting-edge data migration, data integration, data replication and data streaming systems to ensure we make data available with amazing quality and speed in Snowflake.
Primary Responsibilities
* Developing ETL pipelines in and out of data warehouse using combination of Java/Scala/Python Spark jobs for data transformation and aggregation
* Writing SQL queries against Snowflake.
* Provide production support for Data Warehouse issues such data load problems, transformation translation problems
* Develop unit tests for transformations and aggregations
* Develop production grade real time or batch data integrations between systems
* Real time data processing of events from Kafka using streaming processing into data warehouse.
* Design and build data pipelines of medium to high complexity
* Translate requirements for BI and Reporting to Database design and reporting design
* Understanding data transformation and translation requirements and which tools to leverage to get the job done
* Design and build machine learning pipelines of medium to high complexity
* Execute practices such as continuous integration and test-driven development to enable the rapid delivery of working code.
* Deploy production grade data pipelines, data infrastructure and data artifacts as code.
* Develop estimates for data driven solutions
* Communicate technical, product and project information to stakeholders
* Establish standards of good practice such as coding standards and data governance
* Peer review code developed by others
Knowledge & Skills
* Minimum BSc or BTech / B.E in Computer Science, Engineering, or related discipline.
* Relevant professional qualification such as AWS Certified Big Data, SnowPro Core certification, other Data Engineer certifications
* Strong development hands-on background in creating Snow pipe, and complex data transformations and manipulations using Snow Pipe, Snow SQL
* Hands-on experience with Snowflake external tables concepts, Staging, Snow scheduler & performance tuning.
* Good understanding of Snowflake Timetravel concepts and zero-copy cloning, Network policies, clustering, and tasks
* 5+ year experience working in an enterprise big data environment
* Deep knowledge of Spark, Kafka and data warehouse such as snowflake, Hive, Redshift etc
* Hands-on experience in development, deployment and operation of data technologies and platforms such as:
* Integration using APIs, micro-services and ETL patterns
* Low latency/Streaming, batch and micro batch processing
* Data platforms such as Hadoop, Hive, Redshift or Snowflake
* Cloud Services such as AWS
* Cloud query services such as Athena
* DevOps Platforms such as Gitlab
* Containerisation technologies such as Docker and Kubernetes
* Orchestration solutions such as Airflow
* Deep knowledge of key non-functional requirements such as availability, scalability, operability, and maintainability
* Deep knowledge of SQL
* OS knowledge particularly Linux
* High on social skills, team spirit and empathy.
* Willingness to take ownership and ability to show strong personal commitment for the department and the team goals; must be comfortable with being considered a reliable and proactive influential team member who is not afraid to take on responsibility in the team.
* Ability to communicate clearly to business analysts and stakeholders as well as technical representatives.
* Very strong and proven communication and coordination skills, open-minded and determined.
* Responsible for planning, highlighting, and implementing possible improvements for existing and new applications.
* Good to have:
* Migration experience to Snowflake
* Hands on experience with Oracle RDBMS
* Exposure to Streamsets, DBT or other ETL tool
Our Commitment To Diversity And Inclusion
Fiserv is proud to be an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, gender, gender identity, sexual orientation, age, disability, protected veteran status, or any other category protected by law.
#J-18808-Ljbffr