Job Description: As a senior engineer you will onboard new data assets to the data lake, as well as optimize existing data pipelines to the latest data integration standards using mostly SPARK, SCALA and JAVA on Hadoop ecosystem. Job Responsibilities: Development of data integration and analysis solutions using various big data technologies with a focus mainly on Spark/Scala and Java development. Testing and peer code review. Accurately estimate project activities and communicate deviations promptly to the lead or project manager for appropriate action by participating in development planning and progress review sessions. Attend all scoping and planning workshops for BAU and project initiatives and provide input into the design process to ensure that the solution meets the business requirements. Job Requirements: Understanding the end to end solutions delivery lifecycle Strong understanding of batch processing and scripting Strong Hadoop architecture knowledge with the ability to troubleshoot and optimize poorly performing tasks 7+ years of Big Data development experience using Spark, Scala, Java Good understanding of Java programming Some exposure to AWS services utilizing but not limited to EMR, Glue, Athena, S3, RDR, Step Functions, Lambda, Pipeline is beneficial Good understanding of metadata driven architecture
Jobsrican