Job Description: Support in providing infrastructure, tools and frameworks used to deliver end-to-end solutions to business problems. Build scalable infrastructure for supporting the delivery of business insights from raw data sources with a focus on collecting, managing, analysing, visualising data and developing analytical solutions. Job Responsibilities: Responsible for expanding and optimising the organisations data and data pipeline architecture, whilst optimising data flow and collection to ultimately support data initiatives. Job Requirements: Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc. Experience with AWS cloud services: EC2, EMR, RDS, Redshift. Experience with big data tools: Hadoop, Spark, Kafka, etc. Experience with relational SQL and NoSQL databases, including Postgres and Cassandra. Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc. Strong analytic skills related to working with unstructured datasets. Build processes supporting data transformation, data structures, metadata, dependency and workload management. A successful history of manipulating, processing and extracting value from large disconnected datasets. Experience with stream-processing systems: Storm, Spark-Streaming, etc.
Jobsrican