Accessibility Links

SC Cleared - Big Data Platform Engineer

  • Salary: Negotiable
  • Job type: Contract
  • Location: Croydon, Surrey
  • Sector: Other
  • Date posted: 07/11/2018
  • Job reference: BBBH92255

Platform Engineer - Big Data

Croydon

6 months

The successful candidate will hold active SC clearance, or be eligible and willing to undergo the relevant checks

You'll have hands-on experience of new technologies, in particular Hadoop and its surrounding ecosystem. You'll understand the pros and cons of these technologies and have experience on various implementations surrounding them, or of similar tools in this technical space.



Key Responsibilities:



* Ensure scalability, reliability, security and efficiency of systems.
* Contribute to the design and build large-scale distributed big data and ETL systems.
* Build automation to prevent problem recurrence.
* Working with other engineering teams to accelerate software delivery.
* Mentoring other team members.
* Help defining processes and standards, following Open Source best practices.
* Help shaping our department's culture.

Things we seek:



* Hands-on, highly technical with strong experience in systems engineering.
* In-depth understanding of Linux.
* Experience with AWS.
* Has experience in one or more of these programming languages: SQL, Java, Scala, Ruby, Python, Go or Shell.
* Networking experience and understanding of network protocols (e.g. TCP/IP, UDP, ICMP, etc) OSI layers, DNS, VPN and load balancing.
* Has proven experience working with Infrastructure as Code, configuration management systems and CI/CD.
* Has proven experience working with Hadoop/Yarn and associated ecosystems (Kafka, Samza, Spark, Flink, Ambari, Ranger, Hive, PIG).
* Has operational experience working with large, highly available, highly transactional systems on private and public clouds.
* Understands how to analyze and present operational metrics and use that data to optimize systems.
* Can work independently and take initiative.

Desirable:



* Experience with Azure.
* Configuration management using Ansible
* CI build pipelines with Jenkins, Drone, Artifactory, SonarQube
* Terraform, Consul and Vault
* Elasticsearch, Postgres, MySQL, Redis, MongoDB, Neo4j
* Monitoring with Sysdig, Prometheus and ELK
* Knowledge of HIDS, NIDS and security hardening.
* NGINX, HAProxy
* Docker and Kubernetes.

Suitable candidates should submit their CV in the first instance