Senior DevOps Engineer - Big Data

  • Location

    London, England

  • Sector:


  • Job type:


  • Salary:

    £75000 - £83000 per annum + + Excellent Package / Pension

  • Contact:

    Matthew Costello

  • Contact email:

  • Job ref:


  • Published:

    over 1 year ago

  • Expiry date:


  • Client:


Senior DevOps Engineer, Microservices, Azure, Cloudbreak, Hadoop, Hortonworks

My leading client is looking for a Senior DevOps Engineer you will be responsible for the configuration and on-going management of the cloud PaaS/IaaS, working with Big Data platforms such as Hortonworks based on Hadoop clusters configured on cloud data centres such as those of Azure/AWS. You will ensure availability of the platform to services teams and their service users. You will also carry out continuous improvements to platforms, including system development, processes design and automation and diagnostics improvements. Assurance of platform security, privacy and resilience will be your greatest concern.

You will support and maintain the service infrastructure with appropriate security and privacy controls and meets users' needs, Business solutions comply with privacy and security requirements, Work in an agile way and collaborating with colleagues. You will have a strong background in Agile software engineering practices such as CI/CT/CD. You must have experience with architecture/engineering of cloud based distribution systems (such as Azure)

Technical skills

  • Experience of Microservices Architecture and APIs
  • Strong knowledge of Azure and its offerings and the Hadoop ecosystem using Hortonworks (preferred) or Cloudera
  • Cloudbreak, Hive, Ambari, Sqoop, Oozie, SPARK, Atlas, Ranger, HBase, HDFS, YARN and ELK
  • Hands on experience connecting cloud infrastructure to on-prem networks
  • VPNs, tunnelling, AD connectors, routing o Knowledge of network protocols such as TCP, UDP, HTTP/HTTPS, SSL/TLS, and API's
  • Authentication technologies. i.e. LDAP, OAuth, 2FA, SAML, and SSO via Kerberos
  • Bash, Python, Java or Ruby
  • Experienced with security controls o VPNs, encryption & key/certificate management, endpoint protection, virtual firewalls/ACLs/NSGs, setting up bastion nodes, etc
  • Background in distributed systems o databases, security, networking & load balancing, monitoring, scripting, automation
  • A deep understanding of distributed system design and dependency management
  • Operational expertise o troubleshooting skills, understanding of system capacity/bottlenecks, basics of memory, CPU, OS, storage, and networks
  • A strong grasp of protective monitoring tools, approach and implementation
  • Linux system administration (configuration, installs, automation, and monitoring)
  • Experience automating infrastructure and software delivery to production

If this role sounds of interest please send me cv for consideration.