Big Data Build Engineer

Job Type:
Business Transformation
Job reference:
26 days ago

Big Data Build Engineer

Up to £1090 p/d (Umbrella)

Remote - Will be some AdHoc meetings in Sheffield

Inside IR35

6 Months + Possible Extensions


  • Should have deep understanding & technical knowledge of HDP / CDP products offerings and eco system tools across the technology stack
  • Solid background of HDP/CDP platform Installation, administration and maintenance
  • Good experience in performing large scale and complex upgrades from HDP (2.6.x) to CDP (7.1.x) platforms (on premise infrastructure)
  • Experience in doing product upgrades of the core big data platform, cluster expansion, setting up High availability for core services etc.
  • Should possess good Linux administration skills
  • Comfortable working with the shell, understanding scripting, integrated authentication stacks (AD / Linux integration, Krb and ldap based directory services, SSO identity services), file systems, security, secure shell, permissions, process management, regular maintenance tasks.
  • Setting up disaster recovery solutions for clusters using platform native tools and custom code depending on the requirements
  • Should possess good understanding of various enterprise security practices and solutions such as LDAP, Kerberos, KMS, Ranger & Knox
  • Shell / Python scripting / automation knowledge will be an added advantage
  • Role relevant qualifications, i.e. HWX/Cloudera Certified Platform Administrator is desirable but not essential

Principal responsibilities

  • Perform CDP 7.1.x installation
  • Migrate Data and content from HDP to CDP platform
  • Perform in place major / minor upgrades on HDP/CDP clusters
  • Troubleshooting of CDP issues and configuration changes
  • Performance Tuning of the clusters
  • Cluster expansions and HA / DR setups
  • Setup Alerting and health checks
  • Automation of procedural tasks
  • Research and adopt new tooling on CDP eco system
  • Provide level 3 support for the BAU team

Experience required

  • 10 + Years of overall IT experience
  • 3+ years of relevant Big Data Platform engineering experience

If you feel like your skills match the requirement for this role, please submit your CV and we shall be in touch.

Back to Search Results