Automation Engineer

Location:
City of London
Job Type:
Contract
Industry:
Cloud & Infrastructure
Job reference:
BBBH379781_1726737286
Posted:
10 months ago

Automation Engineer

Remote

6 months

Inside ir35

Role description

We are looking for an experienced Automation Engineer with a strong focus on application/platform profiling and performance optimization in an enterprise environment. The ideal candidate will have extensive knowledge of automation tools, especially Chef InSpec, and be proficient in profiling various types of applications, including those based on Hadoop. Additionally, they will need a deep understanding of monitoring tools such as Grafana and Prometheus, particularly in working with Prometheus data formats and creating performance dashboards.

The candidate will be responsible for automating application performance profiling, optimizing configurations, and setting up comprehensive monitoring solutions. Experience with profiling distributed applications like Hadoop will be essential. Furthermore, the engineer will collaborate closely with the operations, development, and DevOps teams to ensure that automated processes are in place for effective performance monitoring and compliance testing.

Key Responsibilities:

  • Application Profiling:
    • Perform detailed profiling of various applications, including distributed systems like Hadoop, identifying performance bottlenecks and optimization opportunities.
  • Automation Implementation:
    • Develop and maintain automation scripts and configurations, particularly using Chef InSpec, to automate application profiling, compliance checks, and infrastructure audits.
  • Performance Monitoring:
    • Configure and manage monitoring systems, particularly Grafana and Prometheus, to track application performance and generate real-time insights.
  • Hadoop Profiling:
    • Utilize profiling tools and techniques specific to Hadoop to analyze job performance, resource utilization, and system health.
  • Data Format Expertise:
    • Work with Prometheus data formats to build efficient monitoring and alerting solutions integrated with Grafana dashboards.
  • Automation for PerformanceOptimization:
    • Create and manage automation processes for continuous application performance tuning, resource allocation, and compliance validation.
  • System Integration:
    • Collaborate with other teams to integrate application profiling and monitoring solutions into broader enterprise architectures, ensuring automation processes are in place for scalability and reliability.

Skill Set:

Technical Skills:

  • Automation Tools:
    • Proficiency with Chef InSpec for automation and compliance testing.
    • Strong scripting skills in Python, Bash, or Ruby for automation tasks.
    • Experience automating infrastructure tasks, especially around application performance.
  • Application Profiling:
    • Expertise in application performance profiling tools (e.g., Apache JProfiler, YourKit, VisualVM) to analyze resource consumption, thread behavior, and response times.
    • Hands-on experience with profiling Hadoop applications, understanding resource allocation, job execution, and data flow optimization within distributed environments.
  • Monitoring & Logging Tools:
    • Grafana for visualizing performance metrics and setting up application monitoring dashboards.
    • Prometheus for collecting, querying, and alerting on performance data.
    • Understanding of Prometheus data formats and the ability to set up custom metrics for application monitoring.
  • DevOps & Automation Processes:
    • Experience with CI/CD pipelines using Jenkins or similar tools for automating the deployment and profiling of applications.
    • Familiarity with infrastructure as code and automation methodologies.
  • Performance Monitoring &Optimization:
    • Experience setting up performance dashboards and monitoring solutions.
    • Strong understanding of resource optimization, bottleneck analysis, and system tuning for enterprise-scale applications.
  • Hadoop Ecosystem:
    • Strong understanding of the Hadoop ecosystem (HDFS, MapReduce, YARN, Hive) and the ability to profile and optimize resource usage within distributed systems.
    • Familiarity with Apache Spark and Hadoop performance tuning is a plus.

Tools & Technologies:

Automation & Profiling:

  • Chef InSpec (for automation and compliance testing)
  • Scripting: Python, Bash, Ruby
  • Application Profiling Tools: Apache JProfiler, YourKit, VisualVM or equivalent tools for Java-based applications
  • Hadoop-specific tools: Hadoop Profiler, Ganglia, Ambari Metrics

Monitoring & Data Visualization:

  • Grafana (for setting up performance dashboards and real-time monitoring)
  • Prometheus (for collecting and querying performance data)
  • Prometheus data formats (for custom metrics creation)

Application & System Monitoring:

  • Hadoop monitoring (using tools like Ambari, Ganglia)
  • Elasticsearch, Loki (for logs management and application performance insights)

DevOps Tools:

  • Jenkins (for CI/CD pipeline automation)
  • Git (for version control and automation)

Qualifications:

  • Experience: 5+ years in automation, application profiling, or DevOps roles, with specific experience working with Hadoop and application performance monitoring.
  • Certifications(if any):
    • Chef Certified Engineer or equivalent certification in automation tools is preferred.
    • Hadoop Certifications (Cloudera Certified Administrator or Developer) are a bonus.

Education: Bachelor's degree in Computer Science, Information Systems, or a related field, or equivalent practical experience

Back job search
Back to Search Results