Principle Site Reliability Engineer

Principle Site Reliability Engineer

Job Description:

Our Site Reliability Engineering group within Enterprise Infrastructure combines Operations Excellence with the Development Experience to deliver services at high scale, high availability with resilience by using automation and Infrastructure Code. We build reliability into our ecosystem by applying best practices in Resiliency Engineering, Automation, Observability & Chaos Testing.

The team comes from diverse technical backgrounds, and the responsibilities provide the opportunity for a variety of challenges. Ideal candidates will have a background in either software engineering or systems engineering with a desire to learn the other or previous experience as an SRE. We are looking for a Systems Thinking, SRE Engineer who has helped teams scale through production insights, operational automation, developer guidance, real-time metrics, automation, automation, automation...!

The Expertise You Have

• Bachelors degree or higher in a technology related field (e.g. Engineering, Computer Science, etc.) required, Masters degree a plus

• 5+ years of hands-on experience deploying and/or supporting highly distributed multi-tiered systems at scale.

• 1-2 years of experience in Cloud development (AWS) and migration skills; Experience with building and operating highly resilient platforms in AWS cloud environments

• 2-4 years of experience in software development with Python, NodeJS, or Java with a focus on SDLC and automation

• Hands-on experience with container orchestration, preferably with Kubernetes

Experience operating and implementing distributed & highly concurrent service-based

The Skills You Bring

• Ability to automate with various scripting languages (Python, Shell scripting, etc)

• Experience managing systems using infrastructure as code tools (IAM, ARM, Terraform, Chef, )

• Solid understanding of Cloud Computing and DevOps concepts including CI/CD pipelines

• Hands-on Kubernetes skills and knowledge.

• Hands on experience with one or more observability tools (Prometheus, Grafana, ELK/OpenSearch, OpenTelemetry, Datadog, etc)

• Experienced in Instrumentation with systems skills on building and operating, monitoring, logging, alerting services of distributed systems at scale

• Proven experience in maintaining scalability and resiliency of complex environment.

• Proven experience in implementing advanced observability practices and techniques at scale.

• Demonstrated ability to utilize modern monitoring tools (DataDog, Prometheus, Splunk, )

• Experienced in Instrumentation with systems skills on building and operating, monitoring, logging, alerting services of distributed systems at scale

• Ability to triage, execute root cause analysis, and be decisive under pressure

• Experience managing and interpreting large datasets using query languages and visualization tools

• Proficient communication skills with an ability to reach both technical and non-technical audience

• Ability to learn new software, method and practices and bringing them to our developers

• Ability to work with a variety of individuals and groups, both in person and virtually, in a constructive and collaborative manner and build and maintain effective relationships

• Demonstrated expertise in Production support, including incident management, root cause analysis, real-time monitoring, and high availability and reliability of critical systems in a fast-paced environment.

• Role requires weekday and weekend on call rotation.

The Value You Deliver

• Help define and execute a comprehensive reliability and observability strategy, ensuring that Fidelitys systems are always available when our customers need them.

• Bring together technical, procedural, and financial data to reduce toil and increase efficiency

• You will execute plans for technical standardization and process refinement within the engineering organization, especially for Site Reliability Engineers.

• Troubleshoot stack-wide engineering issues related to hardware, software, network, applications, and cloud service providers.

• Coach peer SREs and development teams on how to build highly available systems

Fidelitys hybrid working model blends the best of both onsite and offsite work experiences. Working onsite is important for our business strategy and our culture. We also value the benefits that working offsite offers associates. Most hybrid roles require associates to work onsite every other week (all business days, M-F) in a Fidelity office.