layiq

/la·yiq/adjective

worthy; deserving; fitting; suitable.

A role, opportunity, or path that merits attention, time, and pursuit.

LAYIQ - Sr. Observability Systems Engineer at Backblaze, Remote

Technical Leadership
Set technical direction and standards for observability across the organization, including instrumentation conventions, alerting philosophy, and dashboard design patterns.
Mentor engineers on the Observability team and act as a technical resource for SRE, Storage Engineering, and Network Engineering on observability practice.
Lead design reviews for observability work and represent the team in cross-functional architecture discussions.
Evaluate build-versus-buy decisions for observability and AIOps tooling, including data sovereignty and compliance considerations.
Produce and maintain design documentation, runbooks, and architectural decision records that make the platform operable and extensible by future contributors.
Observability Platform
Own and evolve our metrics pipeline, including VictoriaMetrics, Prometheus, and the Consul-based service discovery layer that keeps scrape targets current as infrastructure scales.
Own and expand our logging stack (OpenSearch, Logstash, and Filebeat) across all Backblaze data centers.
Architect and expand distributed tracing with Jaeger and OpenTelemetry, extending coverage beyond the current application instrumentation to new services.
Define and maintain Grafana dashboard standards for infrastructure, storage, and application performance visibility.
Drive expansion of internal application telemetry in partnership with application teams to improve visibility.
Define capacity and retention strategy for the telemetry stack as the fleet grows.
Infrastructure & Automation
Own infrastructure-as-code for observability stack deployments across multiple Kubernetes clusters.
Write and review automated playbooks for configuration management and automation across Debian-based bare metal servers.
Partner with the SRE team on Consul-based service discovery for VictoriaMetrics scrape target automation.
Lead Ceph storage observability, including OSD health, performance metrics, and integration with alerting.
Collaborate with the Network Engineering team on BGP, routing, and network-level observability.
Required Qualifications
8+ years in a systems engineering, SRE, DevOps, or observability-focused role, including demonstrated technical leadership on a platform or infrastructure team.
Deep hands-on experience with Prometheus, VictoriaMetrics, or equivalent metrics platforms, including PromQL, Alertmanager, and operating these systems at scale.
Experience designing and operating log aggregation pipelines such as Elasticsearch/OpenSearch, Logstash, Filebeat, or similar.
Working knowledge of distributed tracing (Jaeger, OpenTelemetry, or equivalent), including instrumentation strategy, not just operation.
Proficiency in Python and/or Go for automation and tooling.
Experience with Kubernetes, including deploying workloads and understanding pod networking and storage.
Experience with infrastructure-as-code tooling and configuration management.
Strong understanding of Linux internals, with Debian/Ubuntu preferred.
Demonstrated experience mentoring engineers and raising the technical bar on a team.
Excellent written and verbal communication skills. You'll work across engineering, SRE, security, and leadership, and you'll be expected to write design documents that drive decisions.

Sr. Observability Systems Engineer

Employer job description

Key Responsibilities

Preferred Qualifications

What You'll Work On

Pay Transparency

About this listing

Listing check

Before you apply

Similar jobs

Similar jobs