Principal Backend Engineer - Grafana Alloy OpenTelemetry Collector (Remote, Canada EST)

This is a remote position and we're considering candidates in the Canadian Eastern timezone.

What is Grafana Cloud?

Grafana Cloud is our composable observability platform that integrates metrics, logs and traces with Grafana. It allows our customers to leverage the best open-source observability software – including Prometheus, Mimir, Loki, and Tempo – without the overhead of installing, maintaining, and scaling their own observability stack

The Observability department is focused on enabling developers to understand the health and performance of their applications and infrastructure in any environment by providing tools to instrument their code, ingest observability data into Grafana Cloud, and visualize and explore it.

Grafana Alloy is an open-source OpenTelemetry Collector distribution with built-in Prometheus pipelines and support for metrics, logs, traces, and profiles. The Alloy team collaborates with the upstream OpenTelemetry collector community on components we include in our distribution. We also cover use cases that are out-of-scope for OpenTelemetry, like the integration of Prometheus exporters.

As a company we are remote-first and global, we embrace people of different experiences and backgrounds to build diverse teams where every person brings a unique perspective to the software. We are looking for Engineers that are passionate about communicating with data and providing seamless experiences for our customers to join our growing team! Engineers at Grafana also have the opportunity to contribute to Open Source communities.

What will you be doing?

At Grafana Labs, ICs have a dedicated career path and do not have to become managers to progress in their career. Principal software engineer is a leadership role for ICs. You are expected to have director-level scope, but with technical depth.

• Support and lead the technical direction of the Alloy project, and drive business conversations on the data ingestion strategy at Grafana Labs.
• Be an active member of the OpenTelemetry collector SIG. Alloy shares a lot of components with the upstream OpenTelemetry collector project. We want to help make OpenTelemetry a success, and be active members of the OpenTelemetry collector SIG. You will lead this effort by example, and encourage others to do the same.
• Be familiar with the Prometheus ecosystem and help maintain the Prometheus exporters included in Alloy.
• Be visible. Teach others, promote best practices, share knowledge, establish the relation of Alloy within the Prometheus and the OpenTelemetry ecosystem. Both within Grafana Labs and at community events like KubeCon.
• Support and partner with GTM (go-to-market). Engage and lead conversations with prospects and customers, when product expertise is needed, involving solutions engineering, customer success, and professional services.
• Drive cross-department programs to ensure Alloy-ingested data meets the needs of our databases (Mimir, Loki, Tempo) and applications built on top of these databases.
• On-call. You participate in on-call rotations for our deployments of Alloy and drive operational improvements.

As we are remote-first and our engineering organization is largely remote, we provide guidance and meet regularly using video calls, so an independent attitude and good communication skills are a must.

What are we looking for in you?

• At Grafana Labs, we define senior roles via the scope of their business impact more than via their expertise or their specific activities. So the most important thing we are looking for in a principal engineer is the ability to take ownership, and drive both technical and business conversations on a director-level scope.
• You have a passion for observability and like to share your knowledge by writing documentation and blog posts.
• You love to engage with customers and help them out.
• You have excellent communication skills.
• You have relevant open source experience, ideally in OpenTelemetry or Prometheus.
• You are willing to become an active member of the OpenTelemetry community.
• You have a good understanding of production environments. Ideally you have been responsible for operating production infrastructure and services, and organizing on-call.
• You actively mentor other team members, identifying areas for focus and improvement.

Links:

In Canada, the Base compensation range for this role is CAD 188,000 - CAD 225,000. Actual compensation may vary based on level, experience, and skillset as assessed in the interview process. Benefits include equity, bonus (if applicable) and other benefits listed .

*Compensation ranges are country-specific. If you are applying for this role from a different location than listed above, your recruiter will discuss your specific market’s defined pay range & benefits at the beginning of the process.