Role overview

Cohere is an enterprise AI company building foundation models and products for real-world business applications. The internal infrastructure team is seeking a Senior Software Engineer to design and operate world-class systems that train, evaluate, and serve these foundational models. In this role, you will build GPU and TPU superclusters across multiple cloud providers while working closely with AI researchers to support their most demanding workloads.

Key responsibilities

Deploy and manage Kubernetes-based GPU and TPU superclusters across multiple clouds, maintaining high throughput and low-latency performance for AI workloads.
Collaborate with cloud providers to optimize infrastructure for cost efficiency, reliability, and performance using technologies such as RDMA, NCCL, and high-speed interconnects.
Identify and resolve infrastructure bottlenecks, performance degradation, and system failures proactively to minimize disruption to AI and ML workflows.
Design self-service tools and workflows that enable researchers to monitor, debug, and optimize their training jobs independently.
Work with AI researchers to understand emerging infrastructure needs such as JAX, PyTorch, and distributed training, then translate those needs into scalable solutions.
Advocate for observability, automation, and infrastructure-as-code practices across the organization to ensure maintainable and resilient systems.
Mentor team members through code reviews, documentation, and cross-team collaboration to foster engineering excellence and knowledge sharing.

Requirements

Deep expertise with GPU and TPU clusters, distributed training frameworks including JAX, PyTorch, and TensorFlow, and high-performance computing environments.
Proven ability to deploy, manage, and troubleshoot Kubernetes clusters at scale for AI workloads.
Strong programming skills in Python for ML tooling and Go for systems engineering, with a preference for leveraging open-source solutions.
Familiarity with Linux internals, RDMA networking, and performance optimization techniques for ML workloads.
Experience working directly with AI researchers or ML engineers to solve infrastructure challenges.
Self-directed approach to identifying bottlenecks, proposing solutions, and driving impact in a fast-paced environment.

What makes this role worth considering

You will work at the forefront of AI infrastructure, directly supporting the development of industry-leading models. Cohere maintains offices across Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul, with remote-friendly arrangements and co-working benefits for those not near an office. The role includes a 24×7 on-call rotation with compensation for on-call participation, along with comprehensive benefits including six weeks of paid vacation, full health and dental coverage, RRSP matching, 100% parental leave top-up for up to six months, and annual enrichment budgets for professional development and wellness.