Role overview

Cohere is an enterprise AI company that develops foundation models and products for real-world business applications. The Internal Infrastructure team builds the systems used to train, evaluate, and serve these models across production environments. As a Software Engineer focused on GPU Infrastructure, you will design and operate the compute clusters that power Cohere’s platform.

Key responsibilities

Deploy and operate GPU Kubernetes clusters across multiple cloud providers, including bringing new hardware generations and providers into service.
Partner with cloud providers and hardware vendors to optimize reliability, performance, and cost, with particular attention to RDMA and high-speed interconnect technologies.
Build automated systems to detect and remediate faults in nodes, GPUs, and fabric infrastructure so that most issues resolve without manual intervention.
Monitor fleet utilization and costs while reducing the time required to bring faulty nodes back into production service.
Investigate and resolve production failures and performance issues, implementing improvements that make the infrastructure easier to operate.
Participate in a compensated 24×7 on-call rotation to support production systems.

Requirements

Demonstrated experience running GPU Kubernetes clusters at scale with thousands of GPUs for machine learning or high-performance computing workloads.
Strong debugging skills across Linux systems, networking, and performance optimization.
Proficiency writing software in Python or Go, with a preference for extending existing open-source projects rather than rebuilding solutions.
Experience working with researchers, machine learning engineers, or cross-functional teams with differing objectives.
Track record of identifying bottlenecks, proposing solutions, and executing fixes in fast-moving environments.
Familiarity with RDMA, NCCL, or accelerator health monitoring is beneficial but not required.

What makes this role worth considering

You will work directly with AI researchers and ML engineers on infrastructure that directly impacts model capabilities and customer value. Cohere is a global organization with offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul, offering remote-friendly arrangements with co-working benefits and regular access to company hubs.

The company provides comprehensive benefits including six weeks of paid vacation, full health and dental coverage with dedicated mental health support, parental leave top-ups, education stipends, and annual enrichment budgets for wellness and professional development.