Role overview

Cohere is an enterprise AI company building foundation models and end-to-end products for real-world business applications. As a Software Engineer on the ML Workload Platform team, you will own the infrastructure that trains, evaluates, and serves Cohere’s frontier models, working alongside AI researchers and ML engineers to ensure the platform scales reliably and performs effectively.

Key responsibilities

Own and operate the Kubernetes-based platform responsible for scheduling and running training, evaluation, and reinforcement-learning jobs, including deciding capacity allocation and managing failure scenarios.
Build networking and runtime isolation capabilities for ML workloads, including agent and sandbox environments that protect job execution.
Collaborate with the data infrastructure team on storage systems for training checkpoints and other artifacts required by jobs.
Debug production failures and performance issues, then implement lasting improvements to system operability and observability.
Develop infrastructure-as-code, automation, and monitoring systems that keep the platform maintainable as it grows.
Participate in a compensated 24×7 on-call rotation to support platform reliability.

Requirements

Production experience running a job platform, batch system, or Kubernetes controllers, with expertise in scheduling, queues, and multi-tenant fairness.
Demonstrated judgment with distributed systems, including understanding of retries, partial failure modes, and knowing when not to automate.
Background working on ML training workflows, reinforcement-learning sandboxes, container networking (CNI), or checkpoint storage at scale; direct GPU-cluster experience is valued.
Proficiency writing software in Python or Go, with a preference for extending mature open-source projects rather than rebuilding them.
Experience collaborating with researchers, ML engineers, or teams with different objectives than your own.
Track record of identifying bottlenecks, proposing solutions, and driving improvements in fast-moving environments.

What makes this role worth considering

Cohere operates as a remote-friendly organization with offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul, allowing flexibility in how and where you work. The company provides comprehensive benefits including six weeks of paid vacation, full health and dental coverage, parental leave top-up, education stipends, and regular access to company offsites and office-to-office travel budgets for distributed team members.