Role overview

Cohere is a security-focused enterprise AI company developing foundation models and products for real-world business applications. As a Staff Forward Deployed Engineer focused on Inference, you will serve as the principal technical bridge between Cohere’s core engineering teams and strategic enterprise customers, architecting and optimizing AI model deployments within complex, customer-specific production environments.

Key responsibilities

Work embedded with customer engineering teams to design the technical integration between Cohere’s AI models and customers’ live production systems and infrastructure.
Architect advanced model serving topologies including multi-tenant systems, intelligent model routing, and cross-datacenter failover strategies for multi-cluster deployments.
Optimize private and localized AI deployments by balancing infrastructure constraints such as Kubernetes orchestration with model performance requirements.
Conduct comprehensive benchmarking and profiling of customer workloads to identify and resolve performance issues through metrics analysis, dashboards, and distributed logging.
Support hardware validation efforts by collaborating with clients to test and deploy Cohere models on next-generation accelerators including various GPU, TPU, and custom ASIC platforms.
Translate operational challenges identified in the field into platform improvements and deliver technical feedback to core product teams.
Establish production-ready deployment patterns for Day-1 reliability, helping customers define service level objectives and observability frameworks.

Requirements

Minimum 8+ years of professional engineering experience managing production infrastructure, designing distributed systems at scale, or deploying enterprise software in complex environments.
Deep hands-on expertise with production Kubernetes including multi-cluster orchestration and hybrid or multi-cloud deployments across GCP, Azure, AWS, and OCI.
Strong technical knowledge of large language model architecture and inference behavior, particularly how attention mechanisms, compute-bound prefill phases, and memory-bound decode phases affect live performance.
Practical experience with modern inference engines and optimization frameworks such as vLLM, TensorRT-LLM, lmd, or Dynamo.
Familiarity with distributed serving techniques including prefill/decode disaggregated serving, distributed KV caching, and multi-GPU workload management.
Demonstrated ability to lead customer-facing technical discussions, scope ambiguous problems, and guide senior technical stakeholders through complex decisions.
Strong programming skills in Python, Go, or C++, with comfort working across backend data pipelines and infrastructure-level optimization.

What makes this role worth considering

You will work at the intersection of frontier AI research and real enterprise deployment challenges, directly influencing how next-generation models are integrated into production systems. Cohere maintains a global presence with offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul, offering flexibility to work remotely or from an office location with comprehensive perks including six weeks of paid vacation, full health and dental coverage, parental leave support, and education stipends.