Software Engineer - Hosted Model Infrastructure

We are a software engineering team with expertise in enabling ML models in production. We deploy AI models to run in variety of environments: air-gapped government networks, forward-deployed defense environments, edge nodes, and enterprises with strict data sovereignty requirements. Our customers rely on us for frontier AI capabilities running on hardware they control, often with constrained GPU resources and limited direct access. Rising to that challenge and meeting those expectations is what Palantir's excels at.

Building high-performance model serving infrastructure that integrates with security models, hardware constraints, and different inference engines

Designing intelligent request handling including authentication, rate limiting, concurrency control, and audit logging for multi-tenant model access

Building and maintaining packaging and deployment pipelines enabling fast, secure, and reliable model rollouts across on-premises and air-gapped environments

Developing observability for production AI systems to enable easy service monitoring and fast incident triage and resolution

Debugging complex issues and performance problems throughout the stack, including open source inference engines, container runtimes, and GPU drivers, in environments you cannot always access directly

Designing and running testing and benchmarking infrastructure that validates model deployments across varying GPU hardware before they reach production

Working with product teams and customers to understand requirements, debug production issues, and deliver the models and capabilities they need

Integrating hosted model infrastructure with Palantir's deployment, configuration, and identity systems