We’re hiring.

Location

Pittsburgh, PA

Education

Pursuing a bachelor’s, master’s, or PhD

Team

Engineering

About Martingale

Martingale tells teams the moment the AI model behind their API changes behavior, before their users notice. We are building a drop-in proxy for LLM providers that forwards every request unchanged while recording metadata about each call, runs hidden canary prompts on a schedule, applies statistical tests that separate real behavior change from noise, and alerts through Slack or PagerDuty with an incident timeline. We also run MartingaleBench, a public drift tracker, and publish martingale-check, a command-line tool that fingerprints any OpenAI-compatible endpoint. We are a small team in Pittsburgh, PA.

About the role

You will own real pieces of the production path: the proxy that sits in the request path under a strict latency budget, the provider adapters, the ingest worker that lands events in Postgres, the canary scheduler and runner, and the incident manager that turns detections into alerts. The codebase is TypeScript in strict mode, tested end to end against real Postgres and Redis. You will ship code that customers’ traffic flows through, so correctness, failure handling, and data minimization matter as much as speed.

What you will do

• Build and maintain provider adapters for OpenAI Chat Completions and Anthropic Messages, including streaming, checked against each provider’s current API reference.
• Keep the proxy fast and safe: it never waits on Martingale’s own storage, writes telemetry asynchronously, and must keep forwarding traffic when our Postgres or Redis is down.
• Implement scheduled canary runs, including multi-step tool-using agentic workflows, and the Redis Streams ingest path that stores their results.
• Build incident management and alert delivery to Slack and PagerDuty with idempotent behavior under retries, restarts, and duplicate events.
• Write tests that name the edge case they cover, run them against real services with Testcontainers, and keep CI green, including the proxy latency benchmark.
• Contribute to the Python and TypeScript SDKs, the Next.js dashboard, and the REST API.
• Review teammates’ pull requests, write short design notes before larger changes, and document operational behavior in the runbook.

Requirements

• Currently pursuing a bachelor’s, master’s, or PhD in computer science, software engineering, or a related field.
• Strong C++, Python, and JavaScript or TypeScript, from production work or substantial projects.
• Fluent with agentic coding tools such as Claude Code, Codex, or Cursor, and in the habit of reviewing and testing what they produce.
• Able to work from Pittsburgh, PA.

Nice to have

• Experience building HTTP services: streaming responses, timeouts, retries, and idempotency.
• SQL with Postgres and a queue or stream such as Redis Streams, BullMQ, Kafka, or SQS.
• Have integrated an LLM provider API, such as OpenAI, Anthropic, or an OpenAI-compatible host.
• Docker, Git, and CI such as GitHub Actions, with tests written alongside code.
• Latency-sensitive proxy or gateway work, such as Envoy, nginx, Cloudflare Workers, or an API gateway.
• Experience with Hono, Drizzle, Zod, Vitest, Turborepo, Next.js, or Tailwind.
• Observability tooling such as OpenTelemetry, Prometheus, or Grafana.
• Open-source contributions or a public project we can read.

How to apply

Choose Apply, pick Software Engineer, and upload your resume as a PDF or DOCX. A link to your GitHub or a project you are proud of helps. We read every application and reply to everyone we interview.

Location

Pittsburgh, PA

Education

Pursuing a bachelor’s, master’s, or PhD

Team

Research

About Martingale

Martingale tells teams the moment the AI model behind their API changes behavior, before their users notice. We are building a drop-in proxy for LLM providers that records metadata about each call, runs hidden canary prompts on a schedule, applies statistical tests that separate real behavior change from noise, and alerts through Slack or PagerDuty with an incident timeline. We also run MartingaleBench, a public drift tracker, and publish martingale-check, a command-line tool that fingerprints any OpenAI-compatible endpoint. We are a small team in Pittsburgh, PA.

About the role

Martingale’s detection claims are only as good as the probes and statistics behind them. You will design the behavioral probes that fingerprint a model, choose and validate the tests that decide whether a change is real, and build the benchmarks and calibration pipelines that show, on held-out data with predeclared thresholds, that a detector catches silent model substitutions and quality downgrades without crying wolf. Your results go straight into the product and into our public methodology.

What you will do

• Design behavioral probes across code execution, reasoning, tool use, formatting, and refusal behavior that separate models and configurations, and verify them on development data before freezing a holdout.
• Evaluate and tune drift detectors built on Fisher’s exact test, Mann-Whitney U, chi-square, change-point detection, and Benjamini-Hochberg false-discovery control, measuring detection delay, false-positive rate, and sensitivity to partial and gradual degradation.
• Build reproducible experiments: synthetic degradation labs, budgeted real-model collection with checkpointed resume and independent controls, and paired comparisons with saved evidence.
• Extend agentic canaries with new tasks, scorers, and series such as steps-to-completion that catch a workflow quietly taking more tool calls to do the same job.
• Maintain MartingaleBench: the probe suites, scorers and hourly runs.
• Write up results honestly in the research docs, stating what each result validates and what it does not.

Requirements

• Currently pursuing a bachelor’s, master’s, or PhD in computer science, statistics, machine learning, mathematics, or a related field.
• Solid grounding in applied statistics: hypothesis testing, nonparametric tests, multiple-comparison correction, confidence intervals, power and sample-size reasoning, and the pitfalls of sequential testing.
• Hands-on experience evaluating LLMs: building eval sets, writing scorers, running models through provider APIs, and reasoning about variance across sampling settings and model versions.
• Strong Python with NumPy, SciPy, and pandas, plus working TypeScript or JavaScript, since the production detectors ship in TypeScript with SciPy-verified fixtures.
• Experimental discipline: development and holdout separation, predeclared thresholds, independent controls, and scripts anyone on the team can rerun.
• Clear technical writing that states assumptions and limitations.
• Able to work from Pittsburgh, PA.

Nice to have

• Research experience or publications in model evaluation, calibration, anomaly or change-point detection, or distribution shift.
• Familiarity with sequential and online testing, such as always-valid p-values or CUSUM.
• Experience with tool-calling and agentic workflows, or with LLM-as-judge evaluation and its failure modes.
• Experience running model experiments under an API spend budget.
• Open-source contributions, a public notebook, or a paper we can read.

How to apply

Choose Apply, pick Research Engineer, and upload your resume as a PDF or DOCX. A link to a write-up, notebook, or paper you are proud of helps. We read every application and reply to everyone we interview.