Article

How to Plan LLM Inference Capacity for a Shared Platform

Published: 2026-07-29 A recurring capacity-planning request for an internal LLM platform is: We have selected a model. Do we have enough capacity, and how many GPUs would we need for N users? The question is missing the information needed to answer it. The model and its serving configuration determine the minimum resources required to start one replica. They don't tell us how many requests that replica can handle while meeting the product's latency and reliability requirements. That depends on prompt and response lengths, arrival rate, concurrency, cache reuse, calls per user task, traffic bur

Andrey Krisanov · EN

Hidden Costs of LiteLLM's Health Checks

Published: 2026-08-15 We run LiteLLM as an internal gateway in front of two kinds of models: local vLLM deployments and paid external models. At the time of the incident, the proxy was running LiteLLM 1.82.x with four replicas, Redis for shared state, and background model health checks enabled. The relevant part of the configuration looked like this: g eneral_settings : b ackground_health_checks : true u se_shared_health_check : true h ealth_check_interval : 30 h ealth_check_concurrency : 4 The problem showed up in the external provider's usage view. Over roughly one to two weeks, a key reserv

Andrey Krisanov · EN

The shelf

Published: 2026-08-23 Updated: 2026-08-23

Andrey Krisanov · EN

About

Published: 2024-01-01 Updated: 2026-09-01 Hi, My name is Andrey Krisanov, and I'm a Staff Software Engineer at Severstal. I build AI infrastructure, ML systems, and distributed systems. A large part of my current work is production LLM serving. I live in Moscow with my wife and son. We're preparing for a major move and the next chapter of our lives. Outside work, I read, take long walks, spend time with my family, travel when I can, and try to stay in shape. My main technical interests are distributed systems, GPU infrastructure, Kubernetes, Go, developer tooling, and running reliable software

Andrey Krisanov · EN

Task Names in Swift Concurrency

Name Swift tasks for clearer debugging and profiling

Artem Novichkov · EN

Using Claude with Apple Foundation Models

Bringing frontier models to LanguageModelSession with model selection and BYOK

Artem Novichkov · EN

Building a custom DynamicProfileModifier in Foundation Models

Build a reusable economy mode modifier that switches a session on device, caps responses, and trims history

Artem Novichkov · EN

Using SwiftUI’s ContentBuilder with Non-View Types

Using SwiftUI’s result builder to assemble a type-safe deep-link router.

Artem Novichkov · EN

Preview Multiple SwiftUI View States with #Preview(arguments:)

Use Xcode 27 to render named SwiftUI states as independently selectable Canvas variants

Artem Novichkov · EN

Headless Xcode: From Prompt to Simulator with MCP

Use Xcode 27's headless MCP server and Apple's agent skills to build, preview, and verify an iOS app

Artem Novichkov · EN

SF Symbols from the Command Line and AI Agents

Search, verify, and export symbols with the sfsymbols CLI

Artem Novichkov · EN

Xcode Projects for the Command Line and Coding Agents

Edit Xcode projects from the terminal with xcodeproj, xcprojformatter, and the new JSON format

Artem Novichkov · EN

Workshop

Devising clocks, calendars, Unsettling shapes of time Boundary-laden Weighty items Atomized Totemic miseries and mercies

Arbourtrary RSS Feed · EN

Balcony

I have not had a winter to emerge from - Nothing for years and years and years Slow, slow, sudden, wild springtime Glorious as anyone says Bud-green, branching, blooming They do this every year? Stoic, stoics all. I'm in awe. I'll drop into the lateral bands soon It'll all surprise me again, soon They do this every year

Arbourtrary RSS Feed · EN

Lilac

Looping split of a rush (Crack a window) spilling apologetics Instructional essentials, ESSENTIALS I will utterly distract myself Utterly (momentous) fleeting Lavish confession

Arbourtrary RSS Feed · EN

Orbic

I trace a flightpath portion - Luminous bird, underbelly Saturated in sunset I recall my childhood hand Palming a flashlight in the dark, My own transparency Glowing orange through

Arbourtrary RSS Feed · EN

Frontier

Relief of being somewhere unknown Place of no pretense No face to put on Tension giving way Some final sigh dropping expectation Into the slowest, kindest aimless walk Eyes panning, absorbing Little joys

Arbourtrary RSS Feed · EN

Velocity

Ramble of work I believe very little in destination Not some grand conviction Just the evidence of experience Stuck in meander

Arbourtrary RSS Feed · EN

Mazarine

Being amongst Swaths and swaths of antiquity Is the deepest relief I am As ignorant, as nameless, As collective, as individuated, As proximate to slivers of knowledge As anyone here I can sit, anonymous awe, and breathe

Arbourtrary RSS Feed · EN

Masonry

I am feeling the passersby of information I am invoking cloudless blue sky beloved astronomer Patient for nightfall I am considering how alchemy Is a matter of sufficient energy How entropy is a matter of rearrangment How simple we lay it I am settling into a former self My lord how it relaxes me How much I need to be here Peaceful dwelling contour

Arbourtrary RSS Feed · EN

Intro to Amazon SQS

SQS is AWS's managed message queue service. Messages go in, consumers pull them out, and the queue handles everything in between. Here's what it is, when to use it, and how to implement the common patterns.

Andrew Gilliland · EN

Real-Time Image Optimization with Lambda on S3 Upload

Automatically optimize images the moment they're uploaded to S3 using Lambda functions triggered by S3 events. Perfect for user-generated content and dynamic workflows.

Andrew Gilliland · EN

Build an Asset CDN on AWS with CDK (S3 + CloudFront + Route53 + ACM)

Build a fast asset pipeline with a private S3 origin, CloudFront caching, custom DNS, and HTTPS certificates using AWS CDK.

Andrew Gilliland · EN

Authorizer Lambdas with Cognito and API Gateway

Cognito can tell you who a user is, but it cannot answer what they are allowed to do. An authorizer Lambda gives you one place to evaluate permissions before a request reaches your backend, with Cognito for identity and a separate permission source for authorization.

Andrew Gilliland · EN