Show HN: Warmctl – Fast cold starts for vLLM inference
Source: Show HN ·
GPU checkpoint/restore for vLLM: snapshot a warmed-up engine (compiled kernels, CUDA graphs, weights, 1-8 GPUs) into an OCI image and restore it in seconds instead of a multi-minute cold start. Sca...
Frequently Asked Questions
What is Show HN: Warmctl – Fast cold starts for vLLM inference?
Show HN: Warmctl – Fast cold starts for vLLM inference is a tool discovery curated from Show HN. GPU checkpoint/restore for vLLM: snapshot a warmed-up engine (compiled kernels, CUDA graphs, weights, 1-8 GPUs) into an OCI image and restore it in seconds instead of a…
How can I access or experience this?
You can experience this content directly on its original host (Show HN) by clicking the "Open Original" button.
Where does this content come from?
Curated from Show HN by the OnSuggest team to surface the most interesting and serendipitous content on the web.
More like this
All Developer tools discoveries →Show HN: Pstview – read-only PST/OST viewer
At my work I sometimes need to recover data from old outlook files or recover emails from backup tools. Opening these files in Classic Outlook is painful and can modify the file by accident, which may be undesired. So I came up with this solution. Maybe someone else can benefit from this. Comments URL: https://news.ycombinator.com/item?id=50584843 Points: 3 # Comments: 0
Show HN: Voxstead – Live captions and bilingual call assistance for macOS
Live captions, AI reply suggestions for bilingual calls, meeting notes, and voice dictation for macOS. On-device speech recognition in a native Swift app. - TriHerm/voxstead
Show HN: yavcpdf - A keyboard-driven, vibecoded PDF viewer
Hi HN, I wanted a PDF viewer I could drive entirely from the keyboard that could also fill in forms and sign documents. You might already be acquainted with PDF viewers like zathura and sioyek. This one is similar, except much more performant and not prone to crashes. You can also draw, annotate and fill & sign entirely via keyboard. It's based off of PdfCraft's (from the ArtCraft project) rendering engine. The config is configured in TOML and live-reloaded. Feel free to try and give feedback. Comments URL: https://news.ycombinator.com/item?id=50584936 Points: 1 # Comments: 0
Show HN: Find and serve fast LLM config for your GPU
I was having a hard time finding settings that can let me utilise the models to maxium throughput without quality compromise so I built rigtune which helps me find the settings for my hardware and model I'm using, will love your feedback and the results if you can share for this. Comments URL: https://news.ycombinator.com/item?id=50584965 Points: 2 # Comments: 0
Show HN: Name the World – An Atlas Quiz
I've been working on this country naming/geography game on and off over the past few months but most of it in Oct/Nov, using only free tier Claude (Sonnet 5 and 5.5) It started off as just wanting to make a fun little thing for myself that included features in this sort of game that I couldn't find on existing sites. As I went on with it adding more stuff and ideas from friends it also turned into sorta like proving to myself that Sonnet and the free tier restrictions were still good enough to work on fun little things like this. Happy to get any sort of feedback anyone is interested in giving
Show HN: AbcTab – Native Swift 6 macOS ⌘-Tab Switcher Without Screen Recording
AbcTab is a native macOS ⌘-Tab alternative and window switcher built with Swift 6 & AppKit — no Screen Recording permission required. Jump by English or Pinyin initial, expand multi-window & terminal tab lists, and toggle Ping-Pong pairs.