RepoAlerts

samir-sarwar/samir-vllm

Resource

Can I trust it?

Be careful with this one.

  • Stars from real accounts No data

    We haven't checked its stars for fakes yet.

  • Active OK

    Last change 12 days ago.

  • Licence Careful

    No licence: legally you can't reuse the code, unless the README says otherwise.

Quick checks from public data. They help, but they are no guarantee.

What it is

A learning-oriented LLM inference engine written in C++17 and CUDA that runs Llama 3.2 1B Instruct. It loads safetensors weights to the GPU, implements tokenizer, prefill and decode with paged KV cache, and batched greedy generation, with accompanying video tutorials and study…

Who it's for

Developers and students who want to understand LLM inference internals by reading a small C++/CUDA implementation

How to use it

Ask your AI to set it up

Paste this into Claude, ChatGPT or your coding assistant.

Help me install and set up samir-sarwar/samir-vllm in my project. Follow its README: https://github.com/samir-sarwar/samir-vllm. Explain each step and check it is safe before running anything.

Read the README

Star curve

223From 0 to 223 stars between 6 Oct 2026 and 16 Nov 2026.

On the radar

10 Nov 2026: #48
Best place #48, on 10 Nov 2026. In the top 100 on 1 of the last 30 days.

Its place among the radar's top 100, day by day, with #1 at the top. Gaps are days it was out of the top 100.

Records

  • Best place on RepoAlerts: #48, on 10 Nov 2026
  • On the Hacker News front page on 9 Nov 2026

Alternatives

Similar projects named in its profile. Each link searches the directory.

On Hacker News

Get alerts for repos like this

We'll email you when something in Models & inference starts rising. Free, once a week, only what matches.

What arrives: one email on Mondays with the repos that match, never more. See last week's. Privacy

For maintainers

Add this badge to your README. It shows the repo's place on the radar, updates every hour and links back to this page.

[![samir-sarwar/samir-vllm on RepoAlerts](https://repoalerts.com/badge/samir-sarwar/samir-vllm.svg)](https://repoalerts.com/repo/samir-sarwar/samir-vllm?ref=badge)
Share on X

  • Veloresearch/kinetic

    A Rust inference engine that runs GGUF models, the llama.cpp format, on CPU and on NVIDIA, AMD and Intel GPUs from one binary. It includes speculative decoding and prompt-state reuse for faster genera

    CLI toolModels & inferenceRust · 90 stars · +89 this week · 6 days old

  • apple-aiml-research/ml-rsi-inference-bench

    A long-horizon benchmark for coding agents that improve the inference stack of large language models in vLLM or SGLang. Tasks cover supporting a model on day 0, optimizing it after launch, and repairi

    AI agentsPython · 166 stars · +26 this week · 31 days old

  • JakeATX/llamAmpere

    A fork of llama.cpp tuned for NVIDIA RTX 30-series Ampere GPUs, especially the RTX 3090 and 3090 Ti. It runs Qwen3.8 models, including Qwen3.8-27B with its built-in MTP draft head and a Flash-Next var

    AppModels & inferenceC++ · 383 stars · +83 this week · 70 days old

  • datawhalechina/zero-to-sglang

    A free, bilingual (English and Chinese) tutorial course on LLM inference. It covers inference concepts such as KV Cache, prefill and decode, then guides readers through building a mini-sglang step by

    ResourceDev infrastructurePython · 2,143 stars · +402 this week · 81 days old

  • manuelgscom/inside-llm

    A native binary that loads one or two GGUF models via llama.cpp and shows in the browser per-token probabilities, discarded alternatives, sampler effects, and the token where two models' outputs diver

    AppModels & inferenceTypeScript · 59 stars · +59 this week · 1 day old

  • Betelgeuse-AI/Betelgeuse

    A llama.cpp fork that runs mixture-of-experts models larger than GPU memory on an RTX 3060 12 GB. Dense weights stay on the GPU, while routed experts are paged from local NVMe into a bounded VRAM slot

    AppModels & inferenceC++ · 154 stars · +31 this week · 16 days old