manuelgscom/inside-llm
App
Can I trust it?
Worth a closer look before you use it.
Warnings Check
Very new: it was created yesterday. Wait a little before installing it.
Stars from real accounts No data
We haven't checked its stars for fakes yet.
Active OK
Last change today.
Licence OK
MIT: you can use it for free, also in a commercial product. Keep the licence notice.
Quick checks from public data. They help, but they are no guarantee.
What it is
A native binary that loads one or two GGUF models via llama.cpp and shows in the browser per-token probabilities, discarded alternatives, sampler effects, and the token where two models' outputs diverge. Runs locally with CUDA or Vulkan, with recorded sessions replayable in a…
Who it's for
ML engineers and researchers comparing GGUF quantizations or fine-tunes, and developers studying local LLM sampling behavior
How to use it
Install
./inside-llm-v1.0.0-linux-vulkan/inside-llmFrom its README. Check it there before running it.
Ready-made installers, when the project publishes them, are on its latest release.
Ask your AI to set it up
Paste this into Claude, ChatGPT or your coding assistant.
Help me install and set up manuelgscom/inside-llm in my project. Follow its README: https://github.com/manuelgscom/inside-llm. Explain each step and check it is safe before running anything.
Star curve
Alternatives
Similar projects named in its profile. Each link searches the directory.
Get alerts for repos like this
We'll email you when something in Models & inference starts rising. Free, once a week, only what matches.
What arrives: one email on Mondays with the repos that match, never more. See last week's. Privacy
For maintainers
Add this badge to your README. It shows the repo's place on the radar, updates every hour and links back to this page.
[](https://repoalerts.com/repo/manuelgscom/inside-llm?ref=badge)Similar repos
- JakeATX/llamAmpere
A fork of llama.cpp tuned for NVIDIA RTX 30-series Ampere GPUs, especially the RTX 3090 and 3090 Ti. It runs Qwen3.8 models, including Qwen3.8-27B with its built-in MTP draft head and a Flash-Next var
- Veloresearch/kinetic
A Rust inference engine that runs GGUF models, the llama.cpp format, on CPU and on NVIDIA, AMD and Intel GPUs from one binary. It includes speculative decoding and prompt-state reuse for faster genera
- huawei-csl/MACaQ
A model-agnostic quantization toolkit that quantizes text, image, video and audio models to low-bit formats, including 2/3/4-bit GGUF for llama.cpp and W4A4 checkpoints for vLLM, Diffusers and PyTorch
- samir-sarwar/samir-vllm
A learning-oriented LLM inference engine written in C++17 and CUDA that runs Llama 3.2 1B Instruct. It loads safetensors weights to the GPU, implements tokenizer, prefill and decode with paged KV cach
- cobanov/PS5LM
A PS5 homebrew app that runs llama.cpp on the console's GPU and serves GGUF models over OpenAI- and Anthropic-compatible HTTP APIs on the local network, with a TV interface and a decision-model endpoi
- orailnoor/cross-platform-llm-client
A Flutter chat client for Android and iOS that runs GGUF language models on-device via llama.cpp with Vulkan or Metal acceleration, and also connects to OpenAI, Anthropic, Google Gemini and Kimi cloud