RepoAlerts

Zguoxu/tiny-transformer-from-scratch

Resource

Can I trust it?

No red flags in our checks.

  • Stars from real accounts No data

    We haven't checked its stars for fakes yet.

  • Active OK

    Last change 14 days ago.

  • Licence OK

    MIT: you can use it for free, also in a commercial product. Keep the licence notice.

Quick checks from public data. They help, but they are no guarantee.

What it is

A small Transformer implemented from scratch in PyTorch without nn.Transformer or nn.MultiheadAttention, covering embeddings, positional encoding, multi-head attention, encoder, decoder, training and greedy generation. It includes an English-to-Chinese translation demo trained…

Who it's for

Learners with basic PyTorch knowledge who want to study the computations inside a Transformer for machine translation

How to use it

Install

python -m pip install -r requirements.txt

From its README. Check it there before running it.

Ask your AI to set it up

Paste this into Claude, ChatGPT or your coding assistant.

Help me install and set up Zguoxu/tiny-transformer-from-scratch in my project. Follow its README: https://github.com/Zguoxu/tiny-transformer-from-scratch. Explain each step and check it is safe before running anything.

Star curve

103From 0 to 103 stars between 2 Nov 2026 and 13 Nov 2026.

Get alerts for repos like this

We'll email you when something in Models & inference starts rising. Free, once a week, only what matches.

What arrives: one email on Mondays with the repos that match, never more. See last week's. Privacy

For maintainers

Add this badge to your README. It shows the repo's place on the radar, updates every hour and links back to this page.

[![Zguoxu/tiny-transformer-from-scratch on RepoAlerts](https://repoalerts.com/badge/Zguoxu/tiny-transformer-from-scratch.svg)](https://repoalerts.com/repo/Zguoxu/tiny-transformer-from-scratch?ref=badge)
Share on X

  • aa1143/NamiGPT

    A Chinese-language step-by-step tutorial that builds GPT-2 from scratch in PyTorch without using existing Transformer modules. It walks through tokenization, embeddings, attention, and blocks using a

    ResourceModels & inferencePython · 236 stars · +27 this week · 20 days old

  • prabeakarve/microgpt

    A dependency-free GPT implementation in pure NumPy, with a byte-level tokenizer, multi-head causal self-attention, hand-written backward pass, AdamW training with warmup and cosine schedule, and top-k

    LibraryModels & inferencePython · 51 stars · 11 days old

  • Interactive-Active-AI/NextTransformer

    A research framework and benchmark for comparing Transformer architectures under a fixed training budget. It trains recurrent-depth, sparse and linear attention, residual-stream variants and hybrids o

    LibraryModels & inferencePython · 427 stars · +27 this week · 16 days old

  • YeonwooSung/llm-architecture-from-scratch

    A collection of from-scratch PyTorch implementations of modern LLM architectures, including GPT-2 XL, Llama variants, DeepSeek V3, Qwen3, and Gemma 3. A comparison table lists each model's normalizati

    Models & inferencePython · 857 stars · 62 days old

  • SheepSeb/tinyworldformer

    A Rust project that runs two small neural networks on a Raspberry Pi Pico 2 W: a looped transformer world model that generates game frames and rewards, and an MLP actor that plays inside the generated

    AppModels & inferenceRust · 372 stars · +7 this week · 25 days old · HN 272 pts

  • datawhalechina/zero-to-sglang

    A free, bilingual (English and Chinese) tutorial course on LLM inference. It covers inference concepts such as KV Cache, prefill and decode, then guides readers through building a mini-sglang step by

    ResourceDev infrastructurePython · 2,143 stars · +402 this week · 82 days old