Skip to content
#

assisted-generation

Star

Here is 1 public repository matching this topic...

JohnScheuer / speculative-decoding-real-v2

Star 0

Measures real speculative decoding speedup using the official HuggingFace assistant_model API across 4 model pairs and output lengths up to 512 tokens. Best result: distilgpt2->gpt2-medium achieves 1.747x speedup at 512 output tokens. Validates that cost_ratio and output_length are the key parameters.

python gpu transformers inference pytorch throughput llm speculative-decoding draft-model rtx-2070 assisted-generation
  • Updated Jul 13, 2026
  • Python

Improve this page

Add a description, image, and links to the assisted-generation topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the assisted-generation topic, visit your repo's landing page and select "manage topics."

Learn more