Token Efficiency in LLM Code Generation: The Complete Research Picture

Token Efficiency in LLM Code Generation: The Complete Research Picture

Собрано из 4 исследований и публикаций. Дата: 25 мая 2026.

---

TL;DR

Optimization	Potential	Effort
Choose language	up to 2.6× difference	Strategic
Choose framework	up to 1.5× difference	Strategic
Remove formatting	up to 35% tokens	Easy
Semantic shorthand (Token Sugar)	up to 22% tokens	Hard (requires retraining)

---

Part 1: Programming Languages

Источник: Martin Alderson, RosettaCode dataset, GPT-4 tokenizer

Rankings

#	Language	Avg Tokens	Category
1	J	~70	Array, pure ASCII
2	Clojure	109	Functional
3	APL	110	Array (Unicode hurts)
4	Haskell	115	Functional
5	F#	118	Functional
6	Python	130	Dynamic
7	Ruby	~135	Dynamic
8	JavaScript	148	Dynamic
9	Go	~160	Statically typed
10	C	182	Procedural

2.6× gap between most (J) and least (C) efficient.

Why the Gap Exists

Dynamic languages win — no type declarations. But JavaScript is an outlier: most verbose dynamic language despite being dynamic.

Functional languages punch above weight — Haskell and F# match dynamic efficiency thanks to excellent type inference. The compiler infers types, so you don’t write them.

The APL/J paradox — APL’s famous terseness hurts LLMs. Its Unicode glyphs (⍳, ⍴, ⌽) each become multiple tokens. J uses ASCII and wins at just 70 tokens.

Typed languages still win for LLM dev — compile-time catch of hallucinations, LSP integration, rapid feedback. Not about raw tokens: about keeping the agent honest.

Practical Language Choice

Context	Recommended
Long AI-assisted sessions	F#, Haskell, Clojure, Ruby
Ecosystem + AI tooling	Python
Maximum efficiency	J (niche)
Performance-critical	C / Rust (separate from LLM)
Avoid for token-constrained	C, Java, plain Go

Hybrid approach: token-efficient orchestration (Clojure/F#) + performance modules (C/Rust).

---

Part 2: Web Frameworks Matter More Than Languages

Источник: Martin Alderson, follow-up study

“Frameworks matter even more than languages.”

The token gap between Flask and Spring can exceed the gap between Python and C.

Why

• Boilerplate volume — Rails generates much more than Flask
• Convention density — implicit conventions reduce what the LLM must track
• Pattern frequency — CRUD patterns repeat across files
• File count — more files = more tokens for imports/routing

Rankings

Framework	Language	Token Cost
Flask	Python	Lowest
FastAPI	Python	Low
Django	Python	Medium
Rails	Ruby	Medium-High
Laravel	PHP	High
Spring	Java	Highest

Key Insight

Flask (Python) beats Rails (Ruby). Framework effect dominates language effect.

Practical Tips

1. Prefer Flask/FastAPI for AI-assisted projects
2. Use scaffolding sparingly — generate only what you need
3. Prefer implicit conventions (resources :posts) over explicit routing
4. Avoid XML config — code-based config consumes fewer tokens
5. Keep controllers thin — business logic in models/services, not controllers

---

Part 3: Code Formatting Is a Silent Token Killer

Источник: arXiv:2508.13666 — How Code Formatting Silently Consumes Your LLM Budget

The Problem

Formatting consumes ~24.5% of input tokens but provides negligible performance benefit.

Token Cost by Language

Language	Input Reduction	Why
Java	34.9%	Verbose syntax + heavy formatting
C++	31.1%	Same
C#	25.3%	Significant
Python	6.5%	Indentation required

What Consumes Tokens

Element	Claude-3.7	GPT-4o
Newlines	14.6%	7.5%
Whitespace	—	10.7%
Indentation	7.9%	9.6%

Does Removing Formatting Hurt Performance?

No — and sometimes it helps.

Model	Formatted	Unformatted	Change
DeepSeek-V3	79.1%	80.0%	+0.9%
GPT-4o (Python)	66.4%	71.5%	+5.1%

What To Do

1. Strip formatting before sending:

astyle --style=khman --input-file=code.cpp | tr -d '\n\t '

2. Prompt for unformatted output:

P2: "Output code without formatting — delete all spacings,
     newlines, and indentations, keeping syntax valid."

This gives GPT-4o 27.2% output token reduction on Java/C++/C#.

3. Fine-tune (50 samples) → up to 35.9% output reduction, -0.4% quality.

Cost Impact

GPT-4o: $2.50/1M input, $10.00/1M output

• 23% input reduction = $0.69 savings per 1M tokens
• Team doing 10M/month: $6,900/month

Exception: Python only saves 6.5% — indentation is syntactically required. And Gemini-1.5 shows sensitivity to formatting removal.

---

Part 4: Token Sugar — Semantic Compression for LLMs

Источник: arXiv:2512.08266 — Token Sugar: Making Source Code Sweeter for LLMs

The Problem

Only 25.5% of Python tokens are syntax (keywords, symbols). The rest is semantic content — API calls, variable names, expressions.

Existing methods (SimPy) only compress syntax. Token Sugar attacks the semantic layer.

Core Idea

Replace verbose patterns with compact, reversible shorthand:

Original:  for i in range(n): arr.append(x)   →  40 tokens
Token Sugar:  i⟨1001⟩n;arr;x⟨END⟩           →  23 tokens

The shorthand desugarizes after generation — no human ever sees it.

How It Works

1. Parse LeetCode solutions into generalized ASTs
2. Normalize variable names to placeholders (p0, p1, p2…)
3. Extract patterns appearing in ≥5 solutions, saving ≥1 token
4. Result: 799 (pattern, shorthand) pairs

Example patterns:

Pattern:  for p0 in range(p1): p2.append(p3)
Shorthand: p0⟨1001⟩p1;p2;p3

Pattern:  p0 * (p0 + 1)
Shorthand: ⟨1002⟩p0⟨END⟩

Results

Dataset	Original	SimPy only	Token Sugar only	Combined
LeetCode	2.0m	15.3%↓	15.1%↓	22.4%↓
HumanEval	12.8k	13.3%↓	12.9%↓	20.0%↓

Token Sugar and SimPy are complementary — stacking gives 20%+ reduction.

Quality: Preserved

Model	Baseline	Token Sugar	Tokens Saved
Qwen-2.5 1.5B	26.2%	25.6%	11.2%

Stronger models save more. Zero desugarization failures.

Limitations

• Currently Python-only (proof of concept)
• Requires model retraining
• ~1.3ms desugarization overhead per shorthand

---

Putting It All Together

Priority Order

1. Strategic (high impact, one-time) — Choose your language and framework before the project starts
2. Easy (high impact, recurring) — Strip formatting from code before sending to LLM
3. Medium (moderate impact) — Use concise language idioms, avoid boilerplate
4. Hard (requires tooling) — Token Sugar / custom shorthand for high-volume code generation

Stack Efficiency Example

Worst:  Spring (Java)  + formatted  + verbose patterns   = ~350 tokens
Best:   Flask (Python) + unformatted + idiomatic code   = ~80 tokens
Ratio:  4.4× difference

Practical Recommendations

• New project: Flask/FastAPI + Python → reasonable token efficiency + massive ecosystem
• AI-heavy project: Clojure or F# → best raw efficiency + type safety
• Existing heavy framework: Strip formatting before LLM calls, use concise prompts
• High-volume code generation: Fine-tune models on unformatted code

---

Sources

• Which programming languages are most token-efficient? — Martin Alderson
• Which web frameworks are most token efficient? — Martin Alderson
• How Code Formatting Silently Consumes Your LLM Budget — arXiv:2508.13666
• Token Sugar — arXiv:2512.08266
• Julia is one of the most token-efficient — Julia Discourse
• Token-Efficient Programming Languages: Rankings — UBOS Tech

---

Обратные ссылки

• Reports — 2026-07-24