[Feature Request] Non-English users (Korean/Japanese/CJK) face structural disadvantage due to tokenization inefficiency — Usage limits should be adjusted accordingly

Preflight Checklist

• [x] I have searched existing requests and this feature hasn't been requested yet
• [x] This is a single feature request (not multiple features)

Problem Statement

Korean, Japanese, and other CJK language users are structurally disadvantaged compared to English users when using Claude Code. The root cause is tokenization inefficiency: the same meaning expressed in Korean requires approximately 1.5x to 3x more tokens than English. Since usage limits (5-hour windows and weekly caps) are measured in tokens, non-English users effectively receive significantly less service for the same subscription price.

This is not a minor inconvenience — it is a systemic fairness issue that affects millions of potential users across Asia and other non-English-speaking regions.

📊 The Problem: Tokenization Inefficiency by Language

Claude's tokenizer was primarily trained on English-dominant datasets. As a result, English text is encoded far more efficiently than Korean, Japanese, Arabic, or other non-Latin script languages.

Real-world token comparison (approximate):

| Phrase (same meaning) | Language | Token count |
|---|---|---|
| "Refactor this function" | English | ~4 tokens |
| "이 함수를 리팩토링해줘" | Korean | ~10–14 tokens |
| "Fix the bug in this file" | English | ~6 tokens |
| "이 파일의 버그를 수정해줘" | Korean | ~14–18 tokens |
| "Explain the architecture" | English | ~4 tokens |
| "아키텍처를 설명해줘" | Korean | ~10–12 tokens |

Result: Korean users consume roughly 2x–3x more tokens to communicate the same information.

Why this happens:

Claude's BPE (Byte Pair Encoding) tokenizer has a large vocabulary for common English words and subwords, allowing it to encode English efficiently. Korean, however, uses a complex morphological system (agglutinative language) with thousands of possible word forms, meaning the tokenizer frequently falls back to character-level or small subword-level encoding — inflating token counts dramatically.

💸 Real Impact on Claude Code Users

1. Usage limits are exhausted faster

A Korean developer using Claude Code for the same coding tasks as an English developer will hit their 5-hour window limit and weekly cap significantly sooner — despite paying the same subscription fee.

• Pro plan ($20/month): A Korean user may effectively get only 30–50% of the coding session time an English user gets.
• - Max plan ($100–$200/month): Even at this price point, Korean users burn through their allocation 2x–3x faster on equivalent tasks.

2. Context window is effectively smaller

The advertised 200K (and 1M beta) token context window sounds large, but for Korean users, it fills up much faster. A codebase that fits comfortably within 200K tokens when discussed in English may approach or exceed the limit when discussed in Korean — forcing more frequent compaction and loss of context quality.

3. Same price, less service

This creates an invisible "language tax." Korean users are essentially paying a premium — not because they chose a more expensive plan, but simply because of the language they speak. This is neither transparent nor fair.

🌍 This Affects a Large User Base

Korea, Japan, China, and other non-English-speaking countries represent a significant and growing portion of Claude's user base. As Anthropic expands globally, this tokenization gap will become an increasingly visible barrier to adoption and retention in these markets.

• South Korea alone has a highly active developer community with strong AI adoption rates.
• - Many Korean developers use Claude Code as their primary coding assistant — and are directly experiencing this disadvantage daily.

Proposed Solution

Option 1: Language-adjusted usage limits (Short-term)

Detect the primary language being used in a session and apply a multiplier to the usage allocation. For example:

• Korean/Japanese users get 1.5x–2x token allocation
• - This levels the playing field without changing the tokenizer

Option 2: Improve the tokenizer for Korean/CJK (Long-term)

Retrain or expand the tokenizer vocabulary to include more Korean morphemes and common word forms. Models like Qwen have demonstrated that multilingual tokenizers can dramatically reduce this gap. This would benefit all non-English users permanently.

Option 3: Transparent communication (Immediate)

At minimum, clearly document in the pricing page and plan comparison that token usage varies by language, so non-English users can make informed decisions about which plan to choose.

Option 4: Usage measurement by "semantic units" rather than tokens

Explore whether usage limits could be partially decoupled from raw token counts — for example, measuring by request count or compute time rather than tokens alone.

Alternative Solutions

Currently, some users try to switch to English for Claude Code interactions to reduce token consumption, but this defeats the purpose of using one's native language and reduces productivity. There is no viable workaround within the current system — the token inefficiency is structural and unavoidable for non-English speakers.

Priority

High - Significant impact on productivity

Feature Category

API and model interactions

Use Case Example

A Korean developer (Max plan, $100–$200/month) uses Claude Code daily for backend development. They interact in Korean because it is their native language and allows for more precise and natural communication. However, they consistently hit their weekly usage cap 2–3x faster than an equivalent English-speaking developer doing the same tasks. The same code review request that costs ~6 tokens in English costs ~16 tokens in Korean — for the same outcome. Over a full work week, this compounds significantly, leaving the Korean developer with far fewer productive sessions despite paying the same (or more) than their English-speaking peers.

Additional Context

📚 Supporting Research

• TASE Benchmark (2025): Korean consistently underperforms vs English across token-level LLM tasks, partly due to tokenization misalignment.
• - Predli Blog Analysis (2025): Non-English languages can cost 65%–340% more tokens than English for equivalent content, depending on the model and language.
• - - VentureBeat (2024): Claude's tokenizer produces 20–30% more tokens than GPT for the same English content — the gap for non-English content is even larger.
• - - - The concept of a "token tariff" — where language choice directly determines cost and capacity — is an emerging fairness concern in the AI industry.

🙋 About This Report

This issue was raised by a Korean Claude Code user (Max plan subscriber) who noticed that their weekly usage limits were being exhausted significantly faster than expected — not due to heavy usage, but due to the language overhead of writing in Korean.

This is not an isolated complaint. Korean developer communities and forums have noted this issue repeatedly, and it is a growing source of frustration for non-English Claude users globally.

📣 Call to Action

If you are also experiencing this issue as a non-English Claude Code user, please 👍 upvote this issue so Anthropic can gauge the scope of impact.