The Local Voice: AI in Persian NLP and Localization

A product does not become Persian because its model can answer in Persian. A production system has to recognize how people type, preserve what a document actually says, retrieve across spelling variants, understand domain terminology, render mixed right-to-left and left-to-right content safely, and know when fluency exceeds evidence.

Persian is not a low-resolution copy of English or Arabic. Its morphology, clitics, ezafe construction, optional short vowels, script variants, half-space conventions, colloquial forms, named entities, calendars, digits, and bidirectional layout all affect accuracy. Formal Iranian Persian is also not interchangeable with Dari, Tajik, every regional variety, or every diaspora writing practice.

This guide reflects public research and standards available on 30 July 2026. Benchmarks establish a limited comparison under their dataset and split; they do not prove fitness for a product, domain, population, or high-impact decision.

Define the locale and workflow before choosing a model

“Support Persian” is not a testable requirement. Specify:

target locale and varieties: for example, formal Persian used in Iran plus a defined set of colloquial inputs;
script and input channels: native keyboard, Arabic keyboard, Latin transliteration, voice, OCR, or mixed text;
domain and consequence: retail search, customer support, legal extraction, financial reconciliation, health navigation, or public information;
tasks: normalization, classification, retrieval, extraction, summarization, translation, generation, speech, or agent action;
user groups, accessibility needs, expected literacy, and device constraints;
authoritative source language and whether translation is informative or legally controlling;
acceptable abstention, correction, escalation, and response time.

Keep Dari and Tajik as explicit locale decisions. Shared linguistic history does not erase different scripts, vocabulary, morphology, institutions, dates, currencies, or user expectations. Evaluate each supported variety with speakers from that context.

Start with one outcome. “Find the correct Persian return-policy passage and draft a cited answer” is measurable. “Build a Persian copilot” is not. Connect the workflow to enterprise knowledge and search rather than assessing the chat layer alone.

Preserve source, canonical, search, and display forms

Persian pipelines face characters that look similar but are not the same code point. Common examples include Persian yeh ی (U+06CC) versus Arabic yeh ي (U+064A), Persian kaf ک (U+06A9) versus Arabic kaf ك (U+0643), and zero-width non-joiner or ZWNJ (U+200C). Users also mix Persian, Arabic, and Latin digits and use inconsistent spaces around affixes.

Do not solve this by destructively rewriting every stored string. Keep four layers:

Source form: the exact text received, with provenance and checksum where needed.
Canonical form: a documented Unicode and application normalization used for deterministic comparison.
Search aliases: reversible variants for matching, including selected character, spacing, digit, transliteration, and colloquial alternatives.
Display form: the human-facing value formatted for its locale and context.

Legal names, account references, quoted evidence, hashes, signatures, identifiers, and source documents may require byte-accurate preservation. Normalization should improve retrieval without silently changing the evidence shown to a reviewer.

Version the normalizer. Test characters that should remain distinct, emoji, combining marks, Arabic diacritics, ZWNJ, nonbreaking spaces, URLs, email, IBAN-like values, dates, negative numbers, and code. Store which transformations produced a match so an operator can explain why two records were linked.

Handle morphology, segmentation, and real user language

Persian carries information through prefixes, suffixes, clitics, compounds, and spacing. The same lexical item can appear joined, separated, or with ZWNJ. Ezafe is usually not visibly marked. Short vowels are generally omitted, increasing ambiguity. Colloquial writing changes pronouns, verbs, word boundaries, and spelling; social and support channels add typos, elongation, emojis, transliteration, and code-switching.

A robust pipeline combines:

character-aware normalization with protected fields;
tokenizer tests on productive affixes, enclitics, numbers, abbreviations, and mixed script;
morphological or lexical signals where the task benefits;
domain dictionaries with preferred term, accepted variants, forbidden translations, and effective dates;
alias expansion and fuzzy retrieval calibrated on real queries;
named-entity gazetteers derived from authorized sources rather than a universal list;
query rewriting that preserves the original and exposes changes;
human correction that records the field and error type.

Do not assume an embedding model will absorb every orthographic issue. Test exact, lexical, and semantic retrieval separately. For high-stakes fields, an exact protected identifier should outrank a semantically similar narrative.

Use public benchmarks as instruments, not launch certificates

Persian NLP research provides valuable baselines:

ParsiNLU introduced more than 14,500 Persian instances across six understanding tasks and compared multilingual and monolingual models.
ParsBERT established a Persian-specific pretrained language model and evaluated it across downstream tasks.
FarsTail contains 10,367 natural-language-inference samples and documents easy and hard subsets plus dataset bias.
PersoNER contributed a manually annotated Persian named-entity corpus.
FarSense extended evaluation of Persian lexical-semantic capability.

These datasets differ in task, era, genre, annotation, population, and metric. A score on one does not imply reliable financial extraction, modern support language, colloquial chat, OCR, code-switching, or safe agent behavior. Benchmark contamination is also a concern when evaluating large general-purpose models on public test sets.

Use public results to select candidates and understand failure modes. Create a product evaluation from real, permissioned examples, including cases after the model’s training cutoff where possible. Separate development and locked test sets, deduplicate against retrieval corpora, document annotator instructions, and report uncertainty instead of a single unqualified score.

Build retrieval that respects Persian evidence

Persian retrieval quality often fails before generation. Index both protected source text and carefully normalized fields. At query time, retain the original query, generate bounded aliases, search exact and semantic channels, merge with calibrated ranking, enforce access control before retrieval, and return source spans with document version and date.

Useful retrieval tests include:

Persian and Arabic keyboard variants of ی and ک;
ZWNJ, joined, and separated compound or affix forms;
Persian, Arabic, and Latin digit variants where the field permits;
formal and common colloquial phrasing;
transliterated brand and product names;
mixed Persian-English technical terms;
Solar Hijri and Gregorian date expressions;
misspellings, OCR noise, and short ambiguous queries;
superseded policies and conflicting documents;
fields that must match exactly, such as invoice or account references.

Answers should cite the retrieved passage and distinguish source language from generated translation. If the authoritative document is English, mark the Persian rendering as a translation. If sources conflict or do not support the requested claim, abstain and show the conflict.

For speech input, use the same discipline. The Persian speech-technology guide explains why transcription quality must be measured by domain, speaker, noise, number handling, and downstream task—not only average word error rate.

Engineer RTL and bidirectional interfaces, not just text

Persian layout is a system property. The Unicode Bidirectional Algorithm, UAX #9, defines how mixed-direction text is ordered. It does not decide the correct product layout, alignment, truncation, or interaction.

Use semantic dir="rtl" at an appropriate container and isolate inserted user or machine values with dir="auto" or bidirectional isolation rather than manually reversing strings. Logical CSS properties such as margin-inline-start, padding-inline-end, and text-align: start are safer than duplicating left/right rules. Never reverse digits, code, URLs, email addresses, file paths, or identifiers.

The W3C Arabic and Persian Layout Requirements is a useful catalogue of writing-system needs, but as of this date it is a Working Group Draft Note, work in progress rather than an endorsed W3C Recommendation. Treat it as engineering guidance and verify behavior in the browsers, fonts, operating systems, and assistive technologies you support.

Test:

punctuation around mixed Persian and English;
inline links, badges, icons, input affordances, tables, charts, and breadcrumbs;
phone numbers, currency, percentages, dates, version numbers, and negative values;
selection, copy/paste, caret movement, search highlights, ellipsis, and wrapping;
screen-reader reading order, form errors, focus order, and keyboard navigation;
responsive cards at realistic Persian title lengths;
generated content containing untrusted directional-control characters.

Sanitize unexpected bidi controls where security policy requires, while preserving legitimate source evidence in a protected view. A visually reordered identifier can cause financial and security mistakes.

Example: a bilingual financial-support assistant

Consider a support assistant that answers Persian questions about invoices while the accounting system, product names, and some contracts remain English. A user writes: «فاکتور ١٢۳۴ چرا دوباره محاسبه شده؟» using mixed Arabic and Persian digits.

The pipeline keeps the original message, derives a normalized query, recognizes the protected invoice identifier, and resolves the authorized customer before retrieving. It searches invoice events by exact ID and the policy base using Persian aliases and bilingual terminology. The answer cites the Persian policy if authoritative; otherwise it clearly labels a Persian translation of the English source.

The interface displays the Persian explanation RTL while isolating invoice number, currency, SKU, and URL. It never translates product codes or silently changes the source ledger. If the calculation and policy conflict, the system does not improvise. It creates a case with retrieved evidence, affected amount, locale, model and index versions, and the user’s original wording.

Human reviewers see both source and normalized forms, retrieval spans, translation, confidence by stage, and suggested action. They can correct the entity, term, citation, or resolution separately. Those reason-coded corrections improve evaluation without treating every agent edit as ground truth.

Evaluate by stage, slice, and consequence

One “Persian quality” score hides the cause of failure. Measure:

Input: script and locale detection, protected-field preservation, OCR or speech error, normalization precision and recall.
Retrieval: recall at k, mean reciprocal rank, citation support, freshness, authorization failures, and exact-ID success.
Understanding: intent, entity, relation, inference, date and number accuracy by domain slice.
Generation: groundedness, unsupported-claim rate, terminology adherence, instruction following, and abstention quality.
Translation: adequacy, meaning preservation, named entity and number preservation, terminology, register, and omission.
Interface: bidi defects, clipping, visual order, keyboard and assistive-technology task success.
Outcome: first-contact resolution, correction, escalation, complaint, harmful-action, and user comprehension.

Slice by formal versus colloquial input, region or variety in scope, device, channel, document type, mixed script, query length, new versus legacy content, and critical user group. Report sample sizes and confidence intervals. Expert review is essential for legal, financial, clinical, safety, and public-service content.

Automatic translation metrics can help compare versions but should not be the acceptance gate by themselves. A fluent sentence that changes a negation, date, amount, party, or obligation is a failure regardless of aggregate score.

Govern terminology, updates, privacy, and feedback

Maintain a versioned terminology service: concept ID, Persian preferred form, English equivalent, accepted variant, prohibited translation, domain, owner, source, effective date, and examples. Apply it consistently to UI, search, generation, documentation, and human workflows.

Record model, prompt, normalizer, tokenizer, index, glossary, document, and UI versions for consequential output. Re-evaluate after any material change. Monitor drift when a new campaign, regulation, product line, or news event changes user vocabulary.

Minimize logged personal data. Persian free text may contain national identifiers, phone numbers, financial details, health information, or private family context. Redact or tokenize fields where appropriate, scope reviewer access, set retention, and keep evaluation examples permissioned and de-identified. Do not send sensitive content to an external model merely because it supports Persian.

Give users a field-level correction path: wrong person, amount, date, terminology, translation, citation, direction, or outcome. A generic thumbs-down is not enough to diagnose the system.

Set explicit launch and rollback gates

Block launch when protected identifiers change during normalization or translation; authoritative sources cannot be distinguished from generated text; access control is applied after retrieval; critical numbers, dates, names, or negation fail acceptance thresholds; unsupported answers are not refused; the UI corrupts mixed-direction values; target varieties lack evaluation; or no qualified Persian reviewer owns the workflow.

Degrade to search results, a template, or human handling when citation support drops, index freshness fails, a new vocabulary domain appears, high-impact error exceeds threshold, the model or normalizer changes without evaluation, or a bidi/security defect can misrepresent an identifier.

Every exception needs an owner, narrow scope, compensating control, review date, and evidence. “The sentence sounded natural” is not an acceptance criterion.

Frequently asked questions

Is a multilingual foundation model enough for Persian?

It can be a strong component. Product quality still depends on normalization, retrieval, terminology, domain evaluation, RTL engineering, human review, and safe fallback.

Should all Arabic characters be converted to Persian forms?

Not destructively. Canonical and search forms can normalize selected variants, but source, legal, identity, signature, and display requirements may demand exact preservation.

Are public Persian benchmarks reliable?

They are useful, bounded research instruments. Their task, sample, genre, date, annotation, bias, and possible exposure must be documented. Validate the actual workflow on a separate product set.

Is Persian localization the same as translating English copy?

No. It includes information architecture, terminology, calendar and number conventions, RTL interaction, cultural context, support operations, search behavior, and product policy.

What is the safest first Persian AI use case?

A narrow, cited retrieval workflow with protected identifiers, qualified reviewers, field-level feedback, and a fallback to source documents. Prove evidence retrieval and UI integrity before granting tools or automating consequential decisions.

A trustworthy Persian AI product does not merely sound local. It preserves the source, retrieves the right evidence across real writing variants, renders the answer safely, exposes uncertainty, and gives Persian-speaking users the same control and recourse as users of the product’s first language.

Source notes

Sources reviewed and current as of July 30, 2026:

ParsiNLU, ParsBERT, and FarsTail — research datasets and model baselines with task-, sample-, date-, and domain-specific limits.
PersoNER and FarSense — Persian named-entity and lexical-semantic research, not product-level certification.
Unicode Bidirectional Algorithm, UAX #9 — normative bidirectional ordering algorithm.
W3C Arabic and Persian Layout Requirements — Working Group Draft Note and work in progress, not an endorsed W3C Recommendation.
Unicode CLDR Persian guidance — locale-specific translation and formatting considerations.