Transliteration Is Not a Replacement for Native Script

A field called arabic_title held 3,302 Urdu titles and 473 Arabic ones. The name was wrong, the rendering was wrong, and fixing both meant a rename, a real BCP-47 tag, and two different typefaces.

Transliteration Is Not a Replacement for Native Script

A commit in this repo starts with a number that undercuts its own field name: "This library is 3,302 Urdu works to 473 Arabic, and the field holding a title in its own script was called arabic_title." The name described the exception, not the rule.

One column, two scripts, two type faces
One column, two scripts, two type faces

Urdu (Nastaliq, more leading) vs Arabic (Naskh) — a real BCP-47 lang tag chooses the face.

Why one column can hold three languages

Urdu, Persian, and Arabic are all written in the Perso-Arabic script family. A column meant to hold "the title in its native script" was never wrong structurally — it's a CharField, it stores whatever string you give it — it was wrong in what it was called, and the name leaked into the UI, the admin help text, and every editor's mental model of what the field was for. The fix, in migration 0022_native_script_names.py, is deliberately inert as data operations go:

migrations.RenameField(model_name='qasida', old_name='arabic_title',
                       new_name='native_title'),
migrations.RenameField(model_name='poet', old_name='arabic_name',
                       new_name='native_name'),

Nothing added, dropped, or rewritten. Every value stays exactly where it was. The commit message is explicit that this matters: "the migration is a rename... and every value stays where it was" — worth stating outright because a rename migration that also transforms data quietly is a much scarier thing to review.

The rendering fault was the bigger one

Naming was cosmetic. The rendering bug was real: Urdu is conventionally set in Nastaliq and Arabic in Naskh, and each looks conspicuously wrong set in the other's face — Nastaliq slopes down and to the left across a line and needs noticeably more line-height than Naskh, which sits on a level baseline. But the templates hardcoded Naskh and marked every work lang="ar", including the 3,300 Urdu ones. Worse, the verse text itself carried lang="urdu" — not a real BCP-47 tag, so a browser given it ignores the attribute entirely, taking with it any chance of the browser choosing an appropriate font on its own.

The fix threads a real language code through to the template layer so the typeface follows the actual lang attribute rather than being chosen per template:

<div dir="rtl" lang="{{ qasida.bcp47_lang }}" class="verse {{ script_class }}">

with Nastaliq loaded and given the extra line-height it needs specifically for verse, not prose — because vocalized Arabic and Nastaliq's descenders both eat vertical space that an ordinary body-text line-height doesn't budget for.

Where this connects to the PDF export

The rule generalizes past HTML. core/pdf.py, which builds the downloadable typeset PDF via PyMuPDF's Story engine and HarfBuzz shaping, deliberately does not trust the stored language field to decide text direction:

def _direction(text):
    """
    Which way a line runs.

    Decided from the text rather than from the language field, which is often
    wrong in this library: several hundred rows labelled Arabic actually hold
    a Latin transliteration.
    """
    return 'rtl' if ARABIC_SCRIPT_RE.search(text or '') else 'ltr'

That's the same lesson from a different angle: a transliteration is not the native-script text with the letters swapped out, it's a separate layer with its own direction, its own typeface needs, and — per the earlier honest admission in transliterate.py — its own accuracy ceiling, since Arabic and Urdu are normally written without short vowels and those cannot be recovered from the consonant skeleton alone without a language model. A transliteration here is stated as a first draft for an editor to correct, never as a finished romanization. Treating it as anything more would be a claim the code can't back up.

Series: Qasida App.