howtosecure.ai
Implementation guide OWASP AISVS v1.01

Input Validation · Prompt Injection Defenses

Normalise input before the model sees it

AISVS 2.1.1 Level 1

Verify that input normalization is applied before tokenization or embedding.

OWASP AISVS v1.01, requirement 2.1.1

In plain English

Convert all input to one standard form before anything else looks at it. Unicode lets the same word be written many ways, so without this step text can look harmless to your filters and still read as an instruction to the model.

What good looks like

  • Unicode normalisation is applied to every text input before any filter, tokeniser or embedding model sees it.
  • Zero-width, bidirectional-override and other invisible characters are removed or handled by a written rule.
  • The same normalisation runs on every route into the prompt: chat, uploads, retrieved documents and tool output.
  • Tests cover look-alike characters, full-width letters and zero-width spaces.

How teams usually do it

  • NFKC normalisation in the input layer, which folds full-width and other compatibility characters into their plain forms. It is one function call in most languages, for example unicodedata.normalize in Python.
  • A second pass that strips characters in the Unicode "format" category (Cf) and the invisible "tag" block, which attackers use to hide text from human reviewers.
  • One shared cleaning function that every input route calls.

Suggested evidence

  • The normalisation code and where it sits in the pipeline, ahead of filtering.
  • Test cases with homoglyphs, zero-width characters and tag characters, and their results.

Common mistakes

  • Normalising after the filters have run, so the filters see the raw text and the model sees the cleaned version.
  • Normalising the chat box and forgetting text extracted from documents.
  • Assuming NFKC handles look-alike letters across scripts. It does not turn a Cyrillic "а" into a Latin "a", so you need a confusables check or a character allow-list as well (see 2.1.5).

Related requirements

2.1.2 Catch encoding tricks hidden in input · 2.1.5 Allow-list the characters you accept · 2.1.3 Screen inputs for prompt injection

Not sure which requirements apply to your system? Take the assessment, or browse every guide in this chapter.

Requirement text from the OWASP AI Security Verification Standard, licensed under CC BY-SA 4.0. This guidance is shared under the same licence.