A recognized token is a short piece of text, enclosed in a segment, that a TM treats as a single word because it is a defined format. For example, if dates are enabled as a recognized token in a TM, the TM recognizes Monday 1 January, 1900 as one word.
The TM settings determine which tokens are recognized.
The TM scans the TU for recognized tokens only when the TU is added to the TM. If you later enable token recognition, the TM does not immediately analyze existing TUs for tokens: you need to re-index the TM. When you do so, the TM uses the current token settings and scans existing TUs for tokens.
The project settings determine if the software does or does not auto-localize these tokens.