| SimpleToken Class |
A Token which represents a simple token, such as a word, whitespace,
or punctuation.
Inheritance HierarchySystemObject
Sdl.LanguagePlatform.CoreSegmentElement
Sdl.LanguagePlatform.Core.TokenizationToken
Sdl.LanguagePlatform.Core.TokenizationSimpleToken
Sdl.LanguagePlatform.Core.TokenizationGenericPlaceableToken
Sdl.LanguagePlatform.CoreSegmentElement
Sdl.LanguagePlatform.Core.TokenizationToken
Sdl.LanguagePlatform.Core.TokenizationSimpleToken
Sdl.LanguagePlatform.Core.TokenizationGenericPlaceableToken
Namespace: Sdl.LanguagePlatform.Core.Tokenization
Assembly: Sdl.LanguagePlatform.Core (in Sdl.LanguagePlatform.Core.dll) Version: 1.6.0.0 (1.7.3164.0)
SyntaxThe SimpleToken type exposes the following members.
Constructors| Name | Description | |
|---|---|---|
 | SimpleToken |
Initializes a new instance with default values
|
| SimpleToken(String) |
Initializes a new instance with the specified token text, and a token type
of Word.
|
| SimpleToken(SimpleToken) |
Initializes a new instance with the values from another instance, creating a deep copy.
|
| SimpleToken(String, TokenType) |
Initializes a new instance with the specified values.
|
Properties| Name | Description | |
|---|---|---|
 | IsPlaceable | IsPlaceable. Returns true if the token represents
an acronym, a variable, a URI, or a generic text placeable.
(Overrides TokenIsPlaceable.) |
| IsPunctuation |
true iff the token is a general punctuation, an opening punctuation, or a closing punctuation
(Inherited from Token.) |
| IsStopword |
Gets or sets whether this token is a stop word
|
| IsSubstitutable | IsSubstitutable. Returns true if this token represents
a variable.
(Overrides TokenIsSubstitutable.) |
| IsWhitespace |
Gets a value which indicates whether this token is a whitespace token or not.
(Inherited from Token.) |
| IsWord |
true iff the token is a word, an abbreviation, or an acronym
(Inherited from Token.) |
| Span |
Gets or sets the span covered by this token in the original segment.
(Inherited from Token.) |
| Stem |
An optional stem to assign to this token. Stems are used for TM search and scoring.
|
| Text |
Gets or sets the token's text.
(Inherited from Token.) |
| Type |
Gets, or sets, the TokenType of this token.
(Inherited from Token.) |
Methods| Name | Description | |
|---|---|---|
| AcceptSegmentElementVisitor |
Visitor pattern implementation. Calls VisitSimpleToken(SimpleToken).
(Overrides SegmentElementAcceptSegmentElementVisitor(ISegmentElementVisitor).) |
| Duplicate |
See Duplicate (Overrides SegmentElementDuplicate.) |
| Equals | Equals(object) (Overrides TokenEquals(Object).) |
 | Finalize | Allows an object to try to free resources and perform other cleanup operations before it is reclaimed by garbage collection. (Inherited from Object.) |
| GetHashCode | GetHashCode(object) (Overrides TokenGetHashCode.) |
| GetSimilarity | (Overrides SegmentElementGetSimilarity(SegmentElement).) |
| GetTokenType | (Overrides TokenGetTokenType.) |
| GetType | Gets the Type of the current instance. (Inherited from Object.) |
| GetWeakHashCode |
See GetWeakHashCode (Inherited from Token.) |
| MemberwiseClone | Creates a shallow copy of the current Object. (Inherited from Object.) |
| SetTokenType |
Overwrites the token type. This should only be done with caution, and to reclassify
exchangeable token types (acronyms, words, abbreviations). In particular, do not set
the token type to one of the special (localizable) token types.
(Overrides TokenSetTokenType(TokenType).) |
| ToString | (Overrides ObjectToString.) |
See AlsoCommunityComments