| TokenBundle Class |
A special Token which represents a set of alternatives (i.e. an ambiguous
analysis) of other tokens which cover the exactly same input span.
Inheritance HierarchySystemObject
Sdl.LanguagePlatform.CoreSegmentElement
Sdl.LanguagePlatform.Core.TokenizationToken
Sdl.LanguagePlatform.Core.TokenizationTokenBundle
Sdl.LanguagePlatform.CoreSegmentElement
Sdl.LanguagePlatform.Core.TokenizationToken
Sdl.LanguagePlatform.Core.TokenizationTokenBundle
Namespace: Sdl.LanguagePlatform.Core.Tokenization
Assembly: Sdl.LanguagePlatform.Core (in Sdl.LanguagePlatform.Core.dll) Version: 1.6.0.0 (1.7.3164.0)
SyntaxThe TokenBundle type exposes the following members.
Constructors| Name | Description | |
|---|---|---|
 | TokenBundle(IListPrioritizedToken) |
Initializes a new instance with a list of prioritized tokens.
|
| TokenBundle(Token, Int32) |
Initializes a new instance with the specified values.
|
Properties| Name | Description | |
|---|---|---|
 | Count |
Gets the number of elements in the collection.
|
| IsPlaceable | IsPlaceable. Is true if any of the alternatives
is a placeable token.
(Overrides TokenIsPlaceable.) |
| IsPunctuation |
true iff the token is a general punctuation, an opening punctuation, or a closing punctuation
(Inherited from Token.) |
| IsSubstitutable | IsSubstitutable. Is true if any of the alternatives
is a substitutable token.
(Overrides TokenIsSubstitutable.) |
| IsWhitespace |
Gets a value which indicates whether this token is a whitespace token or not.
(Inherited from Token.) |
| IsWord |
true iff the token is a word, an abbreviation, or an acronym
(Inherited from Token.) |
| Item |
Gets the prioritized token at the specified index position. If the index position
is invalid, an exception is thrown.
|
| Span |
Gets or sets the span covered by this token in the original segment.
(Inherited from Token.) |
| Text |
Gets or sets the token's text.
(Inherited from Token.) |
| Type |
Gets, or sets, the TokenType of this token.
(Inherited from Token.) |
Methods| Name | Description | |
|---|---|---|
| AcceptSegmentElementVisitor |
Invalid for this class. Will throw a InvalidOperationException (Overrides SegmentElementAcceptSegmentElementVisitor(ISegmentElementVisitor).) |
| Add(Token, Int32) |
Adds a new token to the set of alternatives. Potential duplicates are kept.
|
| Add(Token, Int32, Boolean) |
Adds a new token to the set of alternatives.
|
| Contains |
Determines whether the specified token is contained in the set of alternatives.
|
| Duplicate |
Invalid for this class. Will throw a InvalidOperationException (Overrides SegmentElementDuplicate.) |
| Equals | Equals(object) (Inherited from Token.) |
 | Finalize | Allows an object to try to free resources and perform other cleanup operations before it is reclaimed by garbage collection. (Inherited from Object.) |
| GetBest |
Obtains the token with the highest priority in the set of alternatives. If two tokens
have the same priority, the first one will be the winner.
|
| GetEnumerator |
Gets an enumerator to iterate through the list of alternatives.
|
| GetHashCode | GetHashCode(object) (Inherited from Token.) |
| GetSimilarity | GetSimilarity(SegmentElement). Returns the highest similarity to any
of the alternatives.
(Overrides SegmentElementGetSimilarity(SegmentElement).) |
| GetTokenType | Type. Returns the token type of the first alternative in the collection.
However, be aware that the alternatives may represent different token types.
(Overrides TokenGetTokenType.) |
| GetType | Gets the Type of the current instance. (Inherited from Object.) |
| GetWeakHashCode |
See GetWeakHashCode (Inherited from Token.) |
| MemberwiseClone | Creates a shallow copy of the current Object. (Inherited from Object.) |
| SetTokenType | (Inherited from Token.) |
| SortByDecreasingPriority |
Sorts the set of alternatives by decreasing priority.
|
| ToString | Returns a string that represents the current object. (Inherited from Object.) |
See AlsoCommunityComments