Filter Configuration and Segmentation

You can configure the settings for WorldServer file types in Management > Linguistic Tool Setup > Filter Configuration . You can have multiple filter configurations for each file type.

Each filter is designed to recognize what is translatable text and what is formatting in an asset, and to handle the formatting, subject to certain configuration options. The details in this topic can help you obtain the segmentation you need for translation, and the proper format after saving.

Note: The Binary File Type is not configurable.

Filter configurations customize two of the main jobs performed by filters: segmenting and recomposing.

Segmenting (decomposing the asset into segments presented for translation) occurs when WorldServer users perform the following operations:
- Apply the Segment Asset automatic action
- Open an asset in the Browser Workbench
- Export the asset to a translation kit
- Scope the asset
Recomposing (reassembling the translated target asset into its original format) occurs when you save the translated asset back to the repository (for example, using a Browser Workbench).

Decomposing the Asset: Segmentation

When an asset is segmented, the filter has two basic considerations for determining when to create a new segment: markup and "delimiters." In Browser Workbench, you only see the source segments that are marked for translation by default.

Figure 30: Segments for Translation

You can enable “Show Markup” in the workbench to display the markup segments, which are hidden by default. “Show Markup” is not available for all filters. In cases where it is not, Browser Workbench the markup is not available it is verbose and would not provide useful information.

Figure 31: Segments with Show Markup Enabled

WorldServer processes markup first, segmenting the asset, before the delimiters are processed.

Handling Markup
To determine what to present in translation segments, the content filters take into account things like markup, formatting attributes, and metadata.

In some cases, as for HTML, you can add user-defined elements and attributes, and specify the conditions under which they can be translated.
Other Options for Controlling Segmentation
WorldServer application filters give you control over what segments are presented for translation in other formats, like Microsoft Office, Adobe FrameMaker, and Adobe InDesign. For example:
- For Microsoft Word and RTF file types, you can specify text to exclude from translation based on the text style.
- For Adobe FrameMaker you can extract and translate content and text from formats such as autonumbers, text insets, and conditional text.
- For Adobe InDesign, you can include the content of hidden layers for translation.
Workbench Presentation of Segments
After an asset is segmented, if it is associated with a translation memory, it is leveraged against that TM, and a match score is assigned to each segment. By default, all text segments are displayed when you open the asset in a workbench. However, you can narrow the view of which segments are presented by a view category like “All except ICE and 100%”, “All non-translated”, “All with comments”, or “All pending review”. These “views” are sometimes referred to as filters because they filter out data. However, they should not be confused with the content filters that are applied during segmentation.
Sentence Breaking
After the filter processes markup, it further segments the asset, looking for “sentence breaking” delimiters.

When you open a text file, which has minimal markup, segments basically are sentences, delimited by periods (or question and exclamation marks). When the filter comes to one of these, it ends the segment. The Text Studio File Type also lets you specify structure and inline patterns to use.

If the file contains markup in addition to text, the segmentation process first segments based on the markup, then it makes another pass based on the “sentence breaking” delimiters. For example, the HTML Studio File Type extracts everything in a paragraph (<p>) element first, then breaks up sentences in the paragraph if it contains more than one sentence.

Decomposing the Asset: Formatting Encodings

The content filters offer controls over how formatting encodings such as entities are handled.

In the entities example, a configuration option (in the XML Studio File Type family and HTML Studio File Type) lets you “register” entities. If you register an entity it will always be presented as a character (for example, “ < ”). To have it presented as an entity (for example, “ < ”), you should not register it. You can also control how these entities are handled when you save the asset. See the “XML Entity Conversion Settings” help topic for more information on handling entities.

Recomposing the Asset: Saved Targets

WorldServer also offers controls over how the segments should be handled when it recomposes the target segments into a formatted asset after you save the asset in Browser Workbench. The following are a few of the options that the file types handle.

For the XML and XHTML file types, you can use the Writer Settings page to have the filter insert language information via the XML LANG attribute when generating target documents. The value of the LANG attribute is determined by the target locale.
By default, the XML validation page checks the schema validation when it verifies the target asset. You can opt to not to validate the XML. For example, when the XML contains references to external entities it causes this check to fail, causing the file type processing to stop.
The Adobe FrameMaker and Microsoft Office file types have a Font Mapping feature for specifying what fonts are used in the target asset. You enter the name of the font you wish to map, then enter the name of a corresponding font for every language in which you wish font translation to occur.