Splitting Source Data

Splitting feature enables you to process large files. You must use splitting if your source file is huge. If you do not use splitting in this case, then Data Mapper may not be able to map it. If the data in the source file is huge, you can split the source data into different chunks. You can specify the size of the chunks in terms of the number of records or data streams. In the output, however, only one file is generated at the location specified in the target activity.

Multiple target schema are not supported for data splitting.

Steps to split the source data

Click Advanced Properties on the Create Data Mapping screen to expand the advanced properties of the new Mapping activity (see Figure 465).
Figure 465: Advanced Properties
Select the Splitting Data checkbox to enable the splitting of data.

If you select this checkbox, then you have to split the data. You can do this by setting the Splitter XPath in the Data Mapper applet. Alternately, you can split the data by Enabling Quick Splitting feature.
Enter the number of records into which you want to split the data, in the Split Number of Records field. For example, if you have a source file of 100,000 records and you want split this file into records of 10,000 each, then you need to enter 10,000 in this field.
Select the Parallel Processing checkbox to parallel process these records. By default, records are processed sequentially.
Enter the number of chunks you want to process concurrently, in the Maximum Concurrent Processes field. By default, value of 1 is entered.

It is advised to change this value, if Parallel Processing is selected. Else, only one chunk is processed at a time and hence sequentially.
Select the owner from the Owner dropdown list. By default, Administrator is selected.
Select the Read, Write and Execute checkboxes to set permissions for the Owner, Group or Other entities. By default, permissions are selected for the Owner and Group entities.
If mapping has been saved in the Data Mapper applet, then the XSL code is displayed in Mapping XSL field. This is a very useful feature. You can enter new XSL code or edit existing code to test or debug the mapping. If you save these changes, then they are reflected in the applet. For example, if you need to change the value of a constant, you can change it from this screen itself, instead of the applet. Additionally, any modifications made in the Mapping XSL code in the Data Mapper applet, will overwrite the XSL code in this field.

If the Advanced Properties and the Data Mapper screens are open simultaneously, then the changes made in the Mapping XSL field in the Advanced Properties screen will not be reflected in the Data Mapper screen.
The mapping XML code is displayed in Mapping XML field. You can enter new code or edit existing code. If you save these changes, they will be reflected in the applet.
Click Save to save the splitting details.