Splitting feature enables you to process large files. You must use splitting if your source file is huge. If you do not use splitting in this case, then Web Mapper may not be able to map it. If the data in the source file is huge, you can split the source data into different chunks. You can specify the size of the chunks in terms of the number of records or data streams. In the output, however, only one file is generated at the location specified in the target activity.
Steps to split the source data
- Click Modify Settings on the Create New Mapping screen to expand the Settings of the new Mapping activity.
Select the Splitting Data checkbox to enable the splitting of data.
If you select this checkbox, then you have to split the data. You can do this by setting the Splitter XPath in the Data Mapper. Alternately, you can split the data by Enabling Quick Splitting feature..
- Enter the number of records into which you want to split the data, in the Split Number of Records field. For example, if you have a source file of 100,000 records and you want split this file into records of 10,000 each, then you need to enter 10,000 in this field.
- Select the Parallel Processing checkbox to parallel process these records. By default, records are processed sequentially.
Enter the number of chunks you want to process concurrently, in the Maximum Concurrent Processes field. By default, value of 1 is entered.
It is advised to change this value, if Parallel Processing is selected. Else, only one chunk is processed at a time and hence sequentially.Click Create to open the Select Layout window.