Using the Data Integration Template

You can create a Data Integration Transaction by using prebuilt Data Integration Template bundled with Connect.

To create a Data Integration Transaction:

Click Transactions > Templates.
On the Templates screen, search Data Integration.
Click Use This for the Data Integration Template to create a Data Integration Transaction.
Provide the name and description of the Transaction.
Select the Transaction user type.
Select the Project.
Click Next to go to Transaction Settings step.
On the Transaction Settings screen, do the followings:
1. Enter the details as described in the following table.

Field Name		Description

Field Name		Description
Contact User		Email id(s) listed here will receive a notification email whenever the Transaction is executed successfully or if there is an error during execution. Multiple email ids are separated by a comma. If Adeptia Connect is running on multiple environments and you want to receive environment name in the emails, then you need to specify the environment name in the relevant property (Account > Settings > Server Nodes Settings > Systems > Personalization).
When to notify the contact?		Always: Send an email whenever the execution of the transaction associated with this template is successful. On Error: Send an email if an error occurred during the execution of the transaction associated with this template. Do Not Notify: Select this option if you do not want to receive an email notification after the execution of the transaction associated with this template.
Email Notification Format		This field is visible only when you have selected Always or On Error option to send the notification to the user. You can choose the Email Notification Format to used in the email notification to be sent to the user on successful and failed Transaction. There are two possible values for this: Default and Customized Notification Format. By default, the value of this field is set to Default. If you select the Customized Notification Format, you need to provide the following details: Select the custom success email notification format from the Custom Success Notification Format field. You can also create a new email notification format by clicking Add button. The existing email notification format can also be copied and edited by clicking the Copy and Edit icons respectively. Use refresh icon to refresh the list. This field is visible only when you have selected Always option to send the notification to the user. Select the custom error email notification format from the Custom Error Notification Format field. You can also create a new email notification format by clicking Add icon. The existing email notification format can also be copied and edited by clicking the Copy and Edit icons respectively. Use refresh icon to refresh the list. This field is visible only when you have selected Always option to send the notification to the user.
Data Encoding		Character-set encoding for your source data. The available options are UTF-8, UTF-16, UTF-32, and ISO-8859-1. You can also add and use a valid custom Data Encoding.
Data encryption at rest		Select the particular option in the Data encryption at rest field. You can enable, or disable, and use global Property value to configure EAR for Template. Use Global Policy: Select 'Use Global Policy' to use Global Property value for Template. Yes: Select 'Yes' to enable EAR for Template. It will not use Global Property value. No: Select 'No' to disable EAR for Template. It will not use Global Property value.
Enable External Mapping Validation		You can use this function to validate the REST API in mapping. Follow the steps to enable the External Mapping Validation: Select the Enable External Mapping Validation checkbox to configure the REST account for external mapping validation. Select the Rest Account in the Provide REST Consumer information field. You can also create a new REST Account by clicking Add icon . Select the Do Not Persist Mapping In Case Validation Failed checkbox if you want to restrict saving mapping until the external mapping validation succeeds. Click here to get more details on using Validation Option in Mapping.
Enable External Mapping Function(s)		You can use this function to apply the external map function in the mapping through REST API. Follow the steps to enable the External Mapping Function: Select the Enable External Mapping Function(s) checkbox to select the REST account. Select the Rest Account in the Provide REST Consumer information field. You can also create a new REST Account by clicking Add icon. Click here to get more details on using Available Functions in Mapping.

b. Expand Advanced Settings to set the logging level, repository file retention, and activity logging retention. The available options for the fields are as follows:

Logging Level- INFO, DEBUG, and ERROR

Repository File Retention- DONT DELETE, DELETE, DELETE ON SUCCESS, and DELETE ON SUCCESS EXCEPT SOURCE DATA

Activity Logging Retention- DONT DELETE, DELETE ON SUCCESS, and NO LOGGING.

To learn more about these options, refer to this page.

9. Click Next to define the Source application account. Select FTP as source type and click Next.

10. Select the either of the option to Convert or Pass-thru the source file format to the destination format.

11. Select the Source application account. You can either use an existing account or create a new one. Click to add a new account. For information on how to add a new account, refer to Supported Applications.

12. Choose the event to Trigger the data transfer.

13. Provide the configuration details of your account.

Field Name	Description

Field Name	Description
Folder Location	Folder location from where to pick files.
Process all the existing records/files when this transaction is turned on	To process all the existing files from the specified location.
Don't process duplicate files	This option appears only for FTP application. Selecting this option will not process any new file that has the same name as the file that has already been processed. This option does not process the duplicate files as per the retain time defined in the event metadata property.
Send mail notification for no data on trigger	Selecting this option will check the data and send mail notification to the Partner for no data on trigger.
Send mail notification for error on trigger	Selecting this option will check the data and send mail notification to the Partner for error in data on trigger.
Delete file on success	Delete the file once it is processed.
File type	File format to process the files. Multiple file formats are separated by a comma. Use . to process all file formats.
File exclude	Type the file format to exclude the file type while transaction is running. Multiple file formats are separated by a comma. For example if you want to exclude text files from the FTP location, type .txt in the field.
Trigger Type	You can set trigger through POLLING and CRON EXPRESSION. POLLING: The time interval during which the Transaction triggers. By default, Transaction triggers after every 10 minutes. Select No polling option to trigger Transaction manually. CRON EXPRESSION: Define the specific time schedule for the Transaction to trigger. For more information on how to define Cron Expression, click here.
File Stable Time	Provide the waiting time (for the file to become stable) after which the trigger happens. For example, if you choose 30 Seconds in the stable time then system will wait for 30 seconds for the file to go stable before it triggers the transaction.

14. Expand Other Settings and provides the detail.

Select Trigger in Sequence check box to trigger the Transaction in sequential manner. If you do not select the Trigger in Sequence check box then the associated Transaction will trigger concurrently.

If you want to enable misfire handling for your file event then, select the Run Once option in the Action on Misfire Event field. If you do not wish to enable misfire handling for your file event then, select the Ignore Misfire option. For more details, refer to Handling Misfire Event.

The Connector field lists the APIs to connect to the FTP Server. The options changes as per your selection of the protocols that you want to use to establish FTP connection.

Protocol	Options

Protocol	Options
FTP	Native Secured Inet Factory
SFTP	J2SCH (VFS) Secured Inet Factory J2SSH
FTPS	J2SCH (VFS) Secured Inet Factory

Select Verbose check box, if you want to create a log file of your FTP activity. This check box is enabled only if you select Secure Inet Factory from the Connector field.

15. Click Next to select the source data format.

16. Select the source data format and source data layout for the Source application. Source data format is the format in which the data of the source file exist, for example, text, excel. Source data layout is the layout of the source file. You can either use an existing layout or create a new one. Click Plus icon to create a new layout.

17. Click Next

18. Select the checkbox in case you want to encrypt source data. Else, Click Next.

19. Click Next to define the Destination application account. Choose File as a destination application.

20. Click Next to define the data delivery to destination.

21. Click Next to select the LAN target.

22. Click Next to select the target data format and data layout for the Destination application.

23. Select Data Encryption and check the box if you want to encrypt the destination file.

24. Click Next to define the Mapping between the fields of the selected source and the destination application.

Select the mapping from the Select Mapping drop-down list and click Next.
Or
Click Add button to create a new mapping. A new window will appear, enter the details and click Create.
Map the elements in Mapper and save the mapping.

25. Click Save & Exit to save the Transaction.