FTP Event

An FTP Event enables you to specify when and how frequently a process flow should be executed based on either creation of a new file or existence of a file(s) on an FTP Server or upon its modification.

Steps to create a FTP Event

1. Click Configure > EVENTS > FTP Event.
2. Click Create FTP Event.
   
   
   
3. On the Create FTP Event window, type the name and description of the new FTP event in the Name and Description fields respectively.
4. Type the Host name and Port on which FTP Server is running.

5. Follow the steps below based on whether you are using Hashicorp Vault to fetch the secrets (credentials) or you want to enter the secrets manually.

   To use Hashicorp Vault, you must enable Config Management feature and you may also need to do project mapping. For more details, refer to this page.

6. To fetch the secrets from the Hashicorp Vault, do the followings:

   You can choose to use Hashicorp Vault for all or a specific field that need to contain secret.

   1. In the User Id field, enter the variable in the following format:
      ${Vault:<Key1>}
      Where, <Key1> is the key assigned to the username in the Hashicorp Vault. 

      Each secret is maintained as a key-value pair in Hashicorp Vault.

      

   2. Select the Use Vault checkbox if you want to use Vault for retrieving the password.

   3. In the Password field, enter the variable in the following format:
      ${Vault:<Key2>}
      Where, <Key2> is the key assigned to the password in the Hashicorp Vault.
7. To enter the secrets manually, do the followings:
   1. In the User Id field, enter the username to connect to FTP Server.
   2. In the Password field, enter the password.

   3. In the Confirm Password field, re-enter the password.

8. Select the transfer type (ACTIVE or PASSIVE) from the Transfer Type.
   1. ACTIVE transfer is more secure since the client only initiates communication to the server on one port.
   2.  PASSIVE transfer the client initiates communication with the server over two ports. PASSIVE mode is useful when you are behind a firewall or a proxy.

9. Select the trigger from the Trigger Type drop-down list. 

   On FileModified	To configure the FTP event to check for any modification in the file(s).
   On FileExists	To configure the FTP event to check for the existence of the file(s).
   On FileCreated	To configure the FTP event to check for the creation of a new file(s).
   Trigger Name	Description

10. Select Run Initial Load checkbox to process the files created when the event is not active. This option is enabled only if On FileCreated option is selected in the Trigger Type.

    After the activation of the event, for the first time, the system processes all the files that exist in the base folder location, irrespective whether they were processed earlier or not. However, for subsequent activation, only the unprocessed files are processed each time

11. Select Don't Process Duplicate Files checkbox if you don't want to process the same file again. This option is enabled only if On FileCreated option is selected in the Trigger Type. 
    The next step is to decide if you want to use SFTP, or FTPS. If you do not use any of these two protocols, the default FTP protocol is used, and you need to skip to step 12 and complete all the steps that follow.

    In case you opt to use SFTP or FTPS protocol, follow the steps below.

12. Choose either of the two protocols – SFTP, or FTPS
    If you want to use SFTP protocol, 

    1. Select SFTP checkbox. 
    2. Convert the format of the key that you want to use to OpenSSH key format by using any key conversion tool. Once converted, import the key to Key Manager. Refer to this page to know how to import a key to the Key Manager.
    3. In the Key Manager field, select the key from the list.
    4. Skip to the step 12 and complete all the steps that follow.
    If you want to use FTPS protocol, 
    1. Select FTPS checkbox. 

    2. In the FTPS Mode field, select FTPS mode. It can be Explicit or Implicit depending on FTP Server you are accessing. 

       In Explicit mode (also known as FTPES), an FTPS client must "explicitly request" security from an FTPS server and then step-up to a mutually agreed encryption method. If a client does not request security, FTPS server can either allow the client to continue in insecure mode or refuse/limit the connection.

       In Implicit mode, negotiation is not allowed with implicit FTPS configuration. A client is immediately expected to challenge FTPS server with a TLS/SSL message. If such a message is not received by the FTPS server, the server should drop the connection.

       By default, it should be Explicit as in this mode client requests security from FTPS server.

    3. Select the protection level supported by the FTP Server from the Protection Level drop-down list. 

       Protection Level is a single Telnet character code specifying the data channel protection level. This command indicates server what type of data channel protection the client and server will be using. The default protection level is None. The available options are:

       • None
       • Clear: This protection level indicates that the data channel will carry the raw data of the file transfer with no security applied.

       • Private: This protection level indicates data will be confidentially protected.

    4. Select Validate Server checkbox to validate the certificate sent by the FTPS Server against the certificate imported in Keystore.

    5. Select the activity from the Keystore Name drop-down list. This option is applicable only if you have selected Validate Server. 

13. Type the name of the file(s) that the FTP event needs to verify in the File Include Criteria. To select the file(s), click Define List. The File Include Criteria List window appears.

    1. Type the name of the file or any criteria in the File Include Criteria, for example, .txt. a*, and so on, and then click Add to List. 

    2. Click Save. 
       
       
       
       

14. Type the name of the file that the FTP event does not need to verify in the File Exclude Criteria. For example, you have typed *.txt as a value in the File Include Criteria, and you want following two files Gdata.txt and Gdata1.txt file not to be verified by the FTP Event. Then Gdata.txt and Gdata1.txt file name need to be entered separated by a comma in the File Exclude Criteria. You can also use regular expressions to specify more than one file in the File Include Criteria and File Exclude Criteria.
    
    

    If more than one file is specified in the File Include Criteria, the process flow will trigger for each file.

15. Type the path of the file from where FTP events verify the file in the File Base Location field.  
    

    In File Include Criteria and File Exclude Criteria, you can also give the folder name along with the file name. For example, suppose you have typed C:\Gmdata in the File Base Location field. There are two sub-folders - Purchase and Purchase1 in the Gmdata folder. Now, you don't know whether the file is in Purchase or Purchase1 folder. To handle this scenario, you can define File Include Criteria as defined below:
    Purchase/.txt,Purchase1/.txt

    
    You can also use regular expression in the folder name as given below:
    Pur*/*.txt

    This path includes both the folders - Purchase and Purchase1.

16. Provide the date from which FTP event will start triggering in the Event Start Date. The date must be in MM/dd/yyyy format. Click the Calendar icon to select the required date.
17. Select the start time when the event will start triggering from the Time field.
18. Provide the date on which FTP event will stop triggering in the Event Expiry Date. The date must be in MM/dd/yyyy format. Click the Calendar icon to select the required date.
19. Select the expiry time when the event will stop triggering from the Time field.

20. Select the trigger from the Trigger drop-down list. You can select Simple or Cron from the list.

    By default, Simple is selected as a Trigger type.

21. Provide the time interval for which FTP event will check for the arrival of any file or modification of existing file in the Polling Frequency. Type the digit in the Frequency and select the unit of time (seconds, minutes, hours, day, etc.) from the Duration.

    We recommended minimum Polling Frequency to be more than 60 seconds.
22. If you have selected Cron as a Trigger type then define the specific time schedule to trigger. For more information on how to define Cron Expression, click here.
    
    
    
    
23. Type the time in the File Stable Time for which trigger will wait for the file to become stable and select the duration from the Duration. This is applicable for OnFileCreated and On FileModified Trigger Type. Type 0 to pick up the file without checking its stability (not running MDTM and size commands).  

24. Expand Advanced Properties to view the advanced properties of an FTP Event.
    
    
    
    

25. Select the Option you want to send email notification for. You would view Duplicate file in this box only if you have clicked the option Don't Process Duplicate Files (Step 11).
26. Type the email id in the Notification Receiver Email-ID(s) (comma separated) to receive error emails related to this FTP event.

27. Select Trigger in Sequence check box to trigger the associated process flow in a sequential manner. If you do not select the Trigger in Sequence check box then the associated process flow will trigger concurrently.

28. If you want to enable misfire handling for your event then, select the Run Once option from the Action on Misfire Event drop-down list. If you do not wish to enable misfire handling for your file event then, select the Ignore Misfire option.

    The Run Once option will run only one collective instance of all the misfires for a particular event. Whereas the Ignore Misfire option will not run a misfired event.
29. The Connector lists the APIs to connect to the FTP Server.
    

    By default, Apache Camel is selected in this field.  

    The options of the Connector drop-down list changes as per your selection of the protocols to use to establish the FTP connection. 

    FTP	Apache Camel NATIVE
    SFTP	Apache Camel J2SCH (VFS) J2SSH
    FTPS	Apache Camel J2SCH (VFS)
    Protocol	Options

    Make sure that you've selected an OpenSSH key in the Key Manager field in case the selected connector is Apache Camel and the protocol is SFTP. 

30. Enter the additional configuration details in the Add On Configurations text box. 

    This field is available only if you have selected Apache Camel connector. 

    If you're using SFTP protocol, you need to add the following information in the Add On Configuration field.

    preferredAuthentications=publickey,keyboard-interactive,password

31. Select Verbose checkbox to create a log file of your FTP service.

    You can enable the Verbose check box only if you have selected the Apache Camel option from the Connector field.

32. Select the Project from drop-down list of Project.

33. Click Save.