Page Comparison

Kafka messages sometimes remain unprocessed by a JMS source for unexplained reasons. To ensure that all the Kafka messages are consumed and published successfully, Adeptia Connect integrates Apache Kafka using which the application components can create, send, receive, and read messages using reliable communication.

To use Apache Kafka, you need to create a Kafka account and define a Kafka listener and a target. Kafka messages can be consumed and published by a template, transaction, or process flow. 

Creating Kafka Account

Kafka account is used to connect with Kafka Server. While creating Kafka account, you need to specify the Provider Jar files, define the following properties which are used to connect to Kafka Server. 

1. Click Configure > ACCOUNTS > Kafka.  

2. Click Create Kafka account. 
3. In Create Kafka Account window, do the following:
   1. In the Name and Description fields, enter the name and description respectively for the new Kafka account respectivelyaccount.
   2. In the Broker field, enter the URL of the Kafka brokers, foe example, host1:port1,host2:port2.

   3. In the Security Protocol field, enter the protocol used to communicate with brokers.

      Tip
      The supported protocols are PLAINTEXT and SSL. In case you select SSL, you need to select a Keystore in the Select Keystore field.

      
      

   4. In the Addon Configuration window, enter the Add On Configurations to be used in back end for performing operation on Kafka server (e.g. param1=value,param2=value..).

      Note
      Here you can use properties which are not listed on this Kafka account creation interface, for example, you may need to use a property called Serializer that converts the data to bytes. You  You can find the properties available at this location.

       

   5. In the Select Project field, select the project.
   6. Click Save.

Creating Kafka Listener

Follow the steps below to create Kafka listener:

1. Click Configure > EVENTS > Kafka Listener.
2. Click Create Kafka Listener.
3. In Create Kafka Listener window, do the following:
   1. In the Name and Description fields, enter the name and description respectively for the Kafka listener.
      

   2. In the Kafka Account field, select the Kafka account. You can create a new Kafka account by clicking Image Added.
      

      Note
      Click Test to check if you are able to connect with Kafka server using this Kafka account.

      
      

   3. In the Topic(s) field, select the topic to be used. 

      Tip
      You can select multiple topics.

      
      

   4. In the Consumer Group field, enter the string that uniquely identifies the group of consumer processes to which this consumer belongs. By setting the same consumer group multiple processes indicate that they are all part of the same consumer group.
   5. In the Key(s) to Filter Messages field, enter the Name of the message key to process. You can use comma to provide multiple message keys.
   6. In the Consumer Count field, enter the number of consumers that connect to Kafka server.
   7. In the Process Flow Name field, select the process flow.
   8. In the Auto Offset Reset field, select the value to define the offset when there is no initial offset in ZooKeeper or if an offset is out of range:
      
      • EARLIEST : Automatically reset the offset to the earliest offset
      • LATEST : Automatically reset the offset to the latest offset fail: throw exception to the consumer.
      • NONE
   9. Select the Enable Message Aggregator check box to define the message aggregator properties:
      1. In the Number of Messages for Aggregation field, enter the number of message should be aggregated.
      2. In the Aggregation Timespan field, enter the time in seconds, messages received within the provided timespan will be aggregated.

      3. In the Message Type field, select the message format.
         

         Tip
         The supported message formats are JSON, XML, and PLAINTEXT.

         
         

      4. In the Message Joiner field, enter the delimiter based on which the messages will be aggregated.
         It depends on the Message Type.
         
      • • XML : It will be an XPath.
        • PLAINTEXT : It will be a delimiter.
        • JSON : It can be empty or an element name.
   10. In the Seek To field, select if the Kafka Consumer will read the message from beginning or end on startup.
       
   11. In the Heartbeat Interval field, enter the heartbeat interval.
       The expected time between heartbeats to the consumer coordinator when using Kafka group management facilities. Heartbeats are used to ensure that the consumers session stays active and to facilitate rebalancing when new consumers join or leave the group. The value must be set lower than session timeout, but typically should be set no higher than 1/3 of that value. It can be adjusted even lower to control the expected time for normal rebalances.
   12. In the Session Timeout field, enter the time interval for session timeout.
       The timeout used to detect failures when using Kafka group management facilities.
   13. In the Select Project field, select the project.
   14. Click Save.
       
       

1.  Select the server type from the drop-down list of JMS Server Type.
   

   
   To know more on the JMS Server types supported by Adeptia Connect and their corresponding JAR files, JNDI Factory and URLs, refer Table 4.

2. Click Upload Jars button to upload the driver jar files for the JMS Server. The UploadJar Files screen is displayed.
   
   
   

   
   JMS Jar files are drivers, which are used to connect JMS Servers. There are specific jar files for different JMS Servers. These jar files are not provided with the Adeptia Connect. Users should use the jar files that are available with the JMS Servers.

3. Click the Browse button and select the required jar files. The path of the selected jar file is displayed in the Browse File field.
4. Click the Upload JAR button. The file name is displayed in under the heading File Names list.
5. Repeat the steps to upload additional jar files.

6. Click the Finish button to return to the Create JMS Provider screen. The uploaded jar file(s) is displayed in the textbox Provider Jar Files.
   
   

   
   If required, you can also unload the JAR files. To unload jars files, click the Unload JAR button.

7. Enter the JNDI Factory class name as specified by the JMS Provider in the textbox JNDI Factory. For Example, in case of Apache ActiveMQ, the value is org.apache.activemq.jndi.ActiveMQInitialContextFactory.
8. Enter the URL of the JMS Server in the URL field. For example, for Apache ActiveMQrunning on the same server, use tcp://localhost:61616.
9. Enter the username and password required to connect to JMS Server in the textboxes UserName and Password respectively.
10. Re-enter the password in the textbox Confirm Password.
11. Enter the JMS Provider Queue connection Factory in the Queue Connection Factory field. For example, in case of Apache ActiveMQ, Queue Connection Factory is QueueConnectionFactory.

12. Enter the JMS Provider Topic Connection Factory in the Topic Connection Factory field. For example, in case of OpenJMS, Topic Connection Factory is TopicConnectionFactory.

     

    The above example is only applicable in case of Apache MQ. If we use the Apache Active MQ as JMS Server then we can define the Topic and Queue name (which will be used in the Source/Target/Event) in the Additional Connection parameter table.

13. In case the JMS Server, which you are trying to connect, requires additional connection parameters, the enter those connection parameters and their values in the Additional Connection Parameters table.
    
    
    

14. Expand Advanced Properties and select the project of the activity from the drop-down list of Project.
    
    
    
    
15. Click Save.

...

 
Version of JAR files may vary depending on version of the JMS Servers you are using.

...