How to receive data from TCPIP & HTTPS port and publish the data to an MQTT server in DataScience ?

Introduction

This article explains how to get data from file and send copy of data to TCP/IP and other to HTTPS port. It also describes how to publish that data to MQTT Server with processor of PublishMQTT by using Syncfusion Data Integration Platform.

Configuration of MQTT server on local machine

As per the requirements, install the MQTT server on local machine. Please find the link to configure MQTT broker: https://syncfusion.atlassian.net/wiki/spaces/DPDI/pages/403276787/How+to+configure+MQTT+broker

Required processors

The following processors are used in this sample:

Overall workflow

• Data Sender: To simulate the HTTP and TCP sender, the sample file is used to transmit the data to TCP and HTTP ports.

• Data receiver: It explains how to get/receive data from TCP/IP and publish to MQTT server. For secure communications (HTTPS), configure the StandardRestrictedSSLContextService for those processors.

• Retrieve from MQTT broker: It explains how to receive files from MQTT broker and store in local system using the putfile processor. This part shows the data is received by corresponding topic and stored in local system.

Steps required:

Step 1

Drag the GetFile processor and configure the path of input data in this processor.

Step 2

Drag the PutTCP processor and configure the Hostname and Portno in this processor.

Step 3

• Drag the InvokeHTTP processor and configure the properties field of HTTP method and Remote URL. Configure and enable the controller services of StandardRestrictedSSLContextService.

• After filling the required properties of KeyStore, Trust store filename, and its password as given in the following screenshot, enable the controller service.

• The truststore and key store file is used from installed location, you can create the truststore and KeyStore based on the domain. For more details, please refer to https://help.syncfusion.com/data-integration/how-to/configure-ssl-context-service-for-invokehttp-processor#steps-involved

Note: The previously given link is for StandardSSLConextService, the StandardRestrictedSSLContextService used here also requires the same steps as mentioned in the link.

• These previously given three processors are used to get the input data from specific location and configure with port number to send data in TCP and HTTPS port.

Note: Don’t start these processors initially. After configuring the another processor, start these.

Step 4

Drag the ListenTCP processor and configure the same port number that is set in the PutTCP Processor.

Step 5

• Drag the PublishMQTT processor and configure the properties of broker URI, Client ID, Topic, Quality of service (QoS) and Retain message. This processor is used to send publish a message to an MQTT topic.
• Set the path URL of MQTT server in broker URI field, any name in the Client ID, and Topic field. Set QoS as 1 and retain message as true. Refer to the following screenshot.

Step 6

• Drag the processor ListenHttp processor and configure the properties of base path and listening port.  Configure the controller service as of StandardRestrictedSSLContextService.
• Base path name is represented by any name and port number. Refer to the following screenshot.

Step 7

Drag the PublishtoMQTT processor and configure the properties as previous except the topic field. It will be represented as unique name.

Step 8

• Drag the ConsumeMQTT processor and configure the property fields as the PublishMQTT properties except Client ID and Topic name.
• It will retrieve the data from PublishMQTT by giving respected Topic filter and client ID.

Step 9

Final processor of the PutFile processor is used to write the contents of flow file in local system. Configure the properties of output directory as follows.

Note: After configuring and routing the processor, start the processor from Data Receiver part first and then Data sender part.