Create dashboard for Hive data source using Data Integration Platform

Introduction:

This article explains how to move data from Hive table to SQL table using Syncfusion Data Integration Platform to create dashboards.

Steps Involved:

Using the Syncfusion Data Integration Platform:

1. To fetch the data from Hive table.
2. Clean and process the Avro data into JSON attributes.
3. Move the final processed data into a SQL database.

Using the Syncfusion dashboard:

4. Create a dashboard to showcase the Patient health summary.

Data in Hive table:

For steps 1 to 3, we will be defining a data-flow in the Data Integration Platform as shown in the following image.

       Data flow in Data Integration Platform

Step 1: Fetch the data from Hive table

Drag the SelectHiveQL processor into the canvas and create the HiveConnectionPool service. Specify a query to be executed on “HiveQL Select Query” property and configure it as follows:

Provide the JDBC Connection URL as given below for connecting with the hive databases running on Syncfusion Big Data Platform.

JDBC URL: jdbc:hive2://localhost:10000/default;auth=noSasl

For Hive Configuration Resources property, provide the file location of core-site.xml and hive-site.xml present in the installed location of Syncfusion Big Data Studio.

C:\Syncfusion\BigData\3.2.0.20\BigDataSDK\SDK\Hadoop\etc\hadoop\core-site.xml

C:\Syncfusion\BigData\3.2.0.20\BigDataSDK\SDK\Hive\conf\hive-site.xml

Step 2: Clean and process the Avro data into JSON attributes

Drag the ConvertRecord processor into the canvas and configure as follows to convert the Avro data to Json data.

Configure the AvroReader controller service as follows:

Configure the JsonRecordSetWriter controller service as follows:

Step 3: Move the final processed data into a SQL database

Drag the PutDatabaseRecord processor and configure as follows to move the data into the created SQL table.

Configure the JsonTreeReader controller service as follows:

Enable all the created controller services as below:

Step 4: The final step is to create a Patient Health Summary dashboard like the following in the Syncfusion Dashboard Cloud.

To learn the basics of creating a dashboard, refer to these links:

• Create business dashboards online
• Creating a sales dashboard with SQL Server and Syncfusion dashboards

You can follow the steps covered in these links to create your dashboards easily.

                                                            Patient Health Summary Dashboard