RESTful API

Using weather REST API ..

Use Case - Weather API

Accurate weather information and forecasts are essential elements in critical decision-making scenarios. Both enterprises and individual consumers rely on current and future weather patterns to run their daily operations and manage risks successfully.

As such, the increased demand for accurate weather information has pushed the developer community to invest in building reliable weather apps.

So in this workshop we're going to:

use a weather API.
pull the data into PDI.
display dataset in a CDE dashboard - Pentaho Server service.

The following section is for Reference only.

The CDE has been activated.

You may require to create an Openweathermap account.

Follow the steps outlined below to activate CDE and test the API call with Postman.

To activate the Community Dashboard Editor (CDE) plugin, you will need to change the configuration of several .xml files in the Pentaho solutions folder as described below. Verify that you have the appropriate permissions to read, write, and execute commands in the specified directories in the instructions.

Ensure the Pentaho Server is stopped.

cd
cd /opt/pentaho/server/pentaho-server
sudo ./stop-pentaho.sh

Uncomment the following lines:

/opt/pentaho/server/pentaho-server/pentaho-solutions/system/pentaho-cdf-dd/plugin.xml

cd
cd /opt/pentaho/server/pentaho-server/pentaho-solutions/system/pentaho-cdf-dd
sudo nano plugin.xml

Locate the following two commented blocks in this file and remove the comment tags from these blocks.

<operation>
    <id>EDIT</id>
    <perspective>wcdf.edit</perspective>
</operation>

<overlays>
    <overlay id="launch" resourcebundle="content/pentaho-cdf-dd/lang/messages">
        <button id="launch_new_cde" label="${Launcher.CDE}" command="Home.openFile('${Launcher.CDE}', '${Launcher.CDE_TOOLTIP}', 'api/repos/wcdf/new');$('#btnCreateNew').popover('hide');"/>
    </overlay>
    <overlay id="startup.cde_dashboard"  resourcebundle="content/pentaho-cdf-dd/lang/messages" priority="1">
        <menubar id="newmenu">
            <menuitem id="new-cde_dashboard" label="${Launcher.CDE}" command="mantleXulHandler.openUrl('${Launcher.CDE}','${Launcher.CDE_TOOLTIP}','api/repos/wcdf/new')" />
        </menubar>
    </overlay>
</overlays>

Save.

CTRL + O
Enter
CTRL + X

Uncomment the following lines:

/opt/pentaho/server/pentaho-server/pentaho-solutions/system/pentaho-cdf-dd/settings.xml

cd
cd /opt/pentaho/server/pentaho-server/pentaho-solutions/system/pentaho-cdf-dd
sudo nano settings.xml

Locate the block at end of the file, 'Defining the new-toolbar-button' and remove the comments tags from this block

<new-toolbar-button>1,New CDE Dashboard,CDE Dashboard,api/repos/wcdf/new</new-toolbar-button>

Locate the block at end of the file, 'Defining the new-toolbar-button' and remove the comments tags from this block.
Save.

CTRL + O
Enter
CTRL + X

Save your changes.
Restart the Pentaho Server.

cd
cd /opt/pentaho/server/pentaho-server
sudo ./start-pentaho.sh

Community Dashboard Editor (CDE)

To verify CDE is activated, do the following.

Log on to the Pentaho User Console.
From the Home page, click the Create New button.
From the menu that displays, select the New CDE Dashboard option. You can now begin creating your first CDE dashboard.

Once registered, generate an API key.

Read the API documentation for deployment - Geocoding

http://api.openweathermap.org/geo/1.0/direct?q={city name},{state code},{country code}&limit={limit}&appid={API key}

Parameters

Description

q

required

City name, state code (only for the US) and country code divided by comma. Please use ISO 3166 country codes.

appid

required

Your unique API key (you can always find it on your account page under the "API key" tab)

limit

optional

Number of the locations in the API response (up to 5 results can be returned in the API response)

Example

http://api.openweathermap.org/geo/1.0/direct?q=London&limit=5&appid={API key}

Postman is an API platform for building and using APIs.

Postman

You can also install Postman as a VS Extension or from Snap.

Once logged into your account you can test your API weather call and test the output of various parameters.

The new 'One Call' API requires the location latitude & logitude.

This method usues a deprecated Geocoding API to resolve city, state, country - ISO 3166.

Query Params

Value

https://api.openweathermap.org/data/2.5/weather?

{city name}

london

{state code}

England

{country code}

appid

{API key}

Optional Params

Edit the query to set the mode & units.

Params

Description

mode

Possible values are xml and html. If you don't use the mode parameter format is JSON by default.

units

Units of measurement. standard, metric and imperial units are available. If you do not use the units parameter, standard units will be applied by default.

lang

You can use this parameter to get the output in your language.

https://api.openweathermap.org/data/2.5/weather?q={city name},{state code},{country code}&appid={API key}&mode=xml&units=metric

Resources

REST API documentation API information for building custom solutions of Pentaho products.

CDF API Pentaho Data Integration and Analytics CDF API documentation.

Pentaho Data Integration and Analytics Platform JavaScript API.

The CDE is a Pentaho service.

Weather API

Start Pentaho Pro Suite.

cd
cd /Scripts
./start_pro.sh

Open the following transformation in Pentaho Data Integration.

~/Workshop--Data-Integration/Labs/Module 7 - Workflows/API/tr_weather_cda.ktr

Browse through the following steps.

As you already know, Kettle variables can be used in Jobs and Transformations wherever you see a dollar sign. There is one more way to use variables—getting their values as fields of your stream. For this, you have to use the Get Variables step.

With the Get Variables step, you can get the value for one or more variables. The kind of variable can be any of the Kettle variables types you just learned the variables defined in the kettle.properties file, internal variables, for example, ${user.dir}, named parameters, or other Kettle variables.

Double-click on the step to view settings.

The variables & default values are pulled from 'Named Parameters' set in the Transformation properties.

Community Data Access (CDA) allows you to gather and combine data from several data sources into a single structure, which you can then use in dashboards. Driven by the need to unify access to the Pentaho data layer, CDA was developed to create an abstraction layer between a CTools dashboard and the physical connections to different databases.

CDA has three main objectives:

Combine data from several sources
Ensure security while accessing the data (avoiding, for instance, SQL injection problems)
Ease data exports

View the following CDA file.

~/Workshop--Data-Integration/

<?xml version="1.0" encoding="UTF-8"?>
<CDADescriptor>
   <DataSources>
      <Connection id="weather" type="kettle.TransFromFile">
         <KtrFile>tr_weather_cda.ktr</KtrFile>
         <variables datarow-name="city"/>
         <variables datarow-name="country"/>
       </Connection>
   </DataSources>
   
   <DataAccess access="public" cache="true" cacheDuration="3600" connection="weather"
               id="current"
               type="kettle">
      <Columns/>
      <Parameters>
         <Parameter default="London" name="city" type="String"/>
         <Parameter default="GB" name="country" type="String"/>
      </Parameters>
      <Query>conditions_normalized</Query>
      <Output indexes="0,1"/>
   </DataAccess>
   
 </CDADescriptor>

PreviousCredit Card NextJenkins

Last updated 2 months ago

Was this helpful?