Accurate weather information and forecasts are essential elements in critical decision-making scenarios. Both enterprises and individual consumers rely on current and future weather patterns to run their daily operations and manage risks successfully.
As such, the increased demand for accurate weather information has pushed the developer community to invest in building reliable weather apps.
So in this workshop we're going to:
use a weather API.
pull the data into PDI.
display dataset in a CDE dashboard - Pentaho Server service.
The following section is for Reference only.
The CDE has been activated.
You may require to create an Openweathermap account.
Follow the steps outlined below to activate CDE and test the API call with Postman.
To activate the Community Dashboard Editor (CDE) plugin, you will need to change the configuration of several .xml files in the Pentaho solutions folder as described below. Verify that you have the appropriate permissions to read, write, and execute commands in the specified directories in the instructions.
Ensure the Pentaho Server is stopped.
cd
cd /opt/pentaho/server/pentaho-server
sudo ./stop-pentaho.sh
Postman is an API platform for building and using APIs.
Sign up - for free - & download Postman.
You can also install Postman as a VS Extension or from Snap.
Once logged into your account you can test your API weather call and test the output of various parameters.
The new 'One Call' API requires the location latitude & logitude.
This method usues a deprecated Geocoding API to resolve city, state, country - ISO 3166.
Query Params
Value
https://api.openweathermap.org/data/2.5/weather?
q=
{city name}
london
{state code}
England
{country code}
GB
appid
{API key}
Optional Params
Edit the query to set the mode & units.
Params
Description
mode
Possible values are xml and html. If you don't use the mode parameter format is JSON by default.
units
Units of measurement. standard, metric and imperial units are available. If you do not use the units parameter, standard units will be applied by default.
lang
You can use this parameter to get the output in your language.
As you already know, Kettle variables can be used in Jobs and Transformations wherever you see a dollar sign. There is one more way to use variables—getting their values as fields of your stream. For this, you have to use the Get Variables step.
With the Get Variables step, you can get the value for one or more variables. The kind of variable can be any of the Kettle variables types you just learned the variables defined in the kettle.properties file, internal variables, for example, ${user.dir}, named parameters, or other Kettle variables.
Double-click on the step to view settings.
The variables & default values are pulled from 'Named Parameters' set in the Transformation properties.
The REST Client transformation step enables you to consume RESTfull services.
For a simple http GET request you can also use: http client step.
Double-click on the step to view settings.
Based on the openweathermap API documentation:
• the URL calls the service to return the weather data based on:
${city}=London
${country}=GB
${units}=metric
${mode}=xml
• 'response' is return to the 'results' data stream field.
• the fields are 'mapped' in the fileds tab.
This step provides the ability to read data from any type of XML file using XPath specifications.
The response from the API call - results - is the data source.
Click on the 'Content' tab.
To determine the XPath, you will need to copy and paste a snippet of the API call.
The root is: /current
Click on Fields tab.
The Row Normaliser step converts the columns of an input stream into rows. You can use this step to normalize repeating groups of columns.
Double-click on the step to view settings.
The Select values step can perform all the following actions on fields in the PDI data stream:
• Select values
• Remove values
• Rename values
• Change data types
• Configure length and precision of values
x
x
x
Community Data Access (CDA) allows you to gather and combine data from several data sources into a single structure, which you can then use in dashboards. Driven by the need to unify access to the Pentaho data layer, CDA was developed to create an abstraction layer between a CTools dashboard and the physical connections to different databases.
CDA has three main objectives:
Combine data from several sources
Ensure security while accessing the data (avoiding, for instance, SQL injection problems)
