Building your first Data app

For the purposes of this tutorial, we are going to develop a Gtmhub Data app that connects to swapi.dev - the official Star Wars API. This will enable us to pull some important Star Wars data over and work with it in Gtmhub. Following the tutorial in ths section, requires anything between 15-30 minutes.

The essential part of each Data app is the logic that instructs Gtmhub to connect to the 3rd party system and understand its data types. The first steps towards developing a Data app are:

  • specify the Data app kind
  • specify your API base URL
  • add a test connection call
  • define the data types returned by your REST API
  • add download instructions so your data types can be persisted in the Gtmhub Data warehouse

For this tutorial we’ll use the https://swapi.dev/api/planets endpoint from the Star Wars API. Here's a 10min video walking you through the entire process.

Specify kind

Currently the only supported dat app kind is rest. Your Data App YAML file must begin by adding the following:

kind: rest

This instruct Gtmhub to expect a data connector that uses a REST API, and use your described RESt API endpoints to make the calls that fetch the data.

Specify base URL

The base URL is specified only once in your YAML file. It's used to construct the full path to your REST API endpoints when making requests. For example, with the Star Wars API, we know that our API is located at https://swapi.dev/api so this as our baseUrl. You must specify it by setting the baseUrl property on a root level in the Data app YAML file like this:

baseUrl: "https://swapi.dev/api"

Add test connection

If your Data app exposes any setup parameters that users must configure when setting up the app, Gtmhub enables users to verify whether the provided parameters result in a successful connection. This is useful for early problem discovery as well as preventing later data loss.

To add a test connection call, you must specify a URL where your app makes a test request and verifies if it’s successful. The criterion for success is a response status code between 200 and 399.

If your REST API does not provide a specific test endpoint, you can use any available endpoint that you consider a suitable test. For the purposes of this tutorial, the Star Wars API does not expose a test connection endpoint. That's why we'll just use the /planets one. Describe the API call to your test connection endpoint inside a connectionTest object on the root level in your YAML file. It must have the following data:

  • uri - this is always the uri to the specific endpoint. It is appended to the baseUrl you specified above in the YAML file.
  • verb - the HTTP method (verb) to be used
  • headers - add all the necessary HTTP Headers for this request

In the end, your connectionTest implementation should look something like this:

connectionTest:
  uri: "/planets"
  verb: "GET"
  headers:
    Accept: "application/json"

NOTE: When making a test request Gtmhub combines the value of the baseUrl property and the value of the uri property. That’s why the value of uri above is just /planets.