Atchareeya Jattuporn
- Dec 1, 2020
- 5 min read

ITP Weather Band : API

Updated: Dec 7, 2020

For the final project, Cy, Yiting, and I, with a lot of support from Yeseul, collaborated to rebuild the database and server system in ITP remote server and website for the ITP weather band.

The ITP weather band is part of ITPower, a research group exploring creative uses of technology for a sustainable future. In the weather band, we are creating a musical instrument from environmental data we collected using our DIY weather station and Arduino MKR Env Shield. All of us are working on it since last year, mostly on the instrument part.

Cy's blog post about the databases

Yiting's blog post about the front-end

System Diagram

How to use the API

Our website: http://weatherband.itp.io:3000/

Get all data from the station

We are planning to install multiple weather stations in multiple countries. The user will be able to get data from each station by specifying the mac address of the station.

Parameter: New York station's mac address : A4:CF:12:8A:C8:24

URL:  http://weatherband.itp.io:3000/data/all?macAddress=A4:CF:12:8A:C8:24
cURL command :  
curl -X GET -d macAddress=A4:CF:12:8A:C8:24 http://weatherband.itp.io:3000/data/all

Get data by transaction ID

There will be an ID for all the data points we posted. The user can specify the ID to get all the environmental data of that ID

URL: http://weatherband.itp.io:3000/data/id/:transactionID
cURL command:
curl -X GET http://weatherband.itp.io:3000/data/id/:transactionID

Get data by category

The user also can get the data by category of the environmental data which are wind direction, wind speed, rain, temperature, humidity, illuminate, UVA, UVB, UV index (checkout how to refer to the category in our GitHub)

The endpoint can take a single category (eg. wind_dir )and a comma-separated list of categories (eg. wind_dir,rainin).

URL: http://weatherband.itp.io:3000/data/by-cat?macAddress=A4:CF:12:8A:C8:24&cat=wind_dir,rainin
cURL command:
curl -X GET -d macAddress=A4:CF:12:8A:C8:24 -d cat=wind_dir http://weatherband.itp.io:3000/data/by-cat

Get data by date

The user also can specify the starting date and the ending date to get the data in the date range. The format of the date would be "YYYY-MM-DD-HH:MM:SS" (checking moment.js parsing documentation)

URL: http://weatherband.itp.io:3000/data/date?macAddress=A4:CF:12:8A:C8:24&from='2020-11-27'&to='2020-11-28'
cURL command:
curl -X GET -d macAddress=A4:CF:12:8A:C8:24 -d from='2020-11-25' -d to='2020-11-27' http://weatherband.itp.io:3000/data/date

Post data

The user will need a session key to post the data. They have to send us their mac address and we will send them the verified session key. We used randomkeygen website to generate the random session key for each station.

Link to Github

Building and API

In order to create a server, we've got a lot of support from Yen, ITP network, and web administrator. He provided a digital ocean server for us under ITP's account. We decided to build a Node.js server instead of a PHP server like Yen recommended just because most of us are familiar with Node and we can get a lot of support from the ITP community.

After getting the server up and running, Yeseul, Cy, and I installed the node and set up the server together following this tutorial. When everything is ready, Cy is responsible to create MySQL table (tutorial she used) and I'm responsible to build an API to get and post data to the database. Yiting will working on the front end and the web client after the API is ready.

I started with researching and trying to understand Brent's API code from last year and Don's example for connected devi ces. What I decided to adjust from the previous version is the way to verifying the mac address and session key.

In the previous version, everyone needed to have a verified mac address and session key for getting and posting weather data which is slightly insecure because everyone has the verified mac address and the session key to manipulate the database. So, in this version, I decided to verify the session key only when posting data. In this case, only the posting people will have the session key which will authenticate posting data to the database. In order to do that, I only verify the mac address and session key under API for the post, instead of verifying it at the beginning of the code.

After finished testing, we run our server.js file on PM2 to keep it running in the background.

Bug and Debug

MySQL authentication

When I'm running the server for the first time. I got this error message.

Error: ER_NOT_SUPPORTED_AUTH_MODE: Client does not support authentication protocol requested by server; consider upgrading MySQL client

Cy, Yiting, Yeseul, and I were trying to debug together with a lot of help from Tito, we found this solution on stackoverflow.com. This problem happened because of the version of MySQL. Version 8.0 caused the authentication problem. What we need to do is to alter the privileges of the user.

solution : ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password'

.env File

Another part that I struggle with is to find the correct data to field in the env file. In Brent's example, he hid the data in the .env file which is a secret file. I worked with Cy and Tito to find the correct data to fill.

req.body vs req.query

When I finished writing the API code, there were two endpoints that are not working with the URL. When I enter the information in the URL, it did not pass the data to the API code.

After consulting with Tom, I found out that in HTTP Get request, we cannot have a request body, but we need a request query instead. So what I did in the code instead of using req.body.data, I change it to req.query.data.

What is query string URL

After finish writing all the APIs, I tested it with the curl command and every API is working totally fine. However, Yeseul needed the URL to put in the p5 example sketch to get the data, so I have to find a way to get the correct URL for all the endpoints.

I did some research and found out that it called "query string URL". The query string is part of the URL that assigns values to specified parameters. To create a query string, it needed to have a specific syntax.

image from Caspio

However, after consulting with Tom, he suggested this is not the common way to do the HTTP request because normally we can not add body to the GET request. If we want to add the body to the request the method should be PUT or POST instead.

The benefit of doing PUT request instead of GET request is to get the shorter and memorizable URL. Also, the user will be able to pick which data they what to enter, for example, instead of having 4 GET API endpoints for users to pick which one to use, we can have one PUT API and the user can manipulate which parameter they want to use.

A good example would be Philips hue API.

Node.js package that I used

body-parser > to get HTTP POST body in express.js

MySQL > to access MySQL library

cors > to provide express/connection

moment > to format the date and time format

fs > to create a file system

morgan > to create a log file

path > to creat a path for log file

Reference

Building an API with Node and Express: Programming A-Z

MySQL Node Express API - Walkthrough