For the final project, Cy, Yiting, and I, with a lot of support from Yeseul, collaborated to rebuild the database and server system in ITP remote server and website for the ITP weather band.
The ITP weather band is part of ITPower, a research group exploring creative uses of technology for a sustainable future. In the weather band, we are creating a musical instrument from environmental data we collected using our DIY weather station and Arduino MKR Env Shield. All of us are working on it since last year, mostly on the instrument part.
System Diagram
How to use the API
Our website: http://weatherband.itp.io:3000/
Get all data from the station
We are planning to install multiple weather stations in multiple countries. The user will be able to get data from each station by specifying the mac address of the station.
Parameter: New York station's mac address : A4:CF:12:8A:C8:24
URL: http://weatherband.itp.io:3000/data/all?macAddress=A4:CF:12:8A:C8:24
cURL command :
curl -X GET -d macAddress=A4:CF:12:8A:C8:24 http://weatherband.itp.io:3000/data/allGet data by transaction ID
There will be an ID for all the data points we posted. The user can specify the ID to get all the environmental data of that ID
URL: http://weatherband.itp.io:3000/data/id/:transactionID
cURL command:
curl -X GET http://weatherband.itp.io:3000/data/id/:transactionIDGet data by category
The user also can get the data by category of the environmental data which are wind direction, wind speed, rain, temperature, humidity, illuminate, UVA, UVB, UV index (checkout how to refer to the category in our GitHub)
The endpoint can take a single category (eg. wind_dir )and a comma-separated list of categories (eg. wind_dir,rainin).
URL: http://weatherband.itp.io:3000/data/by-cat?macAddress=A4:CF:12:8A:C8:24&cat=wind_dir,rainin
cURL command:
curl -X GET -d macAddress=A4:CF:12:8A:C8:24 -d cat=wind_dir http://weatherband.itp.io:3000/data/by-catGet data by date
The user also can specify the starting date and the ending date to get the data in the date range. The format of the date would be "YYYY-MM-DD-HH:MM:SS" (checking moment.js parsing documentation)
URL: http://weatherband.itp.io:3000/data/date?macAddress=A4:CF:12:8A:C8:24&from='2020-11-27'&to='2020-11-28'
cURL command:
curl -X GET -d macAddress=A4:CF:12:8A:C8:24 -d from='2020-11-25' -d to='2020-11-27' http://weatherband.itp.io:3000/data/datePost data
The user will need a session key to post the data. They have to send us their mac address and we will send them the verified session key. We used randomkeygen website to generate the random session key for each station.
Building and API
In order to create a server, we've got a lot of support from Yen, ITP network, and web administrator. He provided a digital ocean server for us under ITP's account. We decided to build a Node.js server instead of a PHP server like Yen recommended just because most of us are familiar with Node and we can get a lot of support from the ITP community.
After getting the server up and running, Yeseul, Cy, and I installed the node and set up the server together following this tutorial. When everything is ready, Cy is responsible to create MySQL table (tutorial she used) and I'm responsible to build an API to get and post data to the database. Yiting will working on the front end and the web client after the API is ready.
I started with researching and trying to understand Brent's API code from last year and Don's example for connected devices. What I decided to adjust from the previous version is the way to verifying the mac address and session key.
In the previous version, everyone needed to have a verified mac address and session key for getting and posting weather data which is slightly insecure because everyone has the verified mac address and the session key to manipulate the database. So, in this version, I decided to verify the session key only when posting data. In this case, only the posting people will have the session key which will authenticate posting data to the database. In order to do that, I only verify the mac address and session key under API for the post, instead of verifying it at the beginning of the code.
After finished testing, we run our server.js file on PM2 to keep it running in the background.
Bug and Debug
MySQL authentication
When I'm running the server for the first time. I got this error message.
Error: ER_NOT_SUPPORTED_AUTH_MODE: Client does not support authentication protocol requested by server; consider upgrading MySQL client
Cy, Yiting, Yeseul, and I were trying to debug together with a lot of help from Tito, we found this solution on stackoverflow.com. This problem happened because of the version of MySQL. Version 8.0 caused the authentication problem. What we need to do is to alter the privileges of the user.
solution : ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'password'
.env File
Another part that I struggle with is to find the correct data to field in the env file. In Brent's example, he hid the data in the .env file which is a secret file. I worked with Cy and Tito to find the correct data to fill.
req.body vs req.query
When I finished writing the API code, there were two endpoints that are not working with the URL. When I enter the information in the URL, it did not pass the data to the API code.
After consulting with Tom, I found out that in HTTP Get request, we cannot have a request body, but we need a request query instead. So what I did in the code instead of using req.body.data, I change it to req.query.data.
What is query string URL
After finish writing all the APIs, I tested it with the curl command and every API is working totally fine. However, Yeseul needed the URL to put in the p5 example sketch to get the data, so I have to find a way to get the correct URL for all the endpoints.
I did some research and found out that it called "query string URL". The query string is part of the URL that assigns values to specified parameters. To create a query string, it needed to have a specific syntax.
However, after consulting with Tom, he suggested this is not the common way to do the HTTP request because normally we can not add body to the GET request. If we want to add the body to the request the method should be PUT or POST instead.
The benefit of doing PUT request instead of GET request is to get the shorter and memorizable URL. Also, the user will be able to pick which data they what to enter, for example, instead of having 4 GET API endpoints for users to pick which one to use, we can have one PUT API and the user can manipulate which parameter they want to use.
A good example would be Philips hue API.
Node.js package that I used
body-parser > to get HTTP POST body in express.js
MySQL > to access MySQL library
cors > to provide express/connection
moment > to format the date and time format
fs > to create a file system
morgan > to create a log file
path > to creat a path for log file
Reference