Resty and JQ


Simple REST API Testing and JSON Parsing from the Shell

Posted on Wednesday, November 11, 2015 by Charles Beynon
× Charles Beynon
[Photo: Dark-haired caucasian male, mid-thirties, wearing dark-rimmed eyeglasses]

Full Stack Developer mostly working in Rails. Recovering sysadmin. I like to dabble and tend to ramble. Don't say I didn't warn you ;D

About me

Resty turns this:

$ curl -sLv  "" -X GET -b /path/to/cookie -c /path/to/cookie/jar "" "" "" \
"http://api.wunderground.com/api/API_KEY/conditions/q/NY/New_York.json"

into this

$ GET /conditions/q/NY/New_York.json

Testing an application with a REST API from the command line normally necessitates complex invocations of curl, especially when using verbs other than the basic GET. Resty is a tool by Micha Niskin that makes it a breeze.

Installation

Resty can be installed by following the instructions from the Resty GitHub site. Alternatively, just run the following from anywhere in your path (such as ~/bin) to install the script.

$ curl -O https://raw.githubusercontent.com/micha/resty/master/resty
$ chmod +x resty

We need to initiate Resty before using it, so either add the following to the end of your .zshrc or just run it in your shell session (don’t forget the space in the middle)

$ resty

A Quick Fix for Zsh

Note for zsh users: there is a bug in the current master branch with the lines split by new lines. Running the following replacement from Vim fixed it for me. There is a commit on the GitHub issues which seems to fix it, and if you’re reading this significantly after November 2015 chances are the fix is already merged.

%s/\\\n\s\+//

Pointing to a base URL

Resty needs to be pointed to a base URL in order to use it. The example I gave above is from Weather Underground: you can grab a Free API Key to follow along.

Now that you have your key, run the following from the shell:

$ resty http://api.wunderground.com/api/<APIKEYHERE>
http://api.wunderground.com/api/<APIKEYHERE>

At any time, running the resty command will display the base URL, which is handy if you switch between APIs often. Just keep this in mind if you store sensitive data (like API keys) there, it will be easily visible to anyone else on the console.

Testing it out

Now we can run some simple queries:

$ GET /conditions/q/NY/New_York.json

If you’re using Resty with your own small API, you may be fine. However, Rails (and perhaps oter frameworks) minifies JSON by default, making it difficult to parse. For larger and espcially large third-party APIs like Weather Underground’s, you’ll get a huge JSON document in response.

So we’ve got the requests made easy, but what can we do to tame the responses?

Using jq to parse json data from Resty

If you don’t already have jq installed, now’s a good time. I’m going to write up another post in more detail on jq, but for now we’ll using it simply for syntax coloring, prettification, and pulling out single values

$ brew install jq

Now we can use jq to pick out the parts we’re interested in.1

$ GET /conditions/q/NY/New_York.json | jq .current_observation.observation_location
{
  "full": "Chelsea, West 28th St., New York, New York",
  "city": "Chelsea, West 28th St., New York",
  "state": "New York",
  "country": "US",
  "country_iso3166": "US",
  "latitude": "40.746910",
  "longitude": "-73.992836",
  "elevation": "110 ft"
}
$ GET /conditions/q/NY/New_York.json | jq .current_observations.temp_f
52

On a smaller API (such as the Twitter-like clone I’m building) that minifies by default, you can use jq to make prettify the text, and give it syntax coloring.

$ GET /user/1.json
{"user":{"id":1,"email":"test@example.com","created_at":"2015-11-09T08:16:17.129
Z","updated_at":"2015-11-11T18:18:35.737Z","name":"Example User","username":"Som
eGuy","gravatar_url":"https://secure.gravatar.com/avatar/<somehash>.png?r=PG"}}
$ GET /user/2.json | jq
{
  "user": {
    "id": 1,
    "email": "test@example.com",
    "created_at": "2015-11-09T08:16:17.129Z",
    "updated_at": "2015-11-11T18:18:35.737Z",
    "name": "Example User",
    "username": "SomeGuy",
    "gravatar_url": "https://secure.gravatar.com/avatar/<somehash>.png?r=PG"
  }
}

This is particularly noticeabl when returning arrays of objects, which quickly become unreadable by humans when minified.

Combining Resty and jq is an extremely powerful way to test your JSON APIs, as well as to experiment with third-party ones.

  1. Be careful when experimenting you don’t go over your API call quota, espcially when using an external API with a free account. You may want to cache the results from Resty to a file, then cat those into jq 


Comments