Using Golang to Interact with REST API's

I've become fairly addicted to Golang for the last several months now and I think there's no better application to show why than how you can interact with REST API's with Golang. This article will use the DigitalOcean API with some test data I have present on my DigitalOcean account. If you follow along, you're not going to get the same results I do, and you'll have to spin up a droplet or two on your own account to see results.

Let's get started!

Basic Setup

I spun up a new DigitalOcean API key on my account for demo purposes and dropped it in a .env file. We will load that .env variable at the beginning of our main function:

func main() {
	err := godotenv.Load()
	if err != nil {
		fmt.Errorf("There was an error loading the dotenv file: %s\n", err.Error())
	}
	DOAPIKEY := os.Getenv("DOAPIKEY")
}

Now that we have our API key loaded, let's create the basic URL path we will want to hit to list all of our droplets. If we look at the DigitalOcean API docs, we see it is at the path /v2/droplets/. We will use the URL builder to create this path alongside a BASEURL constant.

const BASEURL = "https://api.digitalocean.com"

func main() {
	err := godotenv.Load()
	if err != nil {
		fmt.Errorf("There was an error loading the dotenv file: %s\n", err.Error())
	}
	DOAPIKEY := os.Getenv("DOAPIKEY")

	dropleturl, err := url.JoinPath(BASEURL, "/v2/droplets")
	if err != nil {
		fmt.Errorf("There was an error trying to create the droplet URL: %s\n", err.Error())
	}
}

Now that we have our URL and our API key, we can create a makeReq() function that will return a properly formatted http.Request{} object with all of the auth headers and request method set.

func makeReq(method string, apikey string, url string) (*http.Request, error) {
	req, err := http.NewRequest(method, url, nil)
	if err != nil {
		return &http.Request{}, err
	}
	req.Header.Add("Authorization", "Bearer "+apikey)
	req.Header.Add("Content-Type", "application/json")
	return req, nil
}

This request will be a GET request, so when we call the function to get our formatted request back, we will pass in "GET" as the method and then pass the formatted http.Request{} object to an HTTP client to service it.

func main() {
	err := godotenv.Load()
	if err != nil {
		fmt.Errorf("There was an error loading the dotenv file: %s\n", err.Error())
	}
	DOAPIKEY := os.Getenv("DOAPIKEY")

	dropleturl, err := url.JoinPath(BASEURL, "/v2/droplets")
	if err != nil {
		fmt.Errorf("There was an error trying to create the droplet URL: %s\n", err.Error())
	}
	fmt.Println(dropleturl)
	req, err := makeReq("GET", DOAPIKEY, dropleturl)
	client := http.Client{}
	res, err := client.Do(req)
	if err != nil {
		fmt.Errorf("There was an error doing the request: %s", err.Error())
	}
}

Now, here is where GoLang gets awesome...

Typecasting and Unmarshaling

After we run the request, we get an http.Response{} object, which isn't particularly useful to us. We can access the body of the response through res.Body using the ioutil.ReadAll() method, but that is just a byte array. We can convert that byte array to a string, but then we just have a big nasty JSON blob string which, again, not super useful.

The way GoLang handles this is beautiful. You create structs with the format of your data types described as struct members. Each member will have a tag that tells GoLang how to parse JSON data into that struct. Then you just unmarshal a byte array into your custom-made structs and you've got beautifully parsed data.

If it doesn't parse correctly, your fields just won't be propagated, so you'll have null values in those struct fields. What this means is that if, say, a JSON blob has a value that may or may not be set or a field that may or may not be present, you can define it in your struct and, if it's not present, GoLang just doesn't set it in the object.

Let's show an example. Our DigitalOcean response value is going to be structured as so:

{
	"droplets":[
    	{
        	"id":123456,
            "value2":"",
            "value3":"",
            ...
			"image":{},
            ...
        }
    ]
}

Basically, you'll have a droplets array that will hold 0 or more droplets, each of which is a blob of strings, integers, and nested objects. Critically, we don't have to care about every single field in the object! Just the ones we care about. In this example, I want to pull out the name field, the id field, the created_at field and the id and name of the image associated with the droplet.

First, I'll create a struct whose sole purpose is to parse out the droplets array.

type Droplets struct {
	Drops []Droplet `json:"droplets"`
}

The type Droplets struct tells GoLang we're constructing a struct named Droplets. Then we define a field named Drops and declare it as an array of Droplet type, which is a struct we're going to define below. The part in between the back-ticks, json:"droplets" is the tag that tells GoLang "If you're trying to parse a JSON blob into this structure, the Drops field will be defined in the droplets field in the JSON blob."

Next, we create our Droplet  struct.

type Droplet struct {
	Id        int    `json:"id"`
	Name      string `json:"name"`
	CreatedAt string `json:"created_at"`
	Image     Img    `json:"image"`
}

All of this is the same as above except for the Image field. You have your field name, like Id, Name, etc., the types of said fields, and the tag to parse JSON from the fields within the JSON blob. The Image field is of type Img, which is another custom struct defined as so:

type Img struct {
	Id   int    `json:"id"`
	Name string `json:"name"`
}

So, to access the Image ID on a Droplet object named drop, we would run:

fmt.Println(drop.Img.Id)

Now let's parse over all of our droplets returned from the DigitalOcean API. First, we will create an empty Droplets object. Then, we will parse (or unmarshal) our response body into that Droplets object. Finally, we will for loop over the array of droplets and print out the information.

func main() {
	err := godotenv.Load()
	if err != nil {
		fmt.Errorf("There was an error loading the dotenv file: %s\n", err.Error())
	}
	DOAPIKEY := os.Getenv("DOAPIKEY")

	dropleturl, err := url.JoinPath(BASEURL, "/v2/droplets")
	if err != nil {
		fmt.Errorf("There was an error trying to create the droplet URL: %s\n", err.Error())
	}
	fmt.Println(dropleturl)
	req, err := makeReq("GET", DOAPIKEY, dropleturl)
	client := http.Client{}
	res, err := client.Do(req)
	if err != nil {
		fmt.Errorf("There was an error doing the request: %s", err.Error())
	}
	resbody, err := ioutil.ReadAll(res.Body)
	if err != nil {
		fmt.Errorf("There was an error reading the response body: %s", err.Error())
	}
	//fmt.Println(string(resbody))
	drops := Droplets{}
	err = json.Unmarshal(resbody, &drops)
	if err != nil {
		fmt.Errorf("There was an error unmarshaling the struct: %s", err.Error())
	}
	for _, drop := range drops.Drops {
		fmt.Printf("Droplet ID %d\n\tName: %s\n\tCreated At: %s\n", drop.Id, drop.Name, drop.CreatedAt)
		fmt.Printf("\tImage info:\n\t\tImage ID: %d\n\t\tImage name: %s\n\n", drop.Image.Id, drop.Image.Name)
	}
}

Why this rules

GoLang has created what is probably the best possible approach to parsing out JSON data. You declare your data structures in a way that is both strict enough to parse out what you need but loose enough to deal with structures that have flexible field definitions. If, for whatever reason, the name field was not set in the example above for a given droplet, that's no big deal, it will just print an empty string for the name field. If the droplets field were empty (as in I didn't have any droplets created) there wouldn't be a critical error that crashes the program, I would just have an empty array and the for loop wouldn't start.

This creates an awesome developer experience for parsing REST API data and is the primary reason I'm stuck on GoLang now.