HTTP Requests

HTTP Requests

In Go, we can use the net/http package to make http requests.

We will use this public endpoint to make a get request: https://jsonplaceholder.typicode.com/posts

GET

To perform a get operation, we can simply pass the url to the get function from the http package.

package main

import (
	"io"
	"log"
	"net/http"
)

func main() {
	resp, err := http.Get("https://jsonplaceholder.typicode.com/posts")
	if err != nil {
		log.Fatalln(err)
	}

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatalln(err)
	}

	stringBody := string(body)
	log.Printf(stringBody)
}

In the example above, we make the request, then read the byte slice inside the response and convert it into a string to be printed.

This will be the output in our program:

.
.
.
 {
    "userId": 1,
    "id": 4,
    "title": "eum et est occaecati",
    "body": "ullam et saepe reiciendis voluptatem adipisci\nsit amet autem assumenda provident rerum culpa\nquis hic commodi nesciunt rem tenetur doloremque ipsam iure\nquis sunt voluptatem rerum illo velit"
  },
  {
    "userId": 1,
    "id": 5,
    "title": "nesciunt quas odio",
    "body": "repudiandae veniam quaerat sunt sed\nalias aut fugiat sit autem sed est\nvoluptatem omnis possimus esse voluptatibus quis\nest aut tenetur dolor neque"
  },
.
.
.

POST

Performing a post operation is also simple, one of the easiest ways is this:

For this example, we will use the public endpoint to perform a post operation: https://postman-echo.com/post

It will return the value we pass as payload within the data field.

package main

import (
	"bytes"
	"encoding/json"
	"io"
	"log"
	"net/http"
)

func main() {
	body, _ := json.Marshal(map[string]string{
		"id":       "01",
		"category": "abc",
	})

	bodyBuffer := bytes.NewBuffer(body)

	resp, err := http.Post("https://postman-echo.com/post", "application/json", bodyBuffer)
	if err != nil {
		log.Fatalf(err.Error())
	}
	defer resp.Body.Close()

	respBody, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatalln(err)
	}

	respString := string(respBody)
	log.Printf(respString)

}

In the example above, the payload is a map of string/string.

body, _ := json.Marshal(map[string]string{
    "id":       "01",
    "category": "abc",
})

Next, we convert our payload into a pointer to a bytes.Buffer, we need to do this because the post function from the http package expects you to pass a payload value that implements the io.Reader interface.

After that, the sequence is basically the same as the previous example, and this will be the output of our program:

2025/01/04 13:05:45 {
  "args": {},
  "data": {
    "category": "abc",
    "id": "01"
  },
  "files": {},
  "form": {},
  "headers": {
    "host": "postman-echo.com",
    "x-request-start": "t1736006745.555",
    "connection": "close",
    "content-length": "28",
    "x-forwarded-proto": "https",
    "x-forwarded-port": "443",
    "x-amzn-trace-id": "Root=1-67795c59-114730206605fb0f1eb8f476",
    "content-type": "application/json",
    "accept-encoding": "gzip",
    "user-agent": "Go-http-client/2.0"
  },
  "json": {
    "category": "abc",
    "id": "01"
  },
  "url": "https://postman-echo.com/post"
}

HEADERS

In a real-world application, most of the time we will need to send headers along with the request.

Here is an example:

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"log"
	"net/http"
)

func main() {
	body, _ := json.Marshal(map[string]string{
		"id":       "01",
		"category": "abc",
	})

	bodyBuffer := bytes.NewBuffer(body)
	request, err := http.NewRequest(http.MethodPost, "https://postman-echo.com/post", bodyBuffer)
	if err != nil {
		log.Fatalf(err.Error())
	}

	request.Header.Set("Content-Type", "application/json")
	response, err := http.DefaultClient.Do(request)
	if err != nil {
		log.Fatalf(err.Error())
	}
	defer response.Body.Close()

	responseData, err := io.ReadAll(response.Body)
	if err != nil {
		log.Fatalln(err)
	}

	fmt.Println(string(responseData))
}

Now we can see that we first create a variable called request, which is a pointer to the http/Request struct.

request, err := http.NewRequest(http.MethodPost, "https://postman-echo.com/post", bodyBuffer)

We created this variable using the NewRequest function, where we pass the method, the url, and our payload as arguments.

In this way, we can now add the header to our request variable.

request.Header.Set("Content-Type", "application/json")

The rest of the execution follows the same pattern as the previous examples:

And this will be the output of our program:

{
  "args": {},
  "data": {
    "category": "abc",
    "id": "01"
  },
  "files": {},
  "form": {},
  "headers": {
    "host": "postman-echo.com",
    "x-request-start": "t1736007539.761",
    "connection": "close",
    "content-length": "28",
    "x-forwarded-proto": "https",
    "x-forwarded-port": "443",
    "x-amzn-trace-id": "Root=1-67795f73-78f91a361ceb881313a5d351",
    "content-type": "application/json",
    "accept-encoding": "gzip",
    "user-agent": "Go-http-client/2.0"
  },
  "json": {
    "category": "abc",
    "id": "01"
  },
  "url": "https://postman-echo.com/post"
}

Using structs

This is a slightly more complete example and more similar to real-world needs.

In it, we will use structs for our payload and also to receive the response from the request.

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"log"
	"net/http"
)

type MyRequest struct {
	Id       string `json:"id"`
	Category string `json:"category"`
}

type ResponseData struct {
	Id       string `json:"id"`
	Category string `json:"category"`
}

//type MyResponse struct {
//	Headers map[string]string `json:"headers"`
//	Data    struct {
//		Id       string `json:"id"`
//		Category string `json:"category"`
//	} `json:"data"`
//}

type MyResponse struct {
	Headers map[string]string `json:"headers"`
	Data    ResponseData      `json:"data"`
}

func main() {

	defaultHeaders := map[string]string{
		"Accept":       "*/*",
		"Content-Type": "application/json",
	}

	myRequest := MyRequest{
		Id:       "01",
		Category: "abc",
	}

	body, _ := json.Marshal(myRequest)

	bodyBuffer := bytes.NewBuffer(body)
	request, err := http.NewRequest(http.MethodPost, "https://postman-echo.com/post", bodyBuffer)
	if err != nil {
		log.Fatalf(err.Error())
	}

	for k, v := range defaultHeaders {
		request.Header.Add(k, v)
	}

	response, err := http.DefaultClient.Do(request)
	if err != nil {
		log.Fatalf(err.Error())
	}
	defer response.Body.Close()

	var myResponse MyResponse
	if err := json.NewDecoder(response.Body).Decode(&myResponse); err != nil {
		log.Fatal(err)
	}

	for v, k := range myResponse.Headers {
		fmt.Printf("%s: %s\n", k, v)
	}

	fmt.Printf("Id: %s - Category: %s\n", myResponse.Data.Id, myResponse.Data.Category)
}

The struct MyRequest will be our payload now.

type MyRequest struct {
	Id       string `json:"id"`
	Category string `json:"category"`
}

This part in quotes next to the fields is called struct tags. Its purpose is to specify that during the serialization/deserialization process, the field should be mapped to the value we define.

Next, we have the ResponseData and MyResponse structs:

type ResponseData struct {
	Id       string `json:"id"`
	Category string `json:"category"`
}

//type MyResponse struct {
//	Headers map[string]string `json:"headers"`
//	Data    struct {
//		Id       string `json:"id"`
//		Category string `json:"category"`
//	} `json:"data"`
//}

type MyResponse struct {
	Headers map[string]string `json:"headers"`
	Data    ResponseData      `json:"data"`
}

They also have struct tags, and they will be responsible for receiving the value of the response from our call.

Note that the ResponseData struct is used inside the MyResponse struct, this is a composition.

But also notice that we left a commented-out struct that serves the same purpose as the composition. The choice of which one to use will depend greatly on the type of need.

Here, we define our headers, using a map of string/string for this:

defaultHeaders := map[string]string{
    "Accept":       "*/*",
    "Content-Type": "application/json",
}

Next, we define the myRequest variable as an instance of MyRequest with the values of our payload.

myRequest := MyRequest{
    Id:       "01",
    Category: "abc",
}

Here, we will serialize the value in myRequest into json:

body, _ := json.Marshal(myRequest)

The process of executing the request is the same as the previous examples.

The difference now is how we handle the response.

var myResponse MyResponse
if err := json.NewDecoder(response.Body).Decode(&myResponse); err != nil {
    log.Fatal(err)
}

We deserialize the response by passing the values to the myResponse variable, which is an instance of MyResponse.

Next, we print all the headers and the Id and Category fields.

This is the output of our program:

gzip: accept-encoding
postman-echo.com: host
t1736008142.131: x-request-start
close: connection
28: content-length
https: x-forwarded-proto
application/json: content-type
443: x-forwarded-port
Root=1-677961ce-2a641835506a36864b6cd976: x-amzn-trace-id
*/*: accept
Go-http-client/2.0: user-agent
Id: 01 - Category: abc

TIMEOUT

Another very important aspect in HTTP requests is defining the maximum time we can wait for that request.

The example below shows how we can define the timeout:

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"log"
	"net/http"
	"time"
)

type MyRequest struct {
	Id       string `json:"id"`
	Category string `json:"category"`
}

type ResponseData struct {
	Id       string `json:"id"`
	Category string `json:"category"`
}

//type MyResponse struct {
//	Headers map[string]string `json:"headers"`
//	Data    struct {
//		Id       string `json:"id"`
//		Category string `json:"category"`
//	} `json:"data"`
//}

type MyResponse struct {
	Headers map[string]string `json:"headers"`
	Data    ResponseData      `json:"data"`
}

func main() {

	defaultHeaders := map[string]string{
		"Accept":       "*/*",
		"Content-Type": "application/json",
	}

	myRequest := MyRequest{
		Id:       "01",
		Category: "abc",
	}

	body, _ := json.Marshal(myRequest)

	bodyBuffer := bytes.NewBuffer(body)
	request, err := http.NewRequest(http.MethodPost, "https://postman-echo.com/post", bodyBuffer)
	if err != nil {
		log.Fatalf(err.Error())
	}

	for k, v := range defaultHeaders {
		request.Header.Add(k, v)
	}

	client := &http.Client{
		Timeout: 15 * time.Second,
	}

	response, err := client.Do(request)
	if err != nil {
		log.Fatalf(err.Error())
	}
	defer response.Body.Close()

	var myResponse MyResponse
	if err := json.NewDecoder(response.Body).Decode(&myResponse); err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Id: %s - Category: %s\n", myResponse.Data.Id, myResponse.Data.Category)
}

In the previous examples, we used the DefaultClient to make the requests.

In the above example, we created a new client so that we can define the timeout.

client := &http.Client{
    Timeout: 15 * time.Second,
}

response, err := client.Do(request)

This indicates that the total time for the request can take a maximum of 15 seconds.

In most cases, specifying just the timeout in this way is enough to keep your application healthy.

But there are cases where you need to better define the maximum wait time for each stage of the request process.

For these cases, we could configure the timeout like this:

transport := &http.Transport{
    DialContext: (&net.Dialer{
        Timeout:   5 * time.Second,
        KeepAlive: 30 * time.Second,
    }).DialContext,
    IdleConnTimeout:       90 * time.Second,
    TLSHandshakeTimeout:   10 * time.Second,
    ExpectContinueTimeout: 1 * time.Second,
}

client := &http.Client{
    Transport: transport,
    Timeout:   15 * time.Second,
}

response, err := client.Do(request)

The Dialer configures how the connections are made, with the following:

Timeout: defines the maximum time for a connection to be established. If this value is exceeded, the operation will fail.

KeepAlive: defines the maximum time a connection can remain idle before sending a keep-alive signal (to keep the connection active). This helps keep the connection open for future requests without needing to reconnect.

In the transport settings, we have:

IdleConnTimeout: defines the maximum time an idle connection can remain open. If this value is exceeded, the connection will be closed.

TLSHandshakeTimeout: defines the maximum allowed time for the handshake process when establishing secure HTTPS connections.

ExpectContinueTimeout: defines the maximum time the client waits to receive a response from the server after sending an HTTP header of type Expect: 100-Continue.

This is the output of our program:

Id: 01 - Category: abc