How to build your first web application with Go

This tutorial will take you through your first web application with Go. We’ll build a news application that utilises the News API to present news articles on a particular topic, and deploy it to a production server in the end.

You can find the complete code used for this tutorial in this GitHub repository.

Prerequistes

The only requirement for this tutorial is that you have Go installed on your computer and that you are vaguely familiar with its syntax and constructs. The version of Go that I used while building the app is also the latest one at the time of writing: 1.12.9. To view the version of Go that you have installed, use the go version command.

If you find this tutorial a bit too advanced for you, head over to my previous introductory tutorial to the language which should get you up to speed.

Get started

Clone the starter files repo on GitHub, and cd into the created directory. We have three main files: The main.go file is where we’ll write all the Go code for this tutorial. The index.html file is the template that will be sent to the browser, while the styles for the application reside in assets/styles.css.

Create a basic web server

Let’s start by creating a basic server that sends a “Hello World!” text to the browser when a GET request is made to the server root. Modify your main.go file to look like this:

package main

import (
	"net/http"
	"os"
)

func indexHandler(w http.ResponseWriter, r *http.Request) {
	w.Write([]byte("<h1>Hello World!</h1>"))
}

func main() {
	port := os.Getenv("PORT")
	if port == "" {
		port = "3000"
	}

	mux := http.NewServeMux()

	mux.HandleFunc("/", indexHandler)
	http.ListenAndServe(":"+port, mux)
}

The first line, package main declares that the code in the main.go file belongs to the main package. Following that we’ve imported the net/http package which provides HTTP client and server implementations for use in our app. This package is part of the standard library and is included with every installation of Go.

In the main function, http.NewServeMux() creates a new HTTP request multiplexer and assigns it to the mux variable. Essentially, a request multiplexer matches the URL of incoming requests against a list of registered paths and calls the associated handler for the path whenever a match is found.

Next, we register our first handler function for the root path /. This handler function is the second argument to HandleFunc and is always of the signature func(w http.ResponseWriter, r *http.Request).

If you look at the indexHandler function, you can see that it has this exact signature making it a valid second argument to HandleFunc. The w parameter is the structure we use to send responses to an HTTP request. It impements a Write() method which accepts a slice of bytes and writes the data to the connection as part of an HTTP response.

On the other hand, the r parameter represents the HTTP request receieved from the client. It’s how we access the data sent by a web browser on the server. We are not yet using it here, but we will definitely make use of it later on.

Finally, we have the http.ListenAndServe() method which starts the server on port 3000 if one is not set in the environment. Feel free to use another port if 3000 is in use on your machine.

Next, compile and execute the code you just wrote:

go run main.go

If you go to http://localhost:3000 in your browser, you should see the text “Hello World!” displayed on your screen.

Brave browser showing Hello World text

Templates in Go

Let’s go over the basics of templating in Go. If you are familiar with templates in other languages, it should be easy enough to understand.

Templates provide an easy way to customize the output of your web application depending on the route without having to write the same code in many places. For example, we can create a template for the navigation bar and use it across all pages of the site without duplicating the code. On top of that, we also get the ability to add some basic logic to our webpages.

Go provides two template libraries in its standard library: text/template and html/template. Both provide the same interface, however the html/template package is used to generate HTML output that is safe against code injection so we’ll be making use of it here.

Import this package in your main.go file, and use it as follows:

package main

import (
	"html/template"
	"net/http"
	"os"
)

var tpl = template.Must(template.ParseFiles("index.html"))

func indexHandler(w http.ResponseWriter, r *http.Request) {
	tpl.Execute(w, nil)
}

func main() {
	port := os.Getenv("PORT")
	if port == "" {
		port = "3000"
	}

	mux := http.NewServeMux()

	mux.HandleFunc("/", indexHandler)
	http.ListenAndServe(":"+port, mux)
}

tpl is a package level variable that points to a template definition from the provided files. The call to template.ParseFiles parses the index.html file in the root of our project directory and validates it.

We wrap the invocation of template.ParseFiles with template.Must so that the code panics if an error is obtained. The reason we panic here instead of trying to handle the error, is because there is no point in continuing code execution if we have an invalid template. It’s a problem that should be fixed before attempting to restart the server.

In the indexHandler function, we execute the template created earlier by providing two arguments: where we want to write the output to, and the data we want to pass to the template.

In the above case, we’re writing the output to the ResponseWriter interface and, since we don’t have any data to pass to our template at this time, nil is passed as the second argument.

Stop the running process in your terminal using Ctrl-C and start it again with go run main.go, then refresh your browser. You should see the text “News App Demo” on the page as shown below:

Brave browser showing News App Demo Text

Add a navigation bar to the page

Replace the contents of the <body> tag in your index.html file as shown below:

<main>
  <header>
    <a class="logo" href="/">News Demo</a>
    <form action="/search" method="GET">
      <input autofocus class="search-input" value=""
      placeholder="Enter a news topic" type="search" name="q">
    </form>
    <a href="https://github.com/freshman-tech/news" class="button
      github-button">View on Github</a>
  </header>
</main>

Then restart the server and refresh your browser. You should see something that looks like this:

Browser showing unstyled navigation bar

Serve static files

Notice that the navigation bar we added above is not styled despite the fact that we’ve already linked the stylesheet in the <head> of our document.

This is because the / path actually matches all paths that are not handled elsewhere. So if you go to http://localhost:3000/assets/style.css, you still get the News Demo homepage instead of the CSS file because the /assets/style.css route was not declared explcitly.

But having to declare explicit handlers for all our static files is not realistic, and cannot scale. Thankfully, we can create one handler to take care of serving all static assets.

The first thing to do is to instantiate a file server object by passing the directory where all our static files are placed:

fs := http.FileServer(http.Dir("assets"))

Next, we need to tell our router to use this file server object for all paths beginning with the /assets/ prefix:

mux.Handle("/assets/", http.StripPrefix("/assets/", fs))

All together now:

// main.go

// beginning of the file

func main() {
	port := os.Getenv("PORT")
	if port == "" {
		port = "3000"
	}

	mux := http.NewServeMux()

	// Add the following two lines
	fs := http.FileServer(http.Dir("assets"))
	mux.Handle("/assets/", http.StripPrefix("/assets/", fs))

	mux.HandleFunc("/", indexHandler)
	http.ListenAndServe(":"+port, mux)
}

Restart your server, and refresh your browser. The styles should kick in as shown below:

Brave browser showing styled navigation bar

Create the /search route

Let’s create the route that handles search requests for news articles. We’ll be making use of the News API for processing the queries, so you’ll need to sign up for a free API key here.

This route expects two query parameters: q represents the user’s query, and page is used to page through the results. This page parameter is optional. If it’s not included in the URL, we’ll just assume that the page is 1.

Add the following handler below indexHandler in your main.go file:

func searchHandler(w http.ResponseWriter, r *http.Request) {
	u, err := url.Parse(r.URL.String())
	if err != nil {
		w.WriteHeader(http.StatusInternalServerError)
		w.Write([]byte("Internal server error"))
		return
	}

	params := u.Query()
	searchKey := params.Get("q")
	page := params.Get("page")
	if page == "" {
		page = "1"
	}

	fmt.Println("Search Query is: ", searchKey)
	fmt.Println("Results page is: ", page)
}

The above code extracts the q and page parameters from the request URL, and prints them both to the terminal.

Next, register the searchHandler function as the handler for the /search path as shown below:

func main() {
	port := os.Getenv("PORT")
	if port == "" {
		port = "3000"
	}

	mux := http.NewServeMux()

	fs := http.FileServer(http.Dir("assets"))
	mux.Handle("/assets/", http.StripPrefix("/assets/", fs))

  // Add the next line
	mux.HandleFunc("/search", searchHandler)
	mux.HandleFunc("/", indexHandler)
	http.ListenAndServe(":"+port, mux)
}

Don’t forget to import the fmt and net/url packages at the top:

import (
	"fmt"
	"html/template"
	"net/http"
	"net/url"
	"os"
)

Now restart your server, type a query into the search input, and check the terminal. You should see your query printed in the terminal in the manner shown below:

Create the data model

When we make a request to News API’s /everything endpoint, we expect a json response in this format:

{
  "status": "ok",
  "totalResults": 4661,
  "articles": [
    {
      "source": {
        "id": null,
        "name": "Gizmodo.com"
      },
      "author": "Jennings Brown",
      "title": "World's Dumbest Bitcoin Scammer Tries to Scam Bitcoin Educator, Gets Scammed in The Process",
      "description": "Ben Perrin is a Canadian cryptocurrency enthusiast and educator who hosts a bitcoin show on YouTube. This is immediately apparent after a quick a look at all his social media. Ten seconds of viewing on of his videos will show that he is knowledgeable about di…",
      "url": "https://gizmodo.com/worlds-dumbest-bitcoin-scammer-tries-to-scam-bitcoin-ed-1837032058",
      "urlToImage": "https://i.kinja-img.com/gawker-media/image/upload/s--uLIW_Oxp--/c_fill,fl_progressive,g_center,h_900,q_80,w_1600/s4us4gembzxlsjrkmnbi.png",
      "publishedAt": "2019-08-07T16:30:00Z",
      "content": "Ben Perrin is a Canadian cryptocurrency enthusiast and educator who hosts a bitcoin show on YouTube. This is immediately apparent after a quick a look at all his social media. Ten seconds of viewing on of his videos will show that he is knowledgeable about..."
    }
  ]
}

To work with this data in Go, we need to generate a struct that mirrors the data when decoding the response body. You can certainly do this manually, but my preferred way is to use the JSON-to-Go website which makes the process really easy. It generates the Go struct (with tags) that will work for that JSON.

All you need to do is copy the JSON object and paste it into the field marked JSON, then copy the output and paste it in your code. Here’s what we get for the above JSON object:

type AutoGenerated struct {
	Status       string `json:"status"`
	TotalResults int    `json:"totalResults"`
	Articles     []struct {
		Source struct {
			ID   interface{} `json:"id"`
			Name string      `json:"name"`
		} `json:"source"`
		Author      string    `json:"author"`
		Title       string    `json:"title"`
		Description string    `json:"description"`
		URL         string    `json:"url"`
		URLToImage  string    `json:"urlToImage"`
		PublishedAt time.Time `json:"publishedAt"`
		Content     string    `json:"content"`
	} `json:"articles"`
}
Brave browser showing JSON to Go tool

I made a few changes to the AutoGenerated struct by separating the slice of Articles into its own struct, and updating the struct name. Paste the following below the tpl variable declaration in main.go and add the time package to your imports:

type Source struct {
	ID   interface{} `json:"id"`
	Name string      `json:"name"`
}

type Article struct {
	Source      Source    `json:"source"`
	Author      string    `json:"author"`
	Title       string    `json:"title"`
	Description string    `json:"description"`
	URL         string    `json:"url"`
	URLToImage  string    `json:"urlToImage"`
	PublishedAt time.Time `json:"publishedAt"`
	Content     string    `json:"content"`
}

type Results struct {
	Status       string    `json:"status"`
	TotalResults int       `json:"totalResults"`
	Articles     []Article `json:"articles"`
}

As you may know, Go requires all exported fields in a struct to start with a capital letter. However, it is convention to represent JSON fields with Camel case or Snake case names which do not start with a capital letter.

Therefore, we make use of struct field tags such as json:"id" to explicitly map a struct field to a JSON field as shown above. This also makes it possible to use competely different names for a struct field and the corresponding json field if necessary.

Finally, let’s create another struct type for each search query. Add this below the Results struct in main.go:

type Search struct {
	SearchKey  string
	NextPage   int
	TotalPages int
	Results    Results
}

This struct represents each search query made by the user. The SearchKey is the query itself, the NextPage field allows us to page through results, TotalPages is the total number of result pages for the query, and Results is the current page of results for the query.

Send request to News API and render results

Now that we have the data model for our application, let’s go ahead and make requests to the News API, then render the results on the page.

Since News API requires an API key, we need to figure out a way to pass this in our application without hardcoding it in the code. Enviromental variables are a common approach but I’ve opted to use command line flags instead. Go provides a flag package supporting basic command-line flag parsing and that’s what we’re going to use here.

First, declare a new apiKey variable under the tpl variable:

var apiKey *string

Then use it in the main function as follows:

func main() {
	apiKey = flag.String("apikey", "", "Newsapi.org access key")
	flag.Parse()

	if *apiKey == "" {
		log.Fatal("apiKey must be set")
	}

	// rest of the function
}

Here, we call the flag.String() method which allows us to define a string flag. The first argument to this method is the flag name, the second is the default value, while the third is the usage description.

After defining all the flags, you need to call flag.Parse() to actually parse them. Finally, since apikey is a required component for this application, we’re making sure the program crashes if this flag is not set when executing the program.

Make sure you’ve added the flag package to your imports, then restart your server and pass in the required apikey flag as shown below:

go run main.go -apikey=<your newsapi access key>

Next, let’s go ahead and update searchHandler so that the user’s search query is sent over to newsapi.org and the results are rendered in our template.

Replace the two fmt.Println() method calls at the end of the searchHandler function with the following code:

func searchHandler(w http.ResponseWriter, r *http.Request) {
	// beginning  of the function

	search := &Search{}
	search.SearchKey = searchKey

	next, err := strconv.Atoi(page)
	if err != nil {
		http.Error(w, "Unexpected server error", http.StatusInternalServerError)
		return
	}

	search.NextPage = next
	pageSize := 20

	endpoint := fmt.Sprintf("https://newsapi.org/v2/everything?q=%s&pageSize=%d&page=%d&apiKey=%s&sortBy=publishedAt&language=en", url.QueryEscape(search.SearchKey), pageSize, search.NextPage, *apiKey)
	resp, err := http.Get(endpoint)
	if err != nil {
		w.WriteHeader(http.StatusInternalServerError)
		return
	}

	defer resp.Body.Close()

	if resp.StatusCode != 200 {
		w.WriteHeader(http.StatusInternalServerError)
		return
	}

	err = json.NewDecoder(resp.Body).Decode(&search.Results)
	if err != nil {
		w.WriteHeader(http.StatusInternalServerError)
		return
	}

	search.TotalPages = int(math.Ceil(float64(search.Results.TotalResults / pageSize)))
	err = tpl.Execute(w, search)
	if err != nil {
		w.WriteHeader(http.StatusInternalServerError)
	}
}

First, we create a new instance of the Search struct and set the SearchKey field on the instance to the value of the q URL parameter in the HTTP request.

Following that, we convert the page variable into an integer, and assign the result to the NextPage field of the search variable. Then we create a pageSize variable and set its value to 20. This pageSize variable represents the number of results that the News API will return in its response. This value can range between 0 and 100.

Next, we construct the endpoint using fmt.Sprintf() and make a GET request to it. If the response from News API is not 200 OK, we’ll return a generic server error to the client. Otherwise, the response body is parsed into search.Results.

Then we compute the total number of pages by dividing the TotalResults field by the pageSize. For example, if a query returns 100 results and we’re only viewing 20 at a time, we will need to page through five pages to view all 100 results for that query.

After that, we execute our template and pass the search variable as the data interface. This allows us to access data from the JSON object in our template as you’ll see.

Before you switch to index.html, make sure to update your imports as shown below:

import (
	"encoding/json"
	"flag"
	"fmt"
	"html/template"
	"log"
	"math"
	"net/http"
	"net/url"
	"os"
	"strconv"
	"time"
)

Let’s go ahead and render the results to the page by modifying index.html as follows. Add this below the <header> tag:

<section class="container">
  <ul class="search-results">
    {{ range .Results.Articles }}
      <li class="news-article">
        <div>
          <a target="_blank" rel="noreferrer noopener" href="{{.URL}}">
            <h3 class="title">{{.Title }}</h3>
          </a>
          <p class="description">{{ .Description }}</p>
          <div class="metadata">
            <p class="source">{{ .Source.Name }}</p>
            <time class="published-date">{{ .PublishedAt }}</time>
          </div>
        </div>
        <img class="article-image" src="{{ .URLToImage }}">
      </li>
    {{ end }}
  </ul>
</section>

To access a struct field in a template, we use the dot operator. This operator refers to the struct object (search in this case) and then inside the template, we just specify the field name (like {{ .Results }}).

The range block allows us to iterate over a slice in Go and output some HTML for each item in the slice. Here, we’re iterating over the slice of Article structs contained in the Articles field and outputting some HTML on each iteration.

Restart your server, refresh your browser and search for news items on a popular topic. You should get a list of 20 results on the page as shown in the GIF below.

Browser showing news listings

Persist the search query in the input

Notice how the search query is gone from the input once the page refreshes with the results. Ideally, the query should be persisted until the user makes a new search. This is how Google Search works for example.

We can fix this easily by updating the value attribute of the input tag in our index.html file as follows:

<input autofocus class="search-input" value="{{ .SearchKey }}" placeholder="Enter a news topic" type="search" name="q">

Restart your browser, and make a new search. The search query will be persisted as shown below:

Format the published date

If you look at the date in each article, you can see that it’s not very readable. This current output is how News API returns the published date for article. But we can change this easily by adding a method to the Article struct and use that to format the date instead of using the default value.

Go ahead and add the following code just below the Article struct in main.go:

func (a *Article) FormatPublishedDate() string {
	year, month, day := a.PublishedAt.Date()
	return fmt.Sprintf("%v %d, %d", month, day, year)
}

Here, a new FormatPublishedDate method is created on the Article struct and this method formats the PublishedAt field on the Article and returns a string in the following format: January 10, 2009.

To use this new method in your template, replace .PublishedAt with .FormatPublishedDate in your index.html file. Then restart your server and repeat the previous search query. This will output the results with the time correctly formatted as shown below:

Brave browser showing correctly formatted date

Show the total number of results

Let’s improve the UI of our news app by showing the total number of results at the top of the page, then also display a message in case no results were found for a particular query.

All you need to do is add the following code as a child of .container, just above the .search-results element in your index.html file:

<div class="result-count">
  {{ if (gt .Results.TotalResults 0)}}
  <p>About <strong>{{ .Results.TotalResults }}</strong> results were found.</p>
  {{ else if (ne .SearchKey "") and (eq .Results.TotalResults 0) }}
  <p>No results found for your query: <strong>{{ .SearchKey }}</strong>.</p>
  {{ end }}
</div>

Go templates support several comparison functions, some of which are used above. We use the gt function to check if the TotalResults field of the Results struct is greater than zero. If it is, the total number of results will be printed at the top of the page.

Otherwise, if SearchKey is not equal to an empty string ((ne .SearchKey "")) and TotalResults is equal to zero ((eq .Results.TotalResults 0)), a “No results found” message is outputted.

Restart your server, and type some gibberish into the search input so that no news items for your query are found. You should see the No results found message on the screen.

Browser showing no results found message

After that, make another search query this time with on a popular topic. The number of results will be outputted at the top of the page as shown below:

Browser showing results count at the top of the page

Pagination

Since we’re only displaying 20 resuts at a time, we need a way for the user to go the the next or previous page of results anytime.

First, we’ll add a Next button at the bottom of the results if the last page of results has not yet been reached. To determine if the last page of results have been reached, create this new method below the Search struct definition in main.go:

func (s *Search) IsLastPage() bool {
	return s.NextPage >= s.TotalPages
}

This method checks if the NextPage field is greater than the TotalPages field on a Search instance. For this to work though, we need to increment NextPage every time a new page of results is rendered. Here’s how to do just that:

func searchHandler(w http.ResponseWriter, r *http.Request) {
	// start of the function
	search.TotalPages = int(math.Ceil(float64(search.Results.TotalResults / pageSize)))
	// Add this if block
	if ok := !search.IsLastPage(); ok {
		search.NextPage++
	}

	// rest of the function
}

Finally, let’s add a button that will allow the user go to the next page of results. This following should be placed below .search-results in your index.html file.

<div class="pagination">
  {{ if (ne .IsLastPage true) }}
    <a href="/search?q={{ .SearchKey }}&page={{ .NextPage }}" class="button next-page">Next</a>
  {{ end }}
</div>

As long as the last page for that query has not been reached, the Next button will be rendered at the bottom of the results list.

As you can see, the href of the anchor tag above points to the /search route and retains the current search query in the q parameter while using the value of NextPage in the page parameter.

Let’s throw in a Previous button as well. This button should only be rendered if the current page is greater than 1. To do this, we’ll need to create a new CurrentPage() method on Search to help us do just that. Add this below the IsLastPage method:

func (s *Search) CurrentPage() int {
	if s.NextPage == 1 {
		return s.NextPage
	}

	return s.NextPage - 1
}

The current page is simply NextPage - 1 except if NextPage is 1. To get the previous page, just subtract 1 from the current page. The following method does just that:

func (s *Search) PreviousPage() int {
	return s.CurrentPage() - 1
}

So we can add the following code to render the Previous button only if the current page is greater than 1. Modify the .pagination element in your index.html as follows:

<div class="pagination">
  {{ if (gt .NextPage 2) }}
    <a href="/search?q={{ .SearchKey }}&page={{ .PreviousPage }}" class="button previous-page">Previous</a>
  {{ end }}
  {{ if (ne .IsLastPage true) }}
    <a href="/search?q={{ .SearchKey }}&page={{ .NextPage }}" class="button next-page">Next</a>
  {{ end }}
</div>

Now restart the server, and make a new search query. You should be able to page through the results as shown below:

Show the current page

Instead of displaying only the total number of results that were found for a query, it is also helpful for the user to see the total number of pages for that query, and what page he is currently on.

To do this, we only need to modify our index.html file as follows:

<div class="result-count">
  {{ if (gt .Results.TotalResults 0)}}
    <p>About <strong>{{ .Results.TotalResults }}</strong> results were found. You are on page <strong>{{ .CurrentPage }}</strong> of <strong> {{ .TotalPages }}</strong>.</p>
  {{ else if (ne .SearchKey "") and (eq .Results.TotalResults 0) }}
    <p>No results found for your query: <strong>{{ .SearchKey }}</strong>.</p>
  {{ end }}
</div>

Once you restart the server and make a new search, the current page and total number of pages will be indicated on top of the page along with the total result count.

Browser showing current page

Deploy to Heroku

Now that our app is feature complete, let’s go ahead and deploy it to Heroku. Sign up for a free account, then follow this link to create a new app. Give it a unique name. I called mine freshman-news.

Next, follow the instructions here to install the Heroku CLI on your machine. Then run the heroku login command in the terminal to login to your Heroku account.

Make sure you’ve initialised a git repository for your project. If not, run the git init command at the root of your project directory, then run the command below to set heroku as a remote for your git repo. Replace freshman-news with the name of your application.

heroku git:remote -a freshman-news

Next, create a Procfile in the root of your project directory (touch Procfile) and paste in the following contents:

web: bin/news-demo -apikey $NEWS_API_KEY

Following that, specify the GitHub repo for your project and the version of Go you are using in your go.mod file as shown below. Create this file if it doesn’t exist in the project root already.

module github.com/freshman-tech/news-demo

go 1.12.9

Before you deploy the app, head over to the Settings tab in the Heroku dashboard and hit Reveal Config Vars. We need to set the NEWS_API_KEY environmental variable so that it can be passed to the binary when starting the server.

Heroku config variables

Finally, commit your code and push it to the Heroku remote using the following commands:

git add .
git commit -m "Initial commit"
git push heroku master

Once the deployment process is complete, you can open https://.herokuapp.com to view and test your project.

Wrap up

In this article, we successfully created a News app and learnt the basics of using Go for web development along the way. We also covered how to deploy the finished application to Heroku.

I hope this exercise was helpful to you. If you have any questions regarding this tutorial, please leave a comment below and I’ll get back to you.

Thanks for reading!