Learning Go – Day 12

by Posted byAndré Jacobs

Featured image by Egon Elbre

Package dependencies

One of the topics that you will encounter and even try to understand in your own projects, is just how exactly to structure Go projects.

I will be creating two new GitHub repositories in order to get a better understanding of how package imports work and just how I might open source some code later that can be used by others.

You can find the two repositories here: go-test1 and go-test2

  • Created a new repository called go-test1 and cloned it locally. This will provide some code to be consumed by another repository.
  • Initialized the go module.
$ cd go-test1
$ go mod init github.com/andrejacobs/go-test1
  • Created another new repository called go-test2 and clone and set it up as well.

Package 1 – Simple quote generator

  • go-test1 is going to be a simple random quote generator.
    • Package quote provides a type to represent a list of quotes along with a method to generate some random quotes.
    • Package internal is used by the generate.go file to generate the random people and quote lines. Also this was a great test to see how Go handles the special internal package importing.
    • The repository also supplies a CLI to generate random quotes.
$ tree
.
├── LICENSE
├── README.md
├── cmd
│   └── quote-cli
│       └── main.go
├── go.mod
├── internal
│   └── random.go
└── quote
    ├── generate.go
    ├── generate_test.go
    ├── list.go
    └── list_test.go

4 directories, 9 files
  • quote/list.go
package quote

type Quote struct {
	Line string
	Who  string
}

type QuoteList struct {
	Quotes []Quote
}

func (ql *QuoteList) Add(who string, line string) {
	q := Quote{
		Who:  who,
		Line: line,
	}

	ql.Quotes = append(ql.Quotes, q)
}
  • quote/list_test.go
package quote_test

import (
	"testing"

	"github.com/andrejacobs/go-test1/quote"
)

func TestListAdd(t *testing.T) {
	ql := &quote.QuoteList{}

	if len(ql.Quotes) > 0 {
		t.Fatal("QuoteList not initialized correctly")
	}

	ql.Add("Person 1", "The quick brown fox")
	ql.Add("Person 2", "Jumped over the lazy dog")

	if len(ql.Quotes) != 2 {
		t.Fatalf("Expected 2 quotes in the list, instead there were: %d", len(ql.Quotes))
	}
}
  • quote/generate.go
package quote

import (
	"github.com/andrejacobs/go-test1/internal"
)

func (ql *QuoteList) Generate(count int) {

	for i := 0; i < count; i++ {
		person := internal.RandomPerson()
		line := internal.RandomLine()
		ql.Add(person, line)
	}
}
  • quote/generate_test.go
package quote_test

import (
	"testing"

	"github.com/andrejacobs/go-test1/quote"
)

func TestGenerate(t *testing.T) {
	ql := &quote.QuoteList{}

	expectedCount := 5
	ql.Generate(expectedCount)
	if len(ql.Quotes) != expectedCount {
		t.Fatalf("Expected %d quotes to be generated, instead we got %d", expectedCount, len(ql.Quotes))
	}
}
  • internal/random.go
package internal

import "math/rand"

func RandomPerson() string {
	var people = []string{
		"Jannie",
		"Sannie",
		"Pieter",
		"Stephan",
		"Kobus",
	}

	person := people[rand.Intn(len(people))]
	return person
}

func RandomLine() string {
	var lines = []string{
		"Lorem ipsum dolor sit amet, consectetur adipiscing elit.",
		"Mauris at metus in lectus cursus placerat.",
		"Quisque gravida mauris vel ante luctus, eget congue est ornare.",
		"Sed semper mauris nec commodo pharetra.",
		"Fusce nec elit sagittis, sodales nulla a, vehicula libero.",
		"Mauris dignissim libero nec neque tristique, nec fringilla nunc pretium.",
		"Maecenas fermentum diam eget justo euismod bibendum.",
		"Nam euismod sapien vitae quam dapibus molestie.",
		"Sed ac ligula eget ipsum semper pharetra.",
		"Morbi vestibulum augue quis nulla pharetra, at finibus leo consectetur.",
	}

	line := lines[rand.Intn(len(lines))]
	return line
}
  • cmd/quote-cli/main.go
package main

import (
	"flag"
	"fmt"
	"io"
	"os"

	"github.com/andrejacobs/go-test1/quote"
)

func main() {
	countFlag := flag.Int("count", 1, "The number of quotes to print on STDOUT")
	flag.Parse()

	if err := run(os.Stdout, *countFlag); err != nil {
		fmt.Fprintln(os.Stderr, err)
		os.Exit(1)
	}
}

func run(out io.Writer, count int) error {
	ql := &quote.QuoteList{}
	ql.Generate(count)

	for _, quote := range ql.Quotes {
		_, err := fmt.Fprintf(out, "%q - %s\n", quote.Line, quote.Who)
		if err != nil {
			return err
		}
	}

	return nil
}
  • Run unit-test
$ cd quote
$ go test -v
=== RUN   TestGenerate
--- PASS: TestGenerate (0.00s)
=== RUN   TestListAdd
--- PASS: TestListAdd (0.00s)
PASS
ok      github.com/andrejacobs/go-test1/quote   0.225s
  • Run the CLI.
$ cd cmd/quote-cli
$ go run .
"Nam euismod sapien vitae quam dapibus molestie." - Sannie

$ go run . -count 5
"Nam euismod sapien vitae quam dapibus molestie." - Sannie
"Morbi vestibulum augue quis nulla pharetra, at finibus leo consectetur." - Pieter
"Sed ac ligula eget ipsum semper pharetra." - Sannie
"Lorem ipsum dolor sit amet, consectetur adipiscing elit." - Jannie
"Lorem ipsum dolor sit amet, consectetur adipiscing elit." - Sannie

Package 2 – Sorted quote generator

  • go-test2 is going to use go-test1 to generate random quotes and extend it by adding the ability to have the quotes sorted alphabetically.
  • Add dependency on go-test1 by using go get 
$ go get -u -x github.com/andrejacobs/go-test1@latest
...
go: downloading github.com/andrejacobs/go-test1 v0.0.0-20220509114742-d186c44a2d88

HANDY TIP: You can pass -x to go get so that it prints out all the commands being run. This is handy for debugging why git or ssh is failing.

  • The package have been made available at $(go env GOPATH)/pkg/mod/github.com/andrejacobs/go-test1@v0.0.0-20220509114742-d186c44a2d88
  • The dependency has been added to go.mod for go-test2.
$ cat go.mod
module github.com/andrejacobs/go-test2

go 1.18

require github.com/andrejacobs/go-test1 v0.0.0-20220509114742-d186c44a2d88 // indirect
  • NOTE: That the entire git repository for go-test1 was cloned. This is still the case even if you tried to just fetch the specific package. For example if you “go get … go-test1/quote@latest”.
  • This is the structure I will be using.
    • Package sorted will provide a new sorted QuoteList that wraps the quote.QuoteList type from go-test1 module.
    • There is also a CLI under cmd/sorted-quote-cli.
$ tree
.
├── LICENSE
├── README.md
├── cmd
│   └── sorted-quote-cli
│       └── main.go
├── go.mod
├── go.sum
└── sorted
    ├── quoteList.go
    └── sort_test.go

3 directories, 7 files
  • sorted/quoteList.go
    • NOTE: One big thing I learned through this exercise is that you can’t add methods to a type that is outside of the package. I.e. you can’t just extend like you can in Swift. However what you can do is, to wrap the existing type inside your own package and then add methods to that type. Like what I have done with QuoteList below.
package sorted

import (
	"sort"

	"github.com/andrejacobs/go-test1/quote"
)

type QuoteList struct {
	quote.QuoteList
}

func (ql *QuoteList) Sort() {
	sort.SliceStable(ql.Quotes, func(i, j int) bool {
		return ql.Quotes[i].Line < ql.Quotes[j].Line
	})
}

func (ql *QuoteList) Add(who string, line string) {
	ql.QuoteList.Add(who, line)
	ql.Sort()
}
  • sorted/sort_test.go
package sorted_test

import (
	"testing"

	"github.com/andrejacobs/go-test2/sorted"
)

func TestSort(t *testing.T) {
	ql := &sorted.QuoteList{}

	ql.Add("p1", "Bravo")
	ql.Add("p2", "Zebra")
	ql.Add("p3", "Alpha")

	if ql.Quotes[0].Line != "Alpha" &&
		ql.Quotes[1].Line != "Bravo" &&
		ql.Quotes[2].Line != "Zebra" {
		t.Fatal("Expected the quotes to be sorted")
	}
}
  • cmd/sorted-quote-cli/main.go
package main

import (
	"flag"
	"fmt"
	"io"
	"os"

	"github.com/andrejacobs/go-test2/sorted"
)

func main() {
	countFlag := flag.Int("count", 1, "The number of quotes to print on STDOUT")
	flag.Parse()

	if err := run(os.Stdout, *countFlag); err != nil {
		fmt.Fprintln(os.Stderr, err)
		os.Exit(1)
	}
}

func run(out io.Writer, count int) error {
	ql := &sorted.QuoteList{}
	ql.Generate(count)
	ql.Sort()

	for _, quote := range ql.Quotes {
		_, err := fmt.Fprintf(out, "%q - %s\n", quote.Line, quote.Who)
		if err != nil {
			return err
		}
	}

	return nil
}
  • Run unit-tests.
$ cd sorted
$ go test -v
=== RUN   TestSort
--- PASS: TestSort (0.00s)
PASS
ok      github.com/andrejacobs/go-test2/sorted  0.185s
  • Run the CLI.
$ cd cmd/sorted-quote-cli
$ go run . -count 5
"Lorem ipsum dolor sit amet, consectetur adipiscing elit." - Jannie
"Lorem ipsum dolor sit amet, consectetur adipiscing elit." - Sannie
"Morbi vestibulum augue quis nulla pharetra, at finibus leo consectetur." - Pieter
"Nam euismod sapien vitae quam dapibus molestie." - Sannie
"Sed ac ligula eget ipsum semper pharetra." - Sannie

Versioning

Now that I have two working packages in two different git repositories it is time to add functionality to go-test1 and see how versioning will work. From the previous “go get” we can see that version v0.0.0 was used.

  • I will be adding a new package jsonquote that will simply be used to encode the QuoteList into JSON.
jsonquote
├── json.go
└── json_test.go
  • jsonquote/json.go
package jsonquote

import (
	"encoding/json"
	"io"

	"github.com/andrejacobs/go-test1/quote"
)

func Encode(out io.Writer, ql *quote.QuoteList) {
	encoder := json.NewEncoder(out)
	encoder.Encode(ql)
}
  • jsonquote/json_test.go
package jsonquote_test

import (
	"bytes"
	"testing"

	"github.com/andrejacobs/go-test1/jsonquote"
	"github.com/andrejacobs/go-test1/quote"
)

func TestJsonEncoding(t *testing.T) {
	ql := &quote.QuoteList{}
	ql.Add("p1", "line1")
	ql.Add("p2", "line2")
	ql.Add("p3", "line3")

	var result bytes.Buffer
	jsonquote.Encode(&result, ql)

	expected := "{\"Quotes\":[{\"Line\":\"line1\",\"Who\":\"p1\"},{\"Line\":\"line2\",\"Who\":\"p2\"},{\"Line\":\"line3\",\"Who\":\"p3\"}]}\n"
	if result.String() != expected {
		t.Fatalf("Expected json:\n%q\n\nInstead we received:\n%q", expected, result.String())
	}
}
  • Modified the CLI to include the option of encoding into JSON.
package main

import (
	"flag"
	"fmt"
	"io"
	"os"

	"github.com/andrejacobs/go-test1/jsonquote"
	"github.com/andrejacobs/go-test1/quote"
)

func main() {
	countFlag := flag.Int("count", 1, "The number of quotes to print")
	jsonFlag := flag.Bool("json", false, "Encode the quotes in JSON")
	flag.Parse()

	if err := run(os.Stdout, *countFlag, *jsonFlag); err != nil {
		fmt.Fprintln(os.Stderr, err)
		os.Exit(1)
	}
}

func run(out io.Writer, count int, outputJson bool) error {
	ql := &quote.QuoteList{}
	ql.Generate(count)

	if outputJson {
		jsonquote.Encode(out, ql)
	} else {
		for _, quote := range ql.Quotes {
			_, err := fmt.Fprintf(out, "%q - %s\n", quote.Line, quote.Who)
			if err != nil {
				return err
			}
		}
	}
	return nil
}
  • Run unit-test and the CLI.
$ cd jsonquote
$ go test -v
=== RUN   TestJsonEncoding
--- PASS: TestJsonEncoding (0.00s)
PASS
ok  	github.com/andrejacobs/go-test1/jsonquote	0.122s

$ cd ../cmd/quote-cli
$ go run . -count 3 -json
{"Quotes":[{"Line":"Nam euismod sapien vitae quam dapibus molestie.","Who":"Sannie"},{"Line":"Morbi vestibulum augue quis nulla pharetra, at finibus leo consectetur.","Who":"Pieter"},{"Line":"Sed ac ligula eget ipsum semper pharetra.","Who":"Sannie"}]}
  • Commit the changes in git and push to origin.
  • Created a v0.1.0 release in GitHub.

NOTE: And now the penny drops! Go doesn’t seem to have consensus yet on how to do versioning and dependency management. There are a number of tools to manage versions for you and also the concept of vendoring. I will have to dig deeper into this later, for now it is safe to say you really have to consider how you structure your API.

  • Getting latest go-test1 into go-test2 to use.
$ go get -u -x github.com/andrejacobs/go-test1@latest

Notice how it still states v0.0.0, however inspecting the files in my GOPATH I can see it grabbed the latest copy from the git repo. My guess is it always grabs latest master and not actually based on the github release / tag.

  • I have added the same -json argument support to go-test2’s CLI and tested it.
$ go run . -count 3 -json
{"Quotes":[{"Line":"Morbi vestibulum augue quis nulla pharetra, at finibus leo consectetur.","Who":"Pieter"},{"Line":"Nam euismod sapien vitae quam dapibus molestie.","Who":"Sannie"},{"Line":"Sed ac ligula eget ipsum semper pharetra.","Who":"Sannie"}]}

Finishing up

All in all this was a very worthwhile experiment and I have learned a lot upfront about how I will be tackling my own initial applications, modules and packages.

I certainly need to brush up a bit on what the best versioning and dependency management is for golang.