A simple Go implementation for graceful shutdown of HTTP servers with Go 1.8+ wrapped in ~20 lines of code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

func main() {
	srv := &http.Server{
		Addr:    ":8080",
		Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			time.Sleep(5 * time.Second)
			_, _ = w.Write([]byte("hello"))
		}),
	}

	if err := gracefulShutdown(srv, 10*time.Second); err != nil {
		log.Println(err)
	}
}

// gracefulShutdown stops the given HTTP server on
// receiving a stop signal and waits for the active connections
// to be closed for {timeout} period of time.
func gracefulShutdown(srv *http.Server, timeout time.Duration) error {
	done := make(chan error, 1)
	go func() {
		c := make(chan os.Signal, 1)
		signal.Notify(c, os.Interrupt, syscall.SIGTERM)
		<-c

		ctx := context.Background()
		var cancel context.CancelFunc
		if timeout > 0 {
			ctx, cancel = context.WithTimeout(ctx, timeout)
			defer cancel()
		}

		done <- srv.Shutdown(ctx)
	}()

	if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
		return err
	}

	return <-done
}

The graceful shutdown function can also be used directly as a package from github.com: Graceful Shutdown

Example for running unit tests with coverage output

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

image: golang:1.16

stages:
  - test

before_script:
  - ln -s /builds /go/src/gitlab.com
  - cd /go/src/gitlab.com/${CI_PROJECT_PATH}

unit tests:
  stage: test
  script:
    - go version
    - go test -race $(go list ./... | grep -v /vendor/) -coverprofile=coverage.out
    - go tool cover -func=coverage.out

Should we commit vendor in our Go projects?

Since the introduction of Go modules, the prevalent opinion on the internet is that we should NOT commit the vendor directory of our projects.

The argument against committing vendor is that it’s unnecessary, because the Go tools can resolve and find dependencies when needed on-the-fly. All public Go modules are also hosted by various Go Proxy servers for redundancy.

While I agree that it’s technically unnecessary and bloats the repo, I’d like to give some arguments in favour of committing vendor, go.mod and go.sum

Here is a list of arguments for committing vendor:

• Building the project doesn’t depend on some code being available on Github/Gitlab/… or the Go Proxy servers. Open source projects may disappear because of censorship, authors incentives, licensing changes or some other reasons I can’t currently think of. Not in your repo, not your code.

• We may use internal or 3rd party Go modules (private) which may also disappear or become inaccessible, but if they are committed in vendor, they are part of our project and nothing breaks unexpectedly.

• Go modules may not follow semantic versioning, which means the Go tools will rely on the latest commit hash when fetching them on-the-fly. Repo history may be rewritten (e.g. rebase) and you, a colleague or your CI job may end up with different code for the dependencies they use.

• Committing vendor can improve your code review process. Typically, I’m used to committing dependency changes in a separate commit, so they can be easily viewed if the reviewer is curious.

Here’s an interesting observation related to bloating the repo. If I make code review, and a team member has included a new dependency with 300 files (or updated one with 300 files changed), I would be pretty curious to deep dive into that and start a discussion about the code quality, the need for this change or alternative Go modules. This may lead us to actually decrease our binary size and overall complexity.

If I just see a single new line in go.mod in new MR, chances are I won’t even think about it.

• CI/CD jobs which perform compilation and build steps need not waste time and network to download the dependencies every time the CI job is executed. All needed dependencies are local and present (go build -mod vendor)

• As a pseudo argument I can ask why important open source Go projects like Kubernetes are committing vendor, if it’s not necessary?

Conclusion

For my own peace of mind I would always prefer to commit the vendor directory alongside go.mod and go.sum while at the same time try to keep the project dependencies at minimum.

If we want to preserve (pass) the command line arguments when running a bash script and pass them to a program defined in the script, we can use the $* pattern after the call. An example bash script called myscript.sh is shown below. It sets some environment variables (optional) and starts a program called myserver. The program will accept all command line args passed to the bash script as if they were passed to the myserver program directly.

1
2
3
4
5

#!/bin/bash

SOME_ENV_VARIABLE=var1 \
ANOTHER_ENV_VARIABLE=var2 \
myserver $*

Calling the script like:

1

./myscript.sh -addr http://example.com -port 8080

will be like calling the myserver program like:

1

./myserver -addr http://example.com -port 8080

The following syntax highlighting rules are placed inside the ~/.nanorc file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

## Go Syntax Highlighting
##
syntax "go" "\.go$"

## Types n stuff
color green "\<((u)?int(8|16|32|64)?|float(32|64)|byte|string|interface|bool)\>"
color yellow "\<(package|import|const|var|type|struct|func|chan|defer)\>"
color yellow "\<(for|range|if|else|case|default|switch)\>"
color magenta "\<(continue|break|return)\>"

## Strings
color cyan "".*""
color cyan "'.'"

## Comment highlighting
color brightblue "//.*"
color brightblue start="/\*" end="\*/"

## Trailing whitespace
color ,green "[[:space:]]+$"

Place the stack traces in a file and run the following command:

1

./retrace.sh [-verbose] /path/to/mapping.txt /path/to/stacktrace.txt

The following HTTP error may happen when adding a private module (repo) to the project even if your SSH keys are properly configured and you have access to the private repo. Private means it’s not a publicly open project on Github/Gitlab/etc.

The reason for that is that the go mod tool tries to verify the module with its public checksum database, but it cannot find the module listed there (because it’s private).

To skip the check we can declare the repo as private:

1
2

export GO111MODULE=on
export GOPRIVATE=github.com/myusername/myproject

Another option is to disable the checksum database verification:

1
2
3

export GO111MODULE=on
export GOPROXY=direct
export GOSUMDB=off

Recently I noticed the adaptive brightness of my phone doesn’t work. Here is the fix:

1. Open Settings and click Apps and notifications

2. Click on the link See all 123 apps

3. Find an app called Device Health Services and click it

4. Open Storage & cache of the app and click Clear storage

5. Click on the button Reset adaptive brightness

6. It seems to work for the moment

To forward requests coming to a local TCP port to another server [host:port] connected via SSH we can use a config file or direct command line arguments. To use a config file, it should be named config and placed inside the ~/.ssh folder. Here is a sample ~/.ssh/config file with some options:

1
2
3
4
5
6

Host my-ssh-connection
  Hostname 10.20.30.40
  User penkovski
  LocalForward 7070 localhost:8080
  LocalForward 7171 localhost:8181
  ControlMaster auto

This will forward all requests hitting the local (client) TCP port 7070 to the localhost:8080 on the server and all requests for local TCP port 7171 to the localhost:8181 on the server. The above configuration will be used automatically when initiating an SSH connection by the alias:

1

ssh my-ssh-connection

In my humble opinion, modifying an XML node value with the Go standard xml package without describing all tags in data structures is not very intuitive. Hence this post is born. There are many ways to do it with their pros and cons. I’ll describe 2 ways. The first is simpler, but loses some XML elements like comments and possibly formatting. The other is more involved but preserves everything from the initial document, including comments and formatting.

Below is the XML document we’ll use as an example and the node we’ll modify is app>description>version. We’d like to set the version value from 1.0 to 2.0. Assume the XML file is called application.xml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

<app>
	<!-- application description -->
	<description>
		<name>My App</name>
		<version>1.0</version>
	</description>
	<!-- developer maintaining the application -->
	<developer>
		<name>Developer Name</name>
		<contacts>
			<email>developer@example.com</email>
			<mobile>+359888888888</mobile>
		</contacts>
	</developer>
	<!-- markets where the app is distributed -->
	<market>
		<googleplay>https://googleplay.com/url</googleplay>
		<appstore>https://appstore.com/url</appstore>
	</market>
</app>