Category Archives: golang

Go memory hints

Below hints are related to very hot places of CPU/memory bound programs. And in regular (i/o bound) programs do not make much sense. Stack vs heap Go’s compiler goal is to allocate on the stack as much as possible. As it will reduce the amount of data that needs to be cleaned by the Garbage …

Useful links for proto generator development

https://rotemtam.com/2021/03/22/creating-a-protoc-plugin-to-gen-go-code/ https://medium.com/@tim.r.coulson/writing-a-protoc-plugin-with-google-golang-org-protobuf-cd5aa75f5777 https://github.com/nametake/protoc-gen-gohttp https://github.com/drekle/protoc-gen-goexample/blob/c60883c9711f3cfbd5191fd697651dbd067e6fdd/main.go#L135 https://github.com/grpc-ecosystem/grpc-gateway/blob/24434e22fb9734f1a62c81c4ea246125d7844645/examples/internal/proto/examplepb/echo_service_grpc.pb.go https://pkg.go.dev/google.golang.org/protobuf/compiler/protogen#Options.Run https://github.com/lyft/protoc-gen-star https://go.dev/blog/protobuf-apiv2 https://docs.buf.build/build/usage/#configuration https://github.com/grpc-ecosystem/grpc-gateway https://github.com/googleapis/googleapis/tree/master/google/api https://github.com/bufbuild/buf https://stackoverflow.com/questions/36540427/cannot-find-package-google-protobuf https://pkg.go.dev/google.golang.org/protobuf https://developers.google.com/protocol-buffers/docs/reference/go-generated#package https://stackoverflow.com/questions/61666805/correct-format-of-protoc-go-package https://grpc.io/docs/languages/go/quickstart/ https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf https://github.com/gogo/protobuf/blob/b03c65ea87cdc3521ede29f62fe3ce239267c1bc/protobuf/google/protobuf/descriptor.proto

Go study

A list of resources for learning golang https://github.com/ardanlabs/gotraining/tree/master/reading

Nullable fields in json generated from protobuf file

Native support With grpc v2 for go, protofile’s optional option is supported, so you don’t need to import proto extensions anymore and use things like google.protobuf.StringValue. my.proto

message FeeView {
    optional string name = 1 [json_name = "name"];
}

message FeeView {

optional string name = 1 [json_name = "name"];

}

go mapper

var name *string
if fee.Name != nil {
	name = &fee.Name
}

fee := &desc.FeeView{
	Name: name,
}

var name *string

if fee.Name != nil {

name = &fee.Name

}

fee := &desc.FeeView{

Name: name,

}

Legacy way with extensions Null for nil value my.proto

import "google/protobuf/wrappers.proto";

message FeeView {
    google.protobuf.StringValue name = 1 [json_name = "name"];
}

import "google/protobuf/wrappers.proto";

message FeeView {

google.protobuf.StringValue name = 1 [json_name = "name"];

}

go mapper

import "github.com/gogo/protobuf/types"

var name *types.StringValue
if fee.Name != nil {
	name = &types.StringValue{
		Value: *fee.Name,
	}
}

fee := &desc.FeeView{
	Name: name,
}

import "github.com/gogo/protobuf/types"

var name *types.StringValue

if fee.Name != nil {

name = &types.StringValue{

Value: *fee.Name,

}

fee := &desc.FeeView{

Name: name,

}

Which will lead to this json response

{"translation_key":"a fee","name":null,"value":5,"style":"body","text_color":"grey","tag":null}

1	{"translation_key":"a fee","name":null,"value":5,"style":"body","text_color":"grey","tag":null}

Omit a field …

Talk about pros and cons of GraphQL

Pros: Very flexible — clients decide which fields of which entities they need. Documentation and test UI are available by default. Cons: Code looks ugly. N+1 selects problem is hard to solve (most probably — with some ad-hoc). It’s harder to test as you don’t know all the use cases. Not a binary protocol — …

Debugging go app in Jetbrains Goland on m1

First download a proper version of GoLand here — select .dmg (MacOS Apple Silicon) here (direct link — https://www.jetbrains.com/go/download/download-thanks.html?type=eap&platform=macM1&build=211.6556.11&code=GO). Then make sure you are using arm version of go (go version go1.16.2 darwin/arm64). If not, download and install arm version of golang (like «go1.16.2.darwin-arm64.pkg») from the list here — https://golang.org/dl/. Now it should work. With …

Example of using embed in go

Nice article on using go templates with embed introduced in go1.16: https://philipptanlak.com/web-frontends-in-go/. That’s the code:

Embedding assets to go binary in go 1.16

That’s how you can use it:

package server

import "embed"

// content holds our static web server content.
//go:embed image/* template/*
//go:embed html/index.html
var content embed.FS
data, _ := f.ReadFile("html/index.html")
print(string(data))

package server

import "embed"

// content holds our static web server content.

//go:embed image/* template/*

//go:embed html/index.html

var content embed.FS

data, _ := f.ReadFile("html/index.html")

print(string(data))

Or embedding one file directly into a variable:

import _ "embed"

//go:embed hello.txt
var s string
print(s)

import _ "embed"

//go:embed hello.txt

var s string

print(s)

More details — https://golang.org/pkg/embed/

Forward multiple kubernetes pods for local development

kubefwd is a great tool that can forward a lot of pods at once to your local machine. The only downside for me is that it touches your local /etc/hosts. That’s how I use it:

sudo KUBECONFIG=<path to your k8s config> kubefwd svc -l "app in (first-api, second-api, third-service)"

1	sudo KUBECONFIG=<path to your k8s config> kubefwd svc -l "app in (first-api, second-api, third-service)"

It does not have huge documentation, but it is written in golang, and you can check how the source …

Force ipv4 for golang http client

Sometimes you want to prevent your http client from using ipv6/tcp6. That’s how you do it.

dialer := &net.Dialer{
	Timeout:   dialTimeout,
	KeepAlive: keepAliveTimeout,
}

transport := &http.Transport{
	Proxy:             http.ProxyFromEnvironment,
	ForceAttemptHTTP2: false,
	DialContext: func(ctx context.Context, network string, addr string) (net.Conn, error) {
		ipv4, err := resolveIPv4(addr)
		if err != nil {
			return nil, err
		}

		return dialer.DialContext(ctx, network, ipv4)
	},
}

httpClient := &http.Client{
	Transport: transport,
	Timeout:   args.HTTPTimeout,
}

// resolveIPv4 resolves an address to IPv4 address.
func resolveIPv4(addr string) (string, error) {
	url := strings.Split(addr, ":")

	m := new(dns.Msg)
	m.SetQuestion(dns.Fqdn(url[0]), dns.TypeA)
	m.RecursionDesired = true

	config, _ := dns.ClientConfigFromFile("/etc/resolv.conf")
	c := new(dns.Client)
	r, _, err := c.Exchange(m, net.JoinHostPort(config.Servers[0], config.Port))
	if err != nil {
		return "", err
	}
	for _, ans := range r.Answer {
		if a, ok := ans.(*dns.A); ok {
			url[0] = a.A.String()
		}
	}

	return strings.Join(url, ":"), nil
}

dialer := &net.Dialer{

Timeout: dialTimeout,

KeepAlive: keepAliveTimeout,

}

transport := &http.Transport{

Proxy: http.ProxyFromEnvironment,

ForceAttemptHTTP2: false,

DialContext: func(ctx context.Context, network string, addr string) (net.Conn, error) {

ipv4, err := resolveIPv4(addr)

if err != nil {

return nil, err

}

return dialer.DialContext(ctx, network, ipv4)

}

httpClient := &http.Client{

Transport: transport,

Timeout: args.HTTPTimeout,

}

// resolveIPv4 resolves an address to IPv4 address.

func resolveIPv4(addr string) (string, error) {

url := strings.Split(addr, ":")

m := new(dns.Msg)

m.SetQuestion(dns.Fqdn(url[0]), dns.TypeA)

m.RecursionDesired = true

config, _ := dns.ClientConfigFromFile("/etc/resolv.conf")

c := new(dns.Client)

r, _, err := c.Exchange(m, net.JoinHostPort(config.Servers[0], config.Port))

if err != nil {

return "", err

}

for _, ans := range r.Answer {

if a, ok := ans.(*dns.A); ok {

url[0] = a.A.String()

}

return strings.Join(url, ":"), nil

}