Category Archives: golang

Golang’s sync.Map internals

Great article on the topic — https://victoriametrics.com/blog/go-sync-map/index.html. The whole series of articles is solid, so please take a look if you are interested. Let me share the most important insights. How it works Internal structure is:

type Map struct {
	mu Mutex
	read atomic.Pointer[readOnly]
	dirty map[any]*entry
	misses int
}

type Map struct {

mu Mutex

read atomic.Pointer[readOnly]

dirty map[any]*entry

misses int

}

Cleanlinter — my first golang linter

I’ve implemented my first go linter, called cleanlinter — https://github.com/bullgare/cleanlinter. It’s a simplistic linter to check golang project internal imports according to Clean Architecture pattern. Installation

go install github.com/bullgare/cleanlinter/cmd/cleanlinter@latest

1	go install github.com/bullgare/cleanlinter/cmd/cleanlinter@latest

Usage

cleanlinter \
-cleanlinter_path_to_domain=github.com/bullgare/cleanlinter/test/testdata/src/project_correct/internal/domain \
-cleanlinter_path_to_usecase=github.com/bullgare/cleanlinter/test/testdata/src/project_correct/internal/usecase \
-cleanlinter_path_to_adapter=github.com/bullgare/cleanlinter/test/testdata/src/project_correct/internal/adapter \
-cleanlinter_path_to_infra=github.com/bullgare/cleanlinter/test/testdata/src/project_correct/internal/infra \
./...

cleanlinter \

-cleanlinter_path_to_domain=github.com/bullgare/cleanlinter/test/testdata/src/project_correct/internal/domain \

-cleanlinter_path_to_usecase=github.com/bullgare/cleanlinter/test/testdata/src/project_correct/internal/usecase \

-cleanlinter_path_to_adapter=github.com/bullgare/cleanlinter/test/testdata/src/project_correct/internal/adapter \

-cleanlinter_path_to_infra=github.com/bullgare/cleanlinter/test/testdata/src/project_correct/internal/infra \

./...

To be honest, writing proper integration tests made much more time than just writing the linter. Articles, discussions, examples that helped me: https://developer20.com/custom-go-linter/ https://stackoverflow.com/questions/72933175/go-get-filepath-from-ast-file https://github.com/bkielbasa/cyclop/blob/master/pkg/analyzer/analyzer.go https://github.com/alingse/asasalint/blob/main/asasalint_test.go

DDD/Clean Architecture go linters

Origin (in Russian) Tech talk (paid access only, unfortunately): https://conf.ontico.ru/online/hl2023/details/5206668 Presentation: https://docs.google.com/presentation/d/1n5jCie-9tBw3QEEF6NsNco4mS9xYkg52/edit#slide=id.g29c6e2d30cb_0_169 Linters Dependencies between layers: https://github.com/OpenPeeDeeP/depguard https://github.com/fe3dback/go-arch-lint/ You can also check mine (very simplistic) — https://blog.bullgare.com/2024/06/cleanlinter-my-first-golang-linter/ Others: https://github.com/nishanths/exhaustive — to check using all microtypes in enums/switches. https://github.com/go-simpler/musttag + https://github.com/maranqz/golangconf2023/blob/main/tags/rules.go — to prevent using struct tags in Domain models (Value Objects). https://github.com/maranqz/gopublicfield — to prevent …

Protobuf: safer usage for buf

A drop-in replacement for buf issued by Ozon. Repository: https://github.com/easyp-tech/server Presentation (for subscribers only, unfortunately): https://conf.ontico.ru/online/hl2023/details/5206585

Maps internals in Go

Idea Map is passed as a value, but it consists of a pointer to hmap which has all the details on map implementation. So, if you change/add to map, it will be reflected everywhere. And that’s why you cannot assign to an uninitialized map (no memory allocated, no hash seed generated yet). runtime/map.go

type hmap struct {
	count     int // # live cells == size of map.  Must be first (used by len() builtin)
	B         uint8  // log_2 of # of buckets (can hold up to loadFactor * 2^B items)
	hash0     uint32 // hash seed

	buckets    unsafe.Pointer // array of 2^B Buckets. may be nil if count==0.
	oldbuckets unsafe.Pointer // previous bucket array of half the size, non-nil only when growing
	nevacuate  uintptr        // progress counter for evacuation (buckets less than this have been evacuated)
}

type hmap struct {

count int // # live cells == size of map. Must be first (used by len() builtin)

B uint8 // log_2 of # of buckets (can hold up to loadFactor * 2^B items)

hash0 uint32 // hash seed

buckets unsafe.Pointer // array of 2^B Buckets. may be nil if count==0.

oldbuckets unsafe.Pointer // previous bucket array of half the size, non-nil only when growing

nevacuate uintptr // progress counter for evacuation (buckets less than this have been evacuated)

}

Buckets …

Go’s Concurrency and Channel Internals

Go is implementing CSP (Communicating Sequential Processing): processes are communicating through channels, they can block each other while waiting for read/writes to channels. Actor model makes inter-process communications more explicit and non-blocking. CSP vs Actor explained — https://dev.to/karanpratapsingh/csp-vs-actor-model-for-concurrency-1cpg. Channels requirements goroutine-safe store and pass data across goroutines FIFO can block/unblock goroutines

Protobuf: add header parameters

They finally added it to the grpc-gateway: https://github.com/grpc-ecosystem/grpc-gateway/pull/3010/files#diff-c255ac405628aada46c25a2c9765605e9f823bc523d3739fd3fa71d4bcbf5c99. So, to use it you can just update to the version 2.14.0 or above — https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v2.14.0, and add something like this to your protofile:

        option (grpc.gateway.protoc_gen_openapiv2.options.openapiv2_operation) = {
            summary: "Create a new foo.";
            parameters: {
                headers: {
                    name: "My-Custom-Header";
                    description: "Some custom header description";
                    required: true;
                };
            };
        };

option (grpc.gateway.protoc_gen_openapiv2.options.openapiv2_operation) = {

summary: "Create a new foo.";

parameters: {

headers: {

name: "My-Custom-Header";

description: "Some custom header description";

required: true;

};

Go scheduler details

Scheduler was implemented by Dmitry Vyukov in go 1.1 and lives in runtime/proc.go. Good resources https://www.youtube.com/watch?v=-K11rY57K7k — video by Dmitry Vyukov (slides as pdf) https://www.ardanlabs.com/blog/2018/08/scheduling-in-go-part2.html — nice article by Bill Kennedy in 3 parts. Most of the images below are taken from this article. Main ideas Goroutines are very light weight (~2KB+) and very cheap …

Golang. Adding a json body to POST request in protofile

It could be a bit tricky, and I failed to find a good example for it.

Mock sql (sqlx) db on golang

I am using this library — https://pkg.go.dev/github.com/data-dog/go-sqlmock. That’s how you can use it for mocking db querying: