LSM-tree for write-intensive storages

LSM, or Log-structured Merge-Tree is used to improve the performance for write-intensive storages. 1. Writes are merged in memory using red-black tree, then flushed to disk sorted — SSTable or Sorted Strings Table. 2. For reads, we use Sparse Index to speed up the seeks, and Bloom Filter to speed up the non-present keys checks. …

Go memory hints

Below hints are related to very hot places of CPU/memory bound programs. And in regular (i/o bound) programs do not make much sense. Stack vs heap Go’s compiler goal is to allocate on the stack as much as possible. As it will reduce the amount of data that needs to be cleaned by the Garbage …

Create SQS queues on localstack container start

Put this file to your filesystem as ←anyname.sh→:

#!/usr/bin/env bash

set -euo pipefail

# enable debug
# set -x


echo "configuring sqs"
echo "==================="
LOCALSTACK_HOST=localhost
AWS_REGION=eu-central-1


# https://docs.aws.amazon.com/cli/latest/reference/sqs/create-queue.html
create_queue() {
	local QUEUE_NAME_TO_CREATE=$1
	awslocal --endpoint-url=http://${LOCALSTACK_HOST}:4566 sqs create-queue --queue-name ${QUEUE_NAME_TO_CREATE} --region ${AWS_REGION} --attributes VisibilityTimeout=30
}

create_queue "queue2"
create_queue "queue1"

#!/usr/bin/env bash

set -euo pipefail

# enable debug

# set -x

echo "configuring sqs"

echo "==================="

LOCALSTACK_HOST=localhost

AWS_REGION=eu-central-1

# https://docs.aws.amazon.com/cli/latest/reference/sqs/create-queue.html

create_queue() {

local QUEUE_NAME_TO_CREATE=$1

awslocal --endpoint-url=http://${LOCALSTACK_HOST}:4566 sqs create-queue --queue-name ${QUEUE_NAME_TO_CREATE} --region ${AWS_REGION} --attributes VisibilityTimeout=30

}

create_queue "queue2"

create_queue "queue1"

For instance, ./localstack_bootstrap/sqs_bootstrap.sh. Don’t forget to run chmod +x ./localstack_bootstrap/sqs_bootstrap.sh. Add this line to your docker-compose.yml: — ./localstack_bootstrap:/docker-entrypoint-initaws.d/. So it could look like this:

services:
  localstack:
    container_name: aws_sqs
    hostname: sqs
    image: localstack/localstack:latest
    environment:
      - AWS_DEFAULT_REGION=ap-southeast-1
      - EDGE_PORT=4566
      - SERVICES=sqs
    ports:
      - '4566:4566'
    volumes:
      - "${TMPDIR:-/tmp/localstack}:/tmp/localstack"
      - "/var/run/docker.sock:/var/run/docker.sock"
      - ./localstack_bootstrap:/docker-entrypoint-initaws.d/

services:

localstack:

container_name: aws_sqs

hostname: sqs

image: localstack/localstack:latest

environment:

- AWS_DEFAULT_REGION=ap-southeast-1

- EDGE_PORT=4566

- SERVICES=sqs

ports:

- '4566:4566'

volumes:

- "${TMPDIR:-/tmp/localstack}:/tmp/localstack"

- "/var/run/docker.sock:/var/run/docker.sock"

- ./localstack_bootstrap:/docker-entrypoint-initaws.d/

Then your queues can be reached by URLs http://localhost:4566/000000000000/queue1 and http://localhost:4566/000000000000/queue2. Further reading: https://docs.aws.amazon.com/cli/latest/reference/sqs/create-queue.html https://joerg-pfruender.github.io/software/docker/microservices/testing/2020/01/25/Localstack_in_Docker.html

Postgresql index naming convention

Use the table name as the prefix. Use the following suffixes: _pkey for a Primary Key constraint _key for a Unique constraint _excl for an Exclusion constraint _idx for any other kind of index _fkey for a Foreign key _check for a Check constraint https://stackoverflow.com/a/4108266/801426 Also implicitly mentioned here — https://www.postgresql.org/docs/current/sql-altertable.html

Heroes 3 on MacOS with M1 Apple Silicon

Initial installation 1. You need Parallels Desktop with M1 support — https://www.parallels.com/ru/products/desktop/trial/ 2. Also you need an insider account on Microsoft web site to download win 11 for arm — https://insider.windows.com/ru-ru/ 3. Then you need to download Win 11 for arm — https://www.microsoft.com/en-us/software-download/windowsinsiderpreviewARM64 4. Download Heroes 3 Hota on the internet (many links are there). …

FreeCAD to generate gcode

You should switch Workbench to «Mesh Design», then select your shape in the left bar, and go to Menu→Meshes→Create Mesh from the shape. More details on parameters — https://www.xsim.info/articles/FreeCAD/en-US/HowTo/Create-surface-meshes-from-solids.html

DMDE to recover HDD disks

https://dmde.com/download.html

MGTS(GPON)+Keenetic and external access to Synology

Adding a comment to an article about using Keenetic router behind an MGTS(gpon) receiver to make Synology services like DS Audio/Video/Photo and so on to be available via Quickconnect feature by Synology. First of all, enable KeenDNS like shown below And then enable it on Synology Control Panel like this: And now on any device …

Useful links for proto generator development

https://rotemtam.com/2021/03/22/creating-a-protoc-plugin-to-gen-go-code/ https://medium.com/@tim.r.coulson/writing-a-protoc-plugin-with-google-golang-org-protobuf-cd5aa75f5777 https://github.com/nametake/protoc-gen-gohttp https://github.com/drekle/protoc-gen-goexample/blob/c60883c9711f3cfbd5191fd697651dbd067e6fdd/main.go#L135 https://github.com/grpc-ecosystem/grpc-gateway/blob/24434e22fb9734f1a62c81c4ea246125d7844645/examples/internal/proto/examplepb/echo_service_grpc.pb.go https://pkg.go.dev/google.golang.org/protobuf/compiler/protogen#Options.Run https://github.com/lyft/protoc-gen-star https://go.dev/blog/protobuf-apiv2 https://docs.buf.build/build/usage/#configuration https://github.com/grpc-ecosystem/grpc-gateway https://github.com/googleapis/googleapis/tree/master/google/api https://github.com/bufbuild/buf https://stackoverflow.com/questions/36540427/cannot-find-package-google-protobuf https://pkg.go.dev/google.golang.org/protobuf https://developers.google.com/protocol-buffers/docs/reference/go-generated#package https://stackoverflow.com/questions/61666805/correct-format-of-protoc-go-package https://grpc.io/docs/languages/go/quickstart/ https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf https://github.com/gogo/protobuf/blob/b03c65ea87cdc3521ede29f62fe3ce239267c1bc/protobuf/google/protobuf/descriptor.proto

Go study

A list of resources for learning golang https://github.com/ardanlabs/gotraining/tree/master/reading