Category Archives: Administration

Shell: repeat a task multiple time

All shells (bash/zsh):

watch -n 3 "curl --location --request GET 'http://...'"

1	watch -n 3 "curl --location --request GET 'http://...'"

will run curl every 3 seconds until you break it. zsh specific:

repeat 5 { curl --location --request GET 'http://...' }

1	repeat 5 { curl --location --request GET 'http://...' }

will repeat it 5 times without any delay

Using tcpdump for k8s pods

You need wireshark to be installed on your local machine — download (more on this below). If you don’t have tcpdump installed inside the pod, you can install it with

# get pod name
kubectl get po -Lversion,cfg,infra-cfg -lapp=<pod-tag>
kubectl exec -it <pod-name> sh
# inside the pod, alpine
apk add tcpdump

# get pod name

kubectl get po -Lversion,cfg,infra-cfg -lapp=<pod-tag>

kubectl exec -it <pod-name> sh

# inside the pod, alpine

apk add tcpdump

Then, on your local machine:

kubectl exec <pod-name> -- tcpdump -i eth0 -w - | wireshark -k -i -

1	kubectl exec <pod-name> -- tcpdump -i eth0 -w - \| wireshark -k -i -

If you want to see packets in grpc/http2, you can add a rule in Wireshark. For this, click …

Forward multiple kubernetes pods for local development

kubefwd is a great tool that can forward a lot of pods at once to your local machine. The only downside for me is that it touches your local /etc/hosts. That’s how I use it:

sudo KUBECONFIG=<path to your k8s config> kubefwd svc -l "app in (first-api, second-api, third-service)"

1	sudo KUBECONFIG=<path to your k8s config> kubefwd svc -l "app in (first-api, second-api, third-service)"

It does not have huge documentation, but it is written in golang, and you can check how the source …

DD_AGENT_MAJOR_VERSION=7 DD_API_KEY=<your-key> DD_SITE="datadoghq.eu" bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)"

sudo nano /etc/datadog-agent/datadog.yaml
# uncomment logs_enabled: true
sudo systemctl restart datadog-agent
sudo mkdir /etc/datadog-agent/conf.d/go.d
sudo nano /etc/datadog-agent/conf.d/go.d/conf.yaml
# and add contents from the url below

DD_AGENT_MAJOR_VERSION=7 DD_API_KEY=<your-key> DD_SITE="datadoghq.eu" bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)"

sudo nano /etc/datadog-agent/datadog.yaml

# uncomment logs_enabled: true

sudo systemctl restart datadog-agent

sudo mkdir /etc/datadog-agent/conf.d/go.d

sudo nano /etc/datadog-agent/conf.d/go.d/conf.yaml

# and add contents from the url below

https://app.datadoghq.eu/logs/onboarding/server https://docs.datadoghq.com/agent/faq/error-restarting-agent-already-listening-on-a-configured-port/ Or, if you want a dockerized version of DataDog agent, and you want to listen to docker logs, you can do this:

docker run -d --name datadog-agent \
           -e DD_API_KEY=<DATADOG_API_KEY> \
           -e DD_LOGS_ENABLED=true \
           -e DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=true \
           -e DD_CONTAINER_EXCLUDE_LOGS="name:datadog-agent" \
           -e DD_SITE="datadoghq.eu"  \
           -v /var/run/docker.sock:/var/run/docker.sock:ro \
           -v /proc/:/host/proc/:ro \
           -v /opt/datadog-agent/run:/opt/datadog-agent/run:rw \
           -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
           datadog/agent:latest

docker run -d --name datadog-agent \

-e DD_API_KEY=<DATADOG_API_KEY> \

-e DD_LOGS_ENABLED=true \

-e DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=true \

-e DD_CONTAINER_EXCLUDE_LOGS="name:datadog-agent" \

-e DD_SITE="datadoghq.eu" \

-v /var/run/docker.sock:/var/run/docker.sock:ro \

-v /proc/:/host/proc/:ro \

-v /opt/datadog-agent/run:/opt/datadog-agent/run:rw \

-v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \

datadog/agent:latest

More info here — https://docs.datadoghq.com/agent/docker/log/?tab=containerinstallation Then you will see your logs here — https://app.datadoghq.eu/logs

Add all lines from a .env file to environment

zsh:

zsh ./configs/dev/dev.env|sed 's/#1.*$//g'|grep '^[^#]'|grep -v '^$'|sed 's/^/export /g'

1	zsh ./configs/dev/dev.env\|sed 's/#1.*$//g'\|grep '^[^#]'\|grep -v '^$'\|sed 's/^/export /g'

bash:

bash ./configs/dev/dev.env|sed 's/#1.*$//g'|grep '^[^#]'|grep -v '^$'|sed 's/^/export /g'

1	bash ./configs/dev/dev.env\|sed 's/#1.*$//g'\|grep '^[^#]'\|grep -v '^$'\|sed 's/^/export /g'

to output to stdout

cat ./configs/dev/dev.env|sed 's/#1.*$//g'|grep '^[^#]'|grep -v '^$'|sed 's/^/export /g'

1	cat ./configs/dev/dev.env\|sed 's/#1.*$//g'\|grep '^[^#]'\|grep -v '^$'\|sed 's/^/export /g'

prepend running a command (did not try it, but something similar):

env $(cat ./configs/dev/dev.env|sed 's/#.*$//g'|grep '^[^#]'|grep -v '^$'|xargs)

1	env $(cat ./configs/dev/dev.env\|sed 's/#.*$//g'\|grep '^[^#]'\|grep -v '^$'\|xargs)

Integration tests with testcontainers-go

Here is the go library that simplifies integration tests with docker containers — https://github.com/testcontainers/testcontainers-go. That’s how you can use it to test sql — https://github.com/testcontainers/testcontainers-go/blob/master/docs/examples/cockroachdb.md. Project documentation — https://golang.testcontainers.org/features/docker_compose/ The idea is to prepare the environment, build your app, then make requests and check the DB state.

How to update your k8s pod limits

kubectl edit deployment ←name-of-your-app→

1	kubectl edit deployment ←name-of-your-app→

Then search for memory (for instance) and update request/limit for it. When you quit, it will be saved automatically for you.

Load testing tools

Read logs from docker container

tail -f `docker inspect --format='{{.LogPath}}' [containername]`

1	tail -f `docker inspect --format='{{.LogPath}}' [containername]`

Interesting alternative to grpc_cli

https://github.com/ktr0731/evans It has REPL mode and CLI mode for e2e tests I think, it could be a good alternative to grpc_cli.