Tag Archives: profiling

How to work with core dumps of Go programs

If you want to check what is going on in your service during load (on production or during stress testing process), you can take a core dump of your app and load it into Goland for further debugging.

# first you need to get your app's PID
ps aux|grep <your-app-bin-name> 
gcore <PID>
# download dump file and binary file to a local machine with
kubectl cp <some-namespace>/<some-pod>:/core.<PID> ~/Downloads
kubectl cp <some-namespace>/<some-pod>:<your-app-bin-path> ~/Downloads

# first you need to get your app's PID

ps aux|grep <your-app-bin-name>

gcore <PID>

# download dump file and binary file to a local machine with

kubectl cp <some-namespace>/<some-pod>:/core.<PID> ~/Downloads

kubectl cp <some-namespace>/<some-pod>:<your-app-bin-path> ~/Downloads

Then open it your Goland: Navigate to Run | Open Core Dump. In the Executable field, …

Container insights for golang engineer (k8s docker CPU throttling)

We’ve run into problems on my work when k8s limits cpu to few processors (throttling), while go application can see all of them (setting GO_MAX_PROCS to maximum CPUs available) and go’s scheduler is going crazy because of that during highload (like stress-tests). It is caused by scheduling for, let’s say, 40 processors while you have …

Инструмент для визуализации зависимостей в go

https://github.com/TrueFurby/go-callvis Вызов:

go-callvis -focus [packagename] ./cmd/main.go > goblin.dot

1	go-callvis -focus [packagename] ./cmd/main.go > goblin.dot

Поиск утечек памяти через Google Chrome

На примере Яндекс-почты: http://habrahabr.ru/company/yandex/blog/195198/ Самое интересное:

Realtime-профилирование проекта от Instagram

Measurements: счетчики и таймеры Дабы следить за всем, что происходить внутри, надо как-то мониторить всю активность. Обычно это два типа активности: какие-то количества (counters, регистраций в секунду, например) и какие-то интервалы времени (timers, сколько по времени занимает лайкнуть винтажное фото, например). Но как быть, когда у тебя не одна машина на балконе, а пара десятков …

SELECT
  t.TABLE_SCHEMA AS `db`, t.TABLE_NAME AS `table`, s.INDEX_NAME AS `index name`
 , s.COLUMN_NAME AS `field name`, s.SEQ_IN_INDEX `seq in index`, s2.max_columns AS `# cols`
 , s.CARDINALITY AS `card`, t.TABLE_ROWS AS `est rows`
 , ROUND(((s.CARDINALITY / IFNULL(t.TABLE_ROWS, 0.01)) * 100), 2) AS `sel %`
FROM INFORMATION_SCHEMA.STATISTICS s
 INNER JOIN INFORMATION_SCHEMA.TABLES t
  ON s.TABLE_SCHEMA = t.TABLE_SCHEMA AND s.TABLE_NAME = t.TABLE_NAME
 INNER JOIN (
  SELECT TABLE_SCHEMA, TABLE_NAME, INDEX_NAME, MAX(SEQ_IN_INDEX) AS max_columns
  FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_SCHEMA != 'mysql'
  GROUP BY TABLE_SCHEMA, TABLE_NAME, INDEX_NAME
 ) AS s2
 ON s.TABLE_SCHEMA = s2.TABLE_SCHEMA AND 
    s.TABLE_NAME = s2.TABLE_NAME AND 
    s.INDEX_NAME = s2.INDEX_NAME
WHERE t.TABLE_SCHEMA != 'mysql'       /* Filter out the mysql system DB */
AND t.TABLE_ROWS > 10                 /* Only tables with some rows */
AND s.CARDINALITY IS NOT NULL         /* Need at least one non-NULL value in the field */
AND (s.CARDINALITY / IFNULL(t.TABLE_ROWS, 0.01)) < 1.00 /* unique indexes are perfect anyway */
ORDER BY `sel %`, s.TABLE_SCHEMA, s.TABLE_NAME          /* DESC for best non-unique indexes */
LIMIT 10;

SELECT

t.TABLE_SCHEMA AS `db`, t.TABLE_NAME AS `table`, s.INDEX_NAME AS `index name`

, s.COLUMN_NAME AS `field name`, s.SEQ_IN_INDEX `seq in index`, s2.max_columns AS `# cols`

, s.CARDINALITY AS `card`, t.TABLE_ROWS AS `est rows`

, ROUND(((s.CARDINALITY / IFNULL(t.TABLE_ROWS, 0.01)) * 100), 2) AS `sel %`

FROM INFORMATION_SCHEMA.STATISTICS s

INNER JOIN INFORMATION_SCHEMA.TABLES t

ON s.TABLE_SCHEMA = t.TABLE_SCHEMA AND s.TABLE_NAME = t.TABLE_NAME

INNER JOIN (

SELECT TABLE_SCHEMA, TABLE_NAME, INDEX_NAME, MAX(SEQ_IN_INDEX) AS max_columns

FROM INFORMATION_SCHEMA.STATISTICS WHERE TABLE_SCHEMA != 'mysql'

GROUP BY TABLE_SCHEMA, TABLE_NAME, INDEX_NAME

) AS s2

ON s.TABLE_SCHEMA = s2.TABLE_SCHEMA AND

s.TABLE_NAME = s2.TABLE_NAME AND

s.INDEX_NAME = s2.INDEX_NAME

WHERE t.TABLE_SCHEMA != 'mysql' /* Filter out the mysql system DB */

AND t.TABLE_ROWS > 10 /* Only tables with some rows */

AND s.CARDINALITY IS NOT NULL /* Need at least one non-NULL value in the field */

AND (s.CARDINALITY / IFNULL(t.TABLE_ROWS, 0.01)) < 1.00 /* unique indexes are perfect anyway */

ORDER BY `sel %`, s.TABLE_SCHEMA, s.TABLE_NAME /* DESC for best non-unique indexes */

LIMIT 10;

Это из лекции товарища Jay Pipes