profile_cpu
The profile cpu gadget takes samples of stack traces.
Requirements
- Minimum Kernel Version : *5.4
*This is the minimal kernel version we have tried for this Gadget, however it's possible that it works with earlier versions.
Getting started
Running the gadget:
- kubectl gadget
- ig
$ kubectl gadget run ghcr.io/inspektor-gadget/gadget/profile_cpu:latest --map-fetch-interval 0 [flags]
$ sudo ig run ghcr.io/inspektor-gadget/gadget/profile_cpu:latest --map-fetch-interval 0 [flags]
Guide
- kubectl gadget
- ig
Here we deploy a small demo pod "random":
$ kubectl run --restart=Never --image=busybox random -- sh -c 'cat /dev/urandom > /dev/null'
pod/random created
Using the profile cpu gadget, we can see the list of stack traces.
The following command filters only for pods named "random", execute the command
and interrupt it after ~30 seconds. The --kernel-stacks-only option is passed to show only the
kernel stack traces.
$ kubectl gadget run ghcr.io/inspektor-gadget/gadget/profile_cpu:latest --podname random --kernel-stacks-only --map-fetch-interval 0
After a while press with Ctrl-C to stop trace collection
$ kubectl gadget run ghcr.io/inspektor-gadget/gadget/profile_cpu:latest --timeout 5 --podname random --kernel-stacks-only
K8S.NODE K8S.NAMESPACE K8S.PODNAME K8S.CONTAINERNAME PID COMM SAMPLES KERN_STACK
minikube-docker default random random 38130 containerd-shim 5 [0]chacha_permute; [1]ge…
minikube-docker default random random 38130 containerd-shim 1 [0]chacha_permute; [1]ge…
minikube-docker default random random 38130 containerd-shim 2 [0]chacha_block_generic;…
minikube-docker default random random 38130 containerd-shim 2 [0]chacha_permute; [1]ge…
minikube-docker default random random 38130 containerd-shim 2 [0]chacha_permute; [1]ge…
From the traces above, you can see that the pod is spending CPU time in the
Linux function urandom_read.
Instead of waiting, you can use the --timeout argument:
$ kubectl gadget run ghcr.io/inspektor-gadget/gadget/profile_cpu:latest --timeout 5 --podname random --kernel-stacks-only --map-fetch-interval 0
K8S.NODE K8S.NAMESPACE K8S.PODNAME K8S.CONTAINERNAME PID COMM SAMPLES KERN_STACK
minikube-docker default random random 38130 containerd-shim 1 [0]chacha_permute; [1]ge…
minikube-docker default random random 38130 containerd-shim 1 [0]_copy_to_iter; [1]get…
minikube-docker default random random 38130 containerd-shim 2 [0]chacha_permute; [1]ge…
minikube-docker default random random 38130 containerd-shim 5 [0]chacha_permute; [1]ge…
minikube-docker default random random 38130 containerd-shim 1 [0]chacha_permute; [1]ge…
minikube-docker default random random 38130 containerd-shim 1 [0]chacha_permute; [1]ge…
minikube-docker default random random 38130 containerd-shim 1 [0]chacha_permute; [1]ge…
minikube-docker default random random 38130 containerd-shim 1 [0]get_random_bytes_user…
Finally, we need to clean up our pod:
$ kubectl delete pod random
- Generate some kernel load:
$ docker run -d --rm --name random busybox cat /dev/urandom > /dev/null
- Start
ig:
$ sudo ig run ghcr.io/inspektor-gadget/gadget/profile_cpu:latest --kernel-stacks-only --containername random --runtimes docker --map-fetch-interval 0
- Observe the results:
RUNTIME.CONTAINERNAME KERNEL_IP COMM PID TID SAMPLES KERN_STACK
random 18446603336363922440 cat 83451 83451 2 [0]chacha_permute; [1]get_random_bytes_user; [2]urandom_read_iter;…
random 18446603336357767960 cat 83451 83451 1 [0]rcu_all_qs; [1]__cond_resched; [2]get_random_bytes_user; [3]ura…
random 18446603336363922292 cat 83451 83451 2 [0]chacha_permute; [1]get_random_bytes_user; [2]urandom_read_iter;…
random 18446603336363922360 cat 83451 83451 1 [0]chacha_permute; [1]get_random_bytes_user; [2]urandom_read_iter;…
random 18446603336363922208 cat 83451 83451 1 [0]chacha_permute; [1]get_random_bytes_user; [2]urandom_read_iter;…
random 18446603336363922380 cat 83451 83451 1 [0]chacha_permute; [1]get_random_bytes_user; [2]urandom_read_iter;…
random 18446603336363862956 cat 83451 83451 1 [0]push_pipe; [1]_copy_to_iter; [2]get_random_bytes_user; [3]urand…
random 18446603336363922252 cat 83451 83451 1 [0]chacha_permute; [1]get_random_bytes_user; [2]urandom_read_iter;…
random 18446603336363922260 cat 83451 83451 1 [0]chacha_permute; [1]get_random_bytes_user; [2]urandom_read_iter;…
random 18446603336357585584 cat 83451 83451 1 [0]mutex_spin_on_owner; [1]__mutex_lock.constprop.0; [2]__mutex_lo…
random 18446603336363922196 cat 83451 83451 1 [0]chacha_permute; [1]get_random_bytes_user; [2]urandom_read_iter;…
random 18446603336363922444 cat 83451 83451 1 [0]chacha_permute; [1]get_random_bytes_user; [2]urandom_read_iter;…
- Remove the docker container:
$ docker stop random