本文主要讨论如何在containerd中使用gvisor作为runtime，以及如何对接k8s。
安装containerd
这个就不用多说了，直接apt install containerd 即可，会安装一下几个可执行文件，如下所示
/usr/bin/containerd 守护进程
/usr/bin/containerd-shim 应该是负责处理容器内stdout & stderr
/usr/bin/containerd-shim-runc-v1 应该是负责启动容器内的pid 1
/usr/bin/containerd-stress 压力测试工具
/usr/bin/ctr cli工具
吐槽一下 ctr可能没有docker的cli那么好用
接下来我们来运行一个容器测试一下containerd：
- 因为containerd默认的pause容器使用的是gcr.k8s.io/pause，如果网络有问题的话，就需要提前导入，或者在修改默认配置。
- 还要提醒一下，containerd有namespace的区别，默认是在default下面，这里不是k8s的namespace 。
- 除此之外还要配置CNI以供容器启动分配网络，可以使用下面的配置：
mkdir -p /etc/cni/net.d
cat >'/etc/cni/net.d'/10-containerd-net.conflist <<EOF
{
"cniVersion": "0.3.1",
"name": "containerd-net",
"plugins": [
{
"type": "bridge",
"bridge": "cni1",
"isGateway": true,
"ipMasq": true,
"promiscMode": true,
"ipam": {
"type": "host-local",
"subnet": "10.88.0.0/16",
"routes": [
{ "dst": "0.0.0.0/0" }
]
}
},
{
"type": "portmap",
"capabilities": {"portMappings": true}
}
]
}
EOF
接下来就可以真正开始运行容器了。
先拉镜像
ctr i pull docker.io/library/nginx:alpine
运行
ctr run -d docker.io/library/nginx:alpine demo
检查一下 containerd会把容器作为task
ctr c ls
CONTAINER IMAGE RUNTIME
demo docker.io/library/nginx:alpine io.containerd.runtime.v1.linux
ctr t ls
TASK PID STATUS
demo 19901 RUNNING
这里停掉containerd的守护进程，可以发现已经运行的容器是不受影响的
pstree
systemd─┬─VBoxService───7*[{VBoxService}]
├─accounts-daemon───2*[{accounts-daemon}]
├─2*[agetty]
├─atd
├─containerd-shim─┬─nginx───2*[nginx]
│ └─9*[{containerd-shim}]
├─cron
├─dbus-daemon
├─irqbalance───{irqbalance}
├─lvmetad
├─lxcfs───6*[{lxcfs}]
├─networkd-dispat───{networkd-dispat}
├─polkitd───2*[{polkitd}]
├─rsyslogd───3*[{rsyslogd}]
├─sshd───sshd───sshd───bash───sudo───su───bash─┬─containerd───7*[{containerd}]
│ └─pstree
├─systemd───(sd-pam)
├─systemd-journal
├─systemd-logind
├─systemd-network
├─systemd-resolve
├─systemd-udevd
└─unattended-upgr───{unattended-upgr}
配置gvisor
首先需要编译或者下载runsc，也就是gvisor的运行时，然后需要下载containerd-shim-runsc-v1 放到PATH任意一个目录中
# 示例配置
root = "/var/lib/containerd"
state = "/run/containerd"
oom_score = 0
[grpc]
address = "/run/containerd/containerd.sock"
uid = 0
gid = 0
max_recv_message_size = 16777216
max_send_message_size = 16777216
[debug]
address = ""
uid = 0
gid = 0
level = ""
[metrics]
address = ""
grpc_histogram = false
[cgroup]
path = ""
[plugins]
[plugins.cgroups]
no_prometheus = false
[plugins.cri]
stream_server_address = "127.0.0.1"
stream_server_port = "0"
enable_selinux = false
sandbox_image = "gcr.azk8s.cn/google_containers/pause:3.1"
stats_collect_period = 10
systemd_cgroup = false
enable_tls_streaming = false
max_container_log_line_size = 16384
[plugins.cri.containerd]
snapshotter = "overlayfs"
no_pivot = false
[plugins.cri.containerd.default_runtime]
runtime_type = "io.containerd.runtime.v1.linux"
runtime_engine = ""
runtime_root = ""
[plugins.cri.containerd.untrusted_workload_runtime]
runtime_type = ""
runtime_engine = ""
runtime_root = ""
[plugins.cri.containerd.runtimes.runsc]
runtime_type = "io.containerd.runsc.v1"
[plugins.cri.cni]
bin_dir = "/opt/cni/bin"
conf_dir = "/etc/cni/net.d"
conf_template = ""
[plugins.cri.registry]
[plugins.cri.registry.mirrors]
[plugins.cri.registry.mirrors."docker.io"]
endpoint = ["https://registry-1.docker.io"]
[plugins.cri.x509_key_pair_streaming]
tls_cert_file = ""
tls_key_file = ""
[plugins.diff-service]
default = ["walking"]
[plugins.linux]
shim = "containerd-shim"
runtime = "runc"
runtime_root = ""
no_shim = false
shim_debug = false
[plugins.opt]
path = "/opt/containerd"
[plugins.restart]
interval = "10s"
[plugins.scheduler]
pause_threshold = 0.02
deletion_threshold = 0
mutation_threshold = 100
schedule_delay = "0s"
startup_delay = "100ms"
对接k8s
对于kubeadm，初始化的时候指定containerd的sock即可，如下所示：
kubeadm init --cri-socket /run/containerd/containerd.sock --image-repository gcr.azk8s.cn/google_containers --kubernetes-version 1.16.3 --apiserver-advertise-address 192.168.2.2 --pod-network-cidr=10.244.0.0/16
# 创建runtimeclass
cat <<EOF | kubectl apply -f -
apiVersion: node.k8s.io/v1beta1
kind: RuntimeClass
metadata:
name: gvisor
handler: runsc
EOF
# 运行pod
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: nginx-gvisor
spec:
runtimeClassName: gvisor
containers:
- name: nginx
image: nginx:alpine
EOF
Ref
Doc