RKE2

RKE2, also known as RKE Government, is Rancher's next-generation Kubernetes distribution.

Install

curl -sfL https://get.rke2.io | sh -

NetworkManager가 호스트에 설치되어 활성화되어 있는 경우 CNI 관리 인터페이스를 무시하도록 구성되어 있는지 확인하십시오. -> #NetworkManager Issues 항목을 참조

Firewalld conflicts with default networking

Firewalld 는 RKE2의 기본 Canal(Calico + Flannel) 네트워킹 스택과 충돌합니다. 예기치 않은 동작을 방지하려면 RKE2를 실행하는 시스템에서 방화벽을 비활성화해야 합니다.

NetworkManager Issues

NetworkManager는 RKE2의 기본값을 포함하여 많은 CNI가 컨테이너 연결을 위한 veth 쌍을 생성하는 기본 네트워크 네임스페이스의 인터페이스에 대한 라우팅 테이블을 조작합니다. 이는 CNI의 올바른 라우팅 기능을 방해할 수 있습니다. 따라서 NetworkManager 지원 시스템에 RKE2를 설치하는 경우 Calico/flannel 관련 네트워크 인터페이스를 무시하도록 NetworkManager를 구성하는 것이 좋습니다.

이를 수행하려면 /etc/NetworkManager/conf.d/rke2-canal.conf 파일에 다음 내용으로 호출 되는 구성 파일을 생성하십시오:

[keyfile]
unmanaged-devices=interface-name:cali*;interface-name:flannel*

RKE2를 아직 설치하지 않은 경우 간단히 systemctl reload NetworkManager구성을 설치하는 것으로 충분합니다.
이미 RKE2가 설치된 시스템에서 이 구성 변경을 수행하는 경우 변경 사항을 효과적으로 적용하려면 노드를 재부팅해야 합니다.

INFORMATION

RHEL 8.4와 같은 일부 운영 체제에서는 NetworkManager에 nm-cloud-setup.service및 nm-cloud-setup.timer이라는 두 가지 추가 서비스가 포함되어 있습니다. 이러한 서비스는 CNI 플러그인 구성을 방해하는 라우팅 테이블을 추가합니다. 안타깝게도 문제 에 설명된 대로 이를 방지할 수 있는 구성은 없습니다 . 따라서 해당 서비스가 존재하는 경우 비활성화해야 합니다.

INFORMATION
NetworkManager-1.30.0-11.el8_4 이전에는 추가 서비스를 비활성화한 후 노드도 재부팅해야 합니다.

Air-Gap Install

Air-Gap Install | RKE2

Troubleshooting

Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"

서비스 재 시작 명령시

sudo systemctl start rke2-server.service

장시간 멎어있다 다음 에러 메시지가 출력된다:

Job for rke2-server.service failed because the control process exited with error code.
See "systemctl status rke2-server.service" and "journalctl -xe" for details.

journalctl -xe명령으로 로그를 확인하면:

--
-- The unit run-k3s-containerd-io.containerd.runtime.v2.task-k8s.io-c6d4f5b0a29a907870230685397e3b39f0e4cfaa6c90b57cc18ed91937ade090-rootfs.mount has successfully entered the 'dead' state.
Jun 13 21:23:55 server rke2[973979]: time="2024-06-13T21:23:55+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:23:56 server systemd[958]: var-lib-rancher-rke2-agent-containerd-tmpmounts-containerd\x2dmount1672672385.mount: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- The unit UNIT has successfully entered the 'dead' state.
Jun 13 21:23:56 server systemd[1]: Started libcontainer container 389741e7c2d451274168c88eb6df535fdba257e75b8ca6d53a505b7817943b87.
-- Subject: A start job for unit cri-containerd-389741e7c2d451274168c88eb6df535fdba257e75b8ca6d53a505b7817943b87.scope has finished successfully
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A start job for unit cri-containerd-389741e7c2d451274168c88eb6df535fdba257e75b8ca6d53a505b7817943b87.scope has finished successfully.
--
-- The job identifier is 56217.
Jun 13 21:23:58 server rke2[973979]: time="2024-06-13T21:23:58+09:00" level=error msg="Kubelet exited: exit status 255"
Jun 13 21:23:58 server rke2[973979]: time="2024-06-13T21:23:58+09:00" level=info msg="Waiting for API server to become available"
Jun 13 21:23:58 server rke2[973979]: time="2024-06-13T21:23:58+09:00" level=info msg="Waiting for API server to become available"
Jun 13 21:24:00 server rke2[973979]: time="2024-06-13T21:24:00+09:00" level=warning msg="Failed to list nodes with etcd role: runtime core not ready"
Jun 13 21:24:00 server rke2[973979]: time="2024-06-13T21:24:00+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:05 server rke2[973979]: time="2024-06-13T21:24:05+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:10 server rke2[973979]: time="2024-06-13T21:24:10+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:15 server rke2[973979]: time="2024-06-13T21:24:15+09:00" level=warning msg="Failed to list nodes with etcd role: runtime core not ready"
Jun 13 21:24:15 server rke2[973979]: time="2024-06-13T21:24:15+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:20 server rke2[973979]: time="2024-06-13T21:24:20+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:25 server rke2[973979]: time="2024-06-13T21:24:25+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:28 server rke2[973979]: time="2024-06-13T21:24:28+09:00" level=info msg="Waiting for API server to become available"
Jun 13 21:24:28 server rke2[973979]: time="2024-06-13T21:24:28+09:00" level=error msg="Kubelet exited: exit status 255"
Jun 13 21:24:28 server rke2[973979]: time="2024-06-13T21:24:28+09:00" level=info msg="Waiting for API server to become available"
Jun 13 21:24:30 server rke2[973979]: time="2024-06-13T21:24:30+09:00" level=warning msg="Failed to list nodes with etcd role: runtime core not ready"
Jun 13 21:24:30 server rke2[973979]: time="2024-06-13T21:24:30+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:35 server rke2[973979]: time="2024-06-13T21:24:35+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:40 server rke2[973979]: time="2024-06-13T21:24:40+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:40 server sudo[977843]:     your : TTY=pts/0 ; PWD=/home/your ; USER=root ; COMMAND=/usr/bin/systemctl daemon-reload
Jun 13 21:24:40 server sudo[977843]: pam_unix(sudo:session): session opened for user root by your(uid=0)
Jun 13 21:24:40 server systemd[1]: Reloading.
Jun 13 21:24:41 server sudo[977843]: pam_unix(sudo:session): session closed for user root
Jun 13 21:24:43 server sudo[977903]:     your : TTY=pts/0 ; PWD=/home/your ; USER=root ; COMMAND=/usr/bin/systemctl start rke2-server.service
Jun 13 21:24:43 server sudo[977903]: pam_unix(sudo:session): session opened for user root by your(uid=0)
Jun 13 21:24:45 server rke2[973979]: time="2024-06-13T21:24:45+09:00" level=warning msg="Failed to list nodes with etcd role: runtime core not ready"
Jun 13 21:24:45 server rke2[973979]: time="2024-06-13T21:24:45+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"
Jun 13 21:24:50 server rke2[973979]: time="2024-06-13T21:24:50+09:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: https://127.0.0.1:9345/v1-rke2/readyz: 500 Internal Server Error"

9345 포트 관련 문제로 생각된다. iptables 같은 방화벽을 확인하자.