☸️Kubernetes 的子部分

Prepare k8s Cluster

There are many ways to build a kubernetes cluster.

Install Kuberctl

MIRROR="files.m.daocloud.io/"
VERSION=$(curl -L -s https://${MIRROR}dl.k8s.io/release/stable.txt)
[ $(uname -m) = x86_64 ] && curl -sSLo kubectl "https://${MIRROR}dl.k8s.io/release/${VERSION}/bin/linux/amd64/kubectl"
[ $(uname -m) = aarch64 ] && curl -sSLo kubectl "https://${MIRROR}dl.k8s.io/release/${VERSION}/bin/linux/arm64/kubectl"
chmod u+x kubectl
mkdir -p ${HOME}/bin
mv -f kubectl ${HOME}/bin

Build Cluster

​
MIRROR="files.m.daocloud.io/"
VERSION=v0.20.0
[ $(uname -m) = x86_64 ] && curl -sSLo kind "https://${MIRROR}github.com/kubernetes-sigs/kind/releases/download/${VERSION}/kind-linux-amd64"
[ $(uname -m) = aarch64 ] && curl -sSLo kind "https://${MIRROR}github.com/kubernetes-sigs/kind/releases/download/${VERSION}/kind-linux-arm64"
chmod u+x kind
mkdir -p ${HOME}/bin
mv -f kind ${HOME}/bin

Creating a Kubernetes cluster is as simple as kind create cluster

kind create cluster --name test

and the you can visit https://kind.sigs.k8s.io/docs/user/quick-start/ for mode detail.

MIRROR="files.m.daocloud.io/"
[ $(uname -m) = x86_64 ] && curl -sSLo minikube "https://${MIRROR}storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64"
[ $(uname -m) = aarch64 ] && curl -sSLo minikube "https://${MIRROR}storage.googleapis.com/minikube/releases/latest/minikube-linux-arm64"
chmod u+x minikube
mkdir -p ${HOME}/bin
mv -f minikube ${HOME}/bin

[Optional] disable aegis service and reboot system for aliyun

sudo systemctl disable aegis && sudo reboot

after you download binary, you can start your cluster

minikube start --kubernetes-version=v1.27.10 --image-mirror-country=cn --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers --cpus=6 --memory=24g --disk-size=100g

add alias for convinence

alias kubectl="minikube kubectl --"

and then you can visit https://minikube.sigs.k8s.io/docs/start/ for more detail.

Prerequisites

  • Hardware Requirements:

    1. At least 2 GB of RAM per machine (minimum 1 GB)
    2. 2 CPUs on the master node
    3. Full network connectivity among all machines (public or private network)
  • Operating System:

    1. Ubuntu 20.04/18.04, CentOS 7/8, or any other supported Linux distribution.
  • Network Requirements:

    1. Unique hostname, MAC address, and product_uuid for each node.
    2. Certain ports need to be open (e.g., 6443, 2379-2380, 10250, 10251, 10252, 10255, etc.)
  • Disable Swap:

    sudo swapoff -a

Steps to Setup Kubernetes Cluster

  1. Prepare Your Servers Update the Package Index and Install Necessary Packages On all your nodes (both master and worker):
sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl

Add the Kubernetes APT Repository

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF

Install kubeadm, kubelet, and kubectl

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
  1. Initialize the Master Node On the master node, initialize the Kubernetes control plane:
sudo kubeadm init --pod-network-cidr=192.168.0.0/16

The –pod-network-cidr flag is used to set the Pod network range. You might need to adjust this based on your network provider

Set up Local kubeconfig

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
  1. Install a Pod Network Add-on You can install a network add-on like Flannel, Calico, or Weave. For example, to install Calico:
​

```shell kubectl apply -f https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml ```

```shell kubectl apply -f https://docs.projectcalico.org/v3.14/manifests/calico.yaml ```

  1. Join Worker Nodes to the Cluster On each worker node, run the kubeadm join command provided at the end of the kubeadm init output on the master node. It will look something like this:
sudo kubeadm join <master-ip>:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>

If you lost the join command, you can create a new token on the master node:

sudo kubeadm token create --print-join-command
  1. Verify the Cluster Once all nodes have joined, you can verify the cluster status from the master node:
kubectl get nodes

This command should list all your nodes with the status “Ready”.

2024年3月7日

Command

2024年3月7日

Container 的子部分

CheatShett

type:
  1. remove specific image
podman rmi <$image_id>
  1. remove all <none> images
podman rmi `podamn images | grep  '<none>' | awk '{print $3}'`
  1. remove all stopped containers
podman container prune
  1. remove all docker images not used
podman image prune
  1. find ip address of a container
podman inspect --format='{{.NetworkSettings.IPAddress}}' minio-server
  1. exec into container
podman run -it <$container_id> /bin/bash
  1. run with environment
podman run -d --replace 
    -p 18123:8123 -p 19000:9000 \
    --name clickhouse-server \
    -e ALLOW_EMPTY_PASSWORD=yes \
    --ulimit nofile=262144:262144 \
    quay.m.daocloud.io/kryptonite/clickhouse-docker-rootless:20.9.3.45 

--ulimit nofile=262144:262144: 262144 is the maximum users process or for showing maximum user process limit for the logged-in user

ulimit is admin access required Linux shell command which is used to see, set, or limit the resource usage of the current user. It is used to return the number of open file descriptors for each process. It is also used to set restrictions on the resources used by a process.

  1. login registry
podman login --tls-verify=false --username=ascm-org-1710208820455 cr.registry.res.cloud.zhejianglab.com -p 'xxxx'
  1. tag image
podman tag 76fdac66291c cr.registry.res.cloud.zhejianglab.com/ay-dev/datahub-s3-fits:1.0.0
  1. push image
podman push cr.registry.res.cloud.zhejianglab.com/ay-dev/datahub-s3-fits:1.0.0
  1. remove specific image
docker rmi <$image_id>
  1. remove all <none> images
docker rmi `docker images | grep  '<none>' | awk '{print $3}'`
  1. remove all stopped containers
docker container prune
  1. remove all docker images not used
docker image prune
  1. find ip address of a container
docker inspect --format='{{.NetworkSettings.IPAddress}}' minio-server
  1. exec into container
docker exec -it <$container_id> /bin/bash
  1. run with environment
docker run -d --replace -p 18123:8123 -p 19000:9000 --name clickhouse-server -e ALLOW_EMPTY_PASSWORD=yes --ulimit nofile=262144:262144 quay.m.daocloud.io/kryptonite/clickhouse-docker-rootless:20.9.3.45 

--ulimit nofile=262144:262144: sssss

  1. copy file

    Copy a local file into container

    docker cp ./some_file CONTAINER:/work

    or copy files from container to local path

    docker cp CONTAINER:/var/logs/ /tmp/app_logs
  2. load a volume

docker run --rm \
    --entrypoint bash \
    -v $PWD/data:/app:ro \
    -it docker.io/minio/mc:latest \
    -c "mc --insecure alias set minio https://oss-cn-hangzhou-zjy-d01-a.ops.cloud.zhejianglab.com/ g83B2sji1CbAfjQO 2h8NisFRELiwOn41iXc6sgufED1n1A \
        && mc --insecure ls minio/csst-prod/ \
        && mc --insecure mb --ignore-existing minio/csst-prod/crp-test \
        && mc --insecure cp /app/modify.pdf minio/csst-prod/crp-test/ \
        && mc --insecure ls --recursive minio/csst-prod/"
2024年3月7日

Template 的子部分

DevContainer Template

2024年3月7日

DEV

2024年3月7日

Operator 的子部分

KubeBuilder

2024年3月7日

Proxy 的子部分

Daocloud

1. install container tools

systemctl stop firewalld && systemctl disable firewalld
sudo dnf install -y podman
podman run -d -P m.daocloud.io/docker.io/library/nginx
2024年3月7日

Serverless 的子部分

Knative 的子部分

Eventing 的子部分

Broker

2024年3月7日

Plugin 的子部分

Eventing Kafka Broker

2024年3月7日

Kserve 的子部分

Serving 的子部分

Inference

2024年3月7日

Generative

2024年3月7日

Canary Policy

2024年3月7日

Auto Scaling