Maitiro ekugadzira Turing Pi 2 Kumba Cluster

Zviri Mukati

• Nyaya
• The Plan
  • Kutora Pi 2
• The Setup
• The Flashing
  • The Storage
  • The Kubernetes
• The Applications
• Chigadzirwa chekupedzisira
• Ramangwana
• Magumo

Nyaya

Muna Chikumi 2022, ndakasangana neKickstarter nezve bhodhi rinokwanisa kuve neanosvika mana Raspberry Pi 4 Compute Modules kamwechete.

Pasina kunyanya kufunga, ndakasarudza kutsigira chirongwa uye kutenga bhodhi.

Kurumidza-mberi makore maviri, uye yangu Turing Pi 2 yakanga ichiri kugara isina kuvhurwa mubhokisi rayo pasherufuHupenyu hwakanga hwapinda munzira, uye ndakanga ndisina chokwadi chekuti nei ndakanga ndazvitenga pakutanga.

Zvisinei, ndakazofunga kumbozviedza. Ini ndaida kudzidza zvakawanda nezve clustering uye ndakanga ndisati ndambovaka yakazara Kubernetes cluster kubva pakutanga. Saka, ndakaenda mumashandisirwo emari ndikawana matatu eRaspberry Pi 4s (8GB RAM, 8GB yekuchengetedza mukati) uye imwe Nvidia Jetson Nano (4GB).

Tichifunga nezvekusiyana-siyana kwebhodhi, ndaigona kusanganisa akasiyana Compute Modules. Ndakafunga kusanganisa Jetson Nano, ndichifunga kuti zvinganditendera kuti ndiedze vatyairi veCUDA mune ramangwana uye kunyura mukudzidza kwemichina. Ndiani anoziva? Ini ndingatozopedzisira ndatambira yangu yangu GPT mubatsiri pane ino Kubernetes cluster.
(Spoilers: It didn't happen)

The Plan

Chirongwa changu chekutanga chaisanganisira iyo 3 Pi 4 CM uye Jetson Nano inogarwa pabhodhi. Zvakare, ndakaronga kushandisa 1TB SSD drive yekuchengetedza uye Wi-Fi kadhi yekuwana internet. Nekudaro, mushure mekusangana nematambudziko mazhinji neiyo Jetson Nano yekuseta maitiro uye isina kunaka zvinyorwa, ndakafunga kuidzosera. Pane kudaro, ndakasarudza yechina Raspberry Pi 4.

Pamusoro pezvo, ndaive neRaspberry Pi 4 yekare ine 4GB ye RAM yakarara yakatenderedza, saka ndakafunga kuibatanidza sechishanu node.

Kutora Pi 2

Iyo Turing Pi 2 ndeye Mini ITX fomu factor board iyo inogona kugara kusvika mana Raspberry Pi Compute Modules (inoenderanawo neJetson Nanos uye Turing Compute Module). Iyo inoratidzira PCI Express chiteshi, maviri NVME ports, maviri SATA ports, uye USB port yekupenya iyo Compute Modules.

• Nhamba 1 :

  • USB 2.0 port (Yekuvheneka iyo Compute Modules)

  • HDMI port (Yekugadzirisa)

  • PCI Express port (Yeiyo Wi-Fi kadhi)

    
    

• Nhamba 2 :

  • Ini ndaizoshandisa iyi yekuchengetedza NVME, asi haienderane neRaspberry Pi 4s.

    
    

• Nhamba 3 :

  • Iwo maSATA ports, zvisinei, anogona kushandiswa pano. Saka, iyi node ichava neNFS yakagovaniswa drive.

    
    

• Nhamba 4 :

  • USB 3.0 ports (Kana ndikambodzida).

    
    

• Raspberry yangu yekare :

  • Ingave iyo Kubernetes master node. Hapana chikonzero chakakosha; Ndinogona kufunga zvirinani nezve setup yangu iyi

  
  

Pakupedzisira, pfungwa ndeyekugamuchira Media Server ine mamwe ma-add-ons.

The Set-up

Pave nechinguva kubva pandakaisa komputa, uye yaive nguva yekutanga kutamba ndichitenderera neCompute Modules nemaadapter avo, saka izvo zvaive zvekunakidzwa kwevhiki. Sezvo chikwama changu chaipisa asi chichiri kutsva, ndakafunga, nei gehena isingawedzere nyaya yakanaka pairi?

Tichipihwa iyo Mini ITX fomu chinhu chebhodhi, ndaigona kuikwana mune chero yakanakisa ITX kesi yandaigona kuwana paAmazon. Iyo Qube 500 yakandipedza nekupedza. Ini ndanga ndatogadzira DIY cluster, yakanakisa kesi yechinhu chakadaro, yaive DIY futi.

Ini ndakawedzerawo 650W magetsi ekupa (yakazara kuwanda), imwe diki Wi-Fi Mini PCI Express kadhi, uye 1TB SATA SSD.

Kuisa "chinhu" pamwe chete kwaive nyore. A bit of thermal paste pakati peCompute Modules nekupisa kwavo kunonyura uye kuabatanidza pamwe chete nemaAdhapter avo vasati vaamisa mukurongeka muTuring Board;

Ndakataura kurongeka nekuti chaive chikamu chakakosha cheprojekiti. Iyo Turing Pi 2 inopa manejimendi emadoko ayo akagoverwa mukati memakomputa modules. Muchiitiko ichi, PCI Express 1 yaitungamirirwa neFirst node apo SSD drive yaitungamirirwa neyechitatu. Yechipiri yaigona kubata chiteshi cheNVME uye chechina pane imwe SSD IIRC, asi ini ndakanga ndisisina basa navo izvozvi.

The Flashing

Ndakaisa Raspberry Pis kare asi handina kumbobvira ndaverenga Modules. Iyo Turing Pi 2 ine USB port kumashure iyo inoshandiswa kupenya iyo Compute Modules.

Nehurombo, ndakaedza kushandisa USB A ku USB A tambo yakanga isiri tambo yekufambisa data, saka pandakanga ndakamirira Amazon kuendesa tambo, ndakawana imwe nzira yekuvheneka maCompute Modules.

Iyo Turing Pi 2 ine CLI chishandiso chinogona kushandiswa kwete kungovheneka maCompute Modules asiwo kubata simba rawo, kuamisazve, kutarisa mamwe manhamba, zvichingodaro.

Murairo wakashandiswa kuisa iyo Compute Modules waive:

 tpi flash -i /path/to/image -n {nodes 1 to 4}

Yakanaka yakatwasuka maitiro ndakazvifunga ndisati ndaona kuti mufananidzo weRaspbian hauuye neSSH inogoneswa nekusarudzika.

Iri, hongu, harisi basa raTuring. Ndingadai ndakamirira tambo iyoyo, asi zvakanaka.

Kuti ndigadzirise izvi, ndaifanira kukwira chifananidzo pamushini wangu wepanzvimbo uye nekuwedzera faira risina chinhu rakanzi ssh muchikamu chebhoti. Izvi zvaizogonesa SSH nekukasira.

 sudo mkdir /mnt/pi-boot sudo mount /dev/sdX1 /mnt/pi-boot sudo touch /mnt/pi-boot/ssh sudo umount /mnt/pi-boot

Zvino, pis dzangu dzese dzakagadzirira kushandiswa. Ndakavabatanidza network ndokutanga kudzigadzirisa. Pakanga paine zvishoma zvekugadziriswa sezvo ini ndaizovashandisa seKubernetes node.

Asi zvinhu zvakaita sevim nekuvandudza sisitimu zvaive zvakakosha.

Izvi zvakandipawo mukana wekudzidza kushandisa Tmux . Chishandiso chakanakisa chandakadzidza munguva pfupi.

The Storage

Kana iwe ukarangarira mashoma ndima pamusoro, ini ndakataura kuti iyo 3rd node yaizoshandiswa kune iyo NFS yakagovaniswa drive. Ndakanga ndine 1TB SSD drive yandaizoshandisa nekuda kweichi chinangwa. Ini ndaifanira kuifomatidza uye nekuiisa mune yechitatu node.

Asi ini ndaidawo kuisa sevha yeNFS mune ino node uye kuigadzirisa mune dzimwe node. Izvi zvinokurudzirwa nharaunda yekugadzira? Helo kwete, asi iboka repamba, saka handina kunyanya kunetseka nazvo.

Heano nhanho dzandakatora kugadzirisa iyo NFS server:

 pi@turing-03:/mnt/ssd/data $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS sda 8:0 0 953.9G 0 disk /mnt/ssd mmcblk0 179:0 0 7.3G 0 disk ├─mmcblk0p1 179:1 0 512M 0 part /boot/firmware └─mmcblk0p2 179:2 0 6.8G 0 part / mmcblk0boot0 179:32 0 4M 1 disk mmcblk0boot1 179:64 0 4M 1 disk

Chekutanga, ndakatarisa dhiraivha yaive yakaiswa nemazvo. Ipapo ndakaisa sevha yeNFS:

 sudo mkdir /mnt/nfs_share sudo mount /dev/sda /mnt/nfs_share

Yakawedzera kune fstab faira kuti ive nechokwadi chekuti yakaiswa pabhoti:

(kuwedzera mutsara unotevera kune /etc/fstab faira)

 echo '/dev/sda /mnt/ssd ext4 defaults 0 0' | sudo tee -a /etc/fstab

Zvino, kuisa nfs-kernel-server:

 sudo apt update sudo apt install nfs-kernel-server

Uye kuwedzera drive yangu kune /etc/exports faira:

 echo '/mnt/ssd *(rw,sync,no_subtree_check,no_root_squash)' | sudo tee -a /etc/exports

Zvino, pane dzimwe node, ndaifanira kuisa nfs-yakajairika:

 sudo apt update sudo apt install nfs-common

Uye kubatanidza drive kune imwe neimwe node:

 sudo mount -t nfs {IP-for-the-drives-node}:/mnt/ssd /mnt

Neofetch yakaiswa mumanode ese nekuti ini ndiri fancy.

Iyo Kubernetes Cluster

Ndakanga ndisati ndambogadzira boka reKubernetes kubva pakutanga, asi ndanga ndichiona mavhidhiyo akawanda aJeff Geerling panyaya iyi ... Ichi chiitiko chakakwana, handizvo?

Jeff akanditungamirira kuK3s ndichishandisa Ansible , kugoverwa kweKubernetes kwakareruka iyo yakakwana yeboka rangu reimba uye nzira yakagara yakatsanangurwa yekuiisa nokuti ini handina pre-requirements kana pfungwa yekuimisa sei neimwe nzira.

Kuiswa kwacho kwaive kwakananga. Ndaifanira kuiisa mumanode ese, asi ndaifanira kuita chokwadi chekuti master node ndiyo yaive yekutanga kuiswa.

Saka chekutanga, ndakagadzira iyo k3s-inogoneka repository:

 git clone https://github.com/k3s-io/k3s-ansible.git

Ipapo ini ndaifanira kugadzirisa iyo inventory faira. Tenzi wangu node, sezvandambotaura kare, yaiva yangu yekare Raspberry Pi 4. Saka ndaifanira kuve nechokwadi kuti ndiyo yekutanga mufaira yezvinyorwa. Ini ndaifanirawo kuona kuti mamwe ma node aive mumapoka chaiwo.:

 k3s_cluster: children: server: hosts: 192.168.2.105: agent: hosts: 192.168.2.101: 192.168.2.102: 192.168.2.103: 192.168.2.104:

Mune iro faira rimwe chete, ndaifanira kuseta encryption tokeni. Iyo faira inoratidza maitiro ekuita izvi, saka ini handisi kuzopinda mune zvakadzama pano.

Ipapo ndaifanira kumhanyisa bhuku rekutamba:

 cd k3s-ansible ansible-playbook playbooks/site.yml -i inventory.yml

Ndizvo. Nezve kuisirwa kunoenda, ini ndaive neKubernetes cluster kumusoro uye kumhanya. Ini ndaifanira kuisa K9s pamushini wangu wepanzvimbo kuti ndibate sumbu uye kusunga sumbu kune iyo ./kube/config faira.

The Applications

Chekupedzisira, ndaifanira kuisa maapplication andaida kumhanyisa musumbu. Ndainge ndine dzimwe pfungwa pane zvandaida.

• Ini ndaida kuve neMedia Server ine yakarongwa kurodha.

• Muenzaniso wePi-hole kune network yangu DNS uye vhara ese ads mumidziyo yese kumba.

• Muenzaniso weRetroarch wekutamba mimwe mitambo yechinyakare uye kugovera mitambo yekuchengetedza panetiweki yekumba kwangu (ndichikutarisa Megaman Battle Network 6 pamidziyo yangu yese)

Ndipo panopinda repository yangu .

Kune Media Server, ndakafunga kushandisa:

• Sonarr
• Radarr
• Prowlarr
• Jellyfin
• Qbittorrent

Semuenzaniso, ini ndinokuratidza kuti ndakaisa sei Sonarr ndichishandisa kubectl . Mamwe maapplication akaiswa nenzira yakafanana.

Kune yega yega Chishandiso, ini ndakagadzira 3 mafaera:

• deployment.yaml ndiko kumisikidzwa kweimwe neimwe yemapodhi ari kuita application

   apiVersion: apps/v1 kind: Deployment metadata: name: sonarr spec: replicas: 1 selector: matchLabels: app: sonarr template: metadata: labels: app: sonarr spec: containers: - name: sonarr image: linuxserver/sonarr ports: - containerPort: 8989 env: - name: PUID value: "911" - name: PGID value: "911" - name: TZ value: "Europe/Amsterdam" volumeMounts: - mountPath: /data name: data - name: config mountPath: /config volumes: - name: data persistentVolumeClaim: claimName: nfs-pvc - name: config persistentVolumeClaim: claimName: nfs-config-pvc

• service.yaml ndiko kumisikidzwa kwesevhisi inozoburitsa application kune cluster

   apiVersion: v1 kind: Service metadata: name: sonarr spec: selector: app: sonarr ports: - port: 80 targetPort: 8989 type: ClusterIP

• ingress.yaml uye uku ndiko kumisikidzwa kweiyo ingress inofumura application kunetiweki yangu

  
  

Ipapo tinoaisa ese tichishandisa kubectl :

 kubectl apply -f sonarr/deployment.yaml kubectl apply -f sonarr/service.yaml kubectl apply -f sonarr/ingress.yaml

Sezvauri kuona, ini ndiri kushandisa NFS-inotsigirwa inoenderera kuchengetedza iyo data uye kumisikidzwa kwezvishandiso.

Munzvimbo inochengeterwa, unogona kuwana nfs-pv.yaml uye nfs-pvc.yaml mafaera andakashandisa kugadzira NFS kuchengetedza.

Pamusoro pezvo, ini ndakagadzira imwe inoenderera vhoriyamu yekuda kumisikidzwa kwemaapplication.

Iyo yekupedzisira Kuvaka

Kunyangwe iyo kesi ichitaridzika inoshamisa, ikati hombe kune Raspberry Pi Cluster. A Mini ITX kesi ingadai yakakodzera zvandinodawo, asi ndinofanira kubvuma, ini ndiri anoyamwa zvinhu zveDIY.

Zvakare, Sucker ye LEDs kazhinji. Ini handina kuwedzera mamwe mwenje kunyaya, asi ndinofunga kuti bhodhi rinotoita basa rakanaka. Nehurombo, mapini efani aisaenderana nebhodhi, uye ini handina kutenga fan fan kana pini yebhodhi reamai. Ndingaite mune ramangwana.

Dzimwe nguva, iwe unongofanirwa kugara kumashure uye kunakidzwa nekuona.

Uye pakupedzisira, iyo Turing Pi 2 Kumba Cluster yasimuka uye ichimhanya, uye imba yangu haisisiri tsvina zvakare.

Ramangwana

Inguva chete ndiyo ichataura zvandichaita nechikwata ichi.

Zvisinei, ndanga ndichifunga kuwedzera Prometheus naGrafana kuti vave nemamwe ma metrics uye magirafu akanaka ekutarisa pasumbu.

Kutamisa mafaera angu ese eKubernetes kuHelm ingave zano rakanaka zvakare.

Chekupedzisira, iyo Retroarch muenzaniso ichiri mumabasa. Pamwe mumabasa mune tariro yakanyanya kupihwa kuti iyo pod inogara muCrashLoopBackOff state. Asi ndichasvikako.

Magumo

Kana masvika kumagumo kwechinyorwa chino, ndinokutendai nenguva yenyu. Ndinovimba wakanakidzwa nazvo sezvandakaita zvese kuisa cluster pamwe nekunyora nezvazvo.