[caasp-beta] Q: CAASP4B4 new cluster / join problem
egov-devops
egov-devops at akdb.de
Thu Jul 18 03:20:01 MDT 2019
Full output:
-----------8<----------
caasp-admin:~/caasp4b4 # skuba node join -v 5 --role master --user sles --sudo --target 10.255.1.4 caasp-master2
** This is a BETA release and NOT intended for production usage. **
I0718 08:05:16.208668 20242 loader.go:359] Config loaded from file: admin.conf
I0718 08:05:16.209494 20242 round_trippers.go:416] GET https://10.255.1.2:6443/api/v1/nodes/caasp-master2
I0718 08:05:16.209504 20242 round_trippers.go:423] Request Headers:
I0718 08:05:16.209510 20242 round_trippers.go:426] Accept: application/json, */*
I0718 08:05:16.209517 20242 round_trippers.go:426] User-Agent: skuba/v0.0.0 (linux/amd64) kubernetes/fd919f4I0718 08:05:16.221502 20242 round_trippers.go:441] Response Status: 404 Not Found in 11 milliseconds
[join] applying states to new node
I0718 08:05:16.221927 20242 states.go:35] === applying state kubernetes.join.upload-secrets ===
I0718 08:05:16.221937 20242 deployments.go:50] uploading local file "pki/ca.crt" to remote file "/etc/kubernetes/pki/ca.crt"
I0718 08:05:16.221988 20242 files.go:29] uploading to remote file "/etc/kubernetes/pki/ca.crt" with contents
[join] failed to apply join to node failed to apply state kubernetes.join.upload-secrets: Process exited with status 1
F0718 08:05:16.503251 20242 join.go:50] error joining node caasp-master2: failed to apply state kubernetes.join.upload-secrets: Process exited with status 1
-----------8<----------
LB Config:
-----------8<----------
user nginx;
worker_processes auto;
# load_module lib64/nginx/modules/ngx_http_fancyindex_module.so;
# load_module lib64/nginx/modules/ngx_http_geoip_module.so;
# load_module lib64/nginx/modules/ngx_http_headers_more_filter_module.so;
# load_module lib64/nginx/modules/ngx_http_image_filter_module.so;
# load_module lib64/nginx/modules/ngx_http_perl_module.so;
# load_module lib64/nginx/modules/ngx_http_xslt_filter_module.so;
# load_module lib64/nginx/modules/ngx_mail_module.so;
# load_module lib64/nginx/modules/ngx_rtmp_module.so;
# load_module lib64/nginx/modules/ngx_stream_geoip_module.so;
load_module lib64/nginx/modules/ngx_stream_module.so;
#error_log /var/log/nginx/error.log;
#error_log /var/log/nginx/error.log notice;
#error_log /var/log/nginx/error.log info;
#pid /run/nginx.pid;
events {
worker_connections 1024;
use epoll;
}
stream {
log_format proxy '$remote_addr [$time_local] '
'$protocol $status $bytes_sent $bytes_received '
'$session_time "$upstream_addr"';
error_log /var/log/nginx/k8s-masters-lb-error.log;
access_log /var/log/nginx/k8s-masters-lb-access.log proxy;
upstream k8s-masters {
#hash $remote_addr consistent;
server caasp-master1:6443 weight=1 max_fails=1;
server caasp-master2:6443 weight=1 max_fails=1;
server caasp-master3:6443 weight=1 max_fails=1;
}
server {
listen 6443;
proxy_connect_timeout 1s;
proxy_timeout 3s;
proxy_pass k8s-masters;
}
}
-----------8<----------
caasp-admin:~/caasp4b4 # systemctl status nginx.service
● nginx.service - The nginx HTTP and reverse proxy server
Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2019-07-02 17:29:59 CEST; 2 weeks 1 days ago
Main PID: 1248 (nginx)
Tasks: 5 (limit: 4915)
Memory: 22.5M
CPU: 2min 31.546s
CGroup: /system.slice/nginx.service
├─1248 nginx: master process /usr/sbin/nginx -g daemon off;
├─1249 nginx: worker process
├─1250 nginx: worker process
├─1251 nginx: worker process
└─1252 nginx: worker process
Jul 02 17:29:59 caasp-admin systemd[1]: Starting The nginx HTTP and reverse proxy server...
Jul 02 17:29:59 caasp-admin nginx[1239]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
Jul 02 17:29:59 caasp-admin nginx[1239]: nginx: configuration file /etc/nginx/nginx.conf test is successful
Jul 02 17:29:59 caasp-admin systemd[1]: Started The nginx HTTP and reverse proxy server.
-----------8<----------
/etc/hosts
...
10.255.1.2 admin
10.255.1.3 master1 caasp-master1
10.255.1.4 master2 caasp-master2
10.255.1.5 master3 caasp-master3
10.255.2.14 node1 caasp-node1
10.255.2.15 node2 caasp-node2
10.255.2.16 node3 caasp-node3
-----------8<----------
/AS
-----Ursprüngliche Nachricht-----
Von: caasp-beta <caasp-beta-bounces at lists.suse.com> Im Auftrag von Ludovic Cavajani
Gesendet: Donnerstag, 18. Juli 2019 11:09
An: caasp-beta at lists.suse.com
Betreff: Re: [caasp-beta] Q: CAASP4B4 new cluster / join problem
Hi,
Updating from `beta3` to `beta4` is possible so there is no need to reset everything. The documentation can be found here:
https://susedoc.github.io/doc-caasp/beta/caasp-admin/single-html/#_cluster_updates
@egov-devops I see you used `-v5` flag, could you share the entire logs if there are no sensitive information from your side ? Is `10.255.1.2` correctly configured as an LB with the 3 masters nodes ? Could you share the configuration if possible ?
Thanks,
On 7/18/19 10:43 AM, Jean Marc Lambert wrote:
> One element, is also, (I think it is not yet documented, as I
> discovered it yesterday) that you need to update the skuba itself on your 'CLI' node. New features in it. Like upgrade plans etc There is a skuba node reset that you may use to reinit the node.
> JMarc
>
> De : caasp-beta <caasp-beta-bounces at lists.suse.com> De la part de
> egov-devops Envoyé : jeudi 18 juillet 2019 08:40 À :
> caasp-beta at lists.suse.com Cc : Gergo Kovacs <gergo.kovacs at suse.com>
> Objet : [caasp-beta] Q: CAASP4B4 new cluster / join problem
>
> Howdy,
>
> yesterday, finally I got our Airgap install of CAASP4B3 working (only 1 master so far) - Needed to solve internal (AKDB) and external (SUSE) problems to make it work.
>
> Today I learned that CASSP4B4 is out, so I tried to set up CAASP4B4 fresh.
>
> What have I done so far:
> 1) made a "kubeadm reset" on master1. To reset the master I set up
> with CAASP4B3, yesterday
> 2) updated our "fake" registry.suse.com with new images (see
> attachment)
> 3) made a "skuba cluster init --control-plane 10.255.1.2 caasp4b4"
> 4) made a "skuba node bootstrap -v 5 --user sles --sudo --target 10.255.1.3 caasp-master1"
> --> [bootstrap] successfully bootstrapped node "10.255.1.3" with Kubernetes: "1.15.0"
> 5) made a "skuba node join -v 5 --role master --user sles --sudo --target 10.255.1.4 caasp-master2"
> --> I0718 08:05:16.221988 20242 files.go:29] uploading to remote file "/etc/kubernetes/pki/ca.crt" with contents [join] failed to apply join to node failed to apply state kubernetes.join.upload-secrets: Process exited with status 1
> --> F0718 08:05:16.503251 20242 join.go:50] error joining node caasp-master2: failed to apply state kubernetes.join. upload-secrets: Process exited with status 1
>
> Some Background:
>
> * ssh from "caasp-admin" via pubkey as sles or root works fine.
>
> - Each planned master and node is installed/updated with "zypper in kubernetes-kubeadm kubernetes-kubelet kubernetes-client cri-o cni-plugins sudo"
>
> * SUSECOnnect activated:
>
> caasp-master1:~ # SUSEConnect --list-extensions
>
> AVAILABLE EXTENSIONS AND MODULES
>
>
>
> Basesystem Module 15 SP1 x86_64 (Activated)
>
> Deactivate with: SUSEConnect -d -p
> sle-module-basesystem/15.1/x86_64
>
>
>
> Containers Module 15 SP1 x86_64 (Activated)
>
> Deactivate with: SUSEConnect -d -p /15.1/x86_64
>
>
>
> SUSE CaaS Platform 4.0 x86_64 (BETA) (Activated)
>
> Deactivate with: SUSEConnect -d -p caasp/4.0/x86_64
>
>
>
> Server Applications Module 15 SP1 x86_64 (Activated)
>
> Deactivate with: SUSEConnect -d -p
> sle-module-server-applications/15.1/x86_64
>
>
>
> Public Cloud Module 15 SP1 x86_64 (Activated)
>
> Deactivate with: SUSEConnect -d -p
> sle-module-public-cloud/15.1/x86_64
>
>
> Any idea what I made wrong? Where can I check? What can I fix? :-/
>
> Many Thanks...
>
> Mit freundlichen Grüßen
>
> Alexander Schießl
> Team DevOps
> Geschäftsfeld eGovernment
>
> Telefon +49 800 2553222-65
> egov-devops at adkb.de<mailto:egov-devops at adkb.de>
> __________________________________________________________
>
> AKDB * Anstalt des öffentlichen Rechts * Hauptverwaltung München
> Hansastr. 12 - 16 * 80686 München * www.akdb.de<http://www.akdb.de/>
>
>
>
> _______________________________________________
> caasp-beta mailing list
> caasp-beta at lists.suse.com
> Check the mailing list archives or Unsubscribe at
> http://lists.suse.com/mailman/listinfo/caasp-beta
More information about the caasp-beta
mailing list