[caasp-beta] caasp-beta

Ns, Rushi rushi.ns at sap.com
Sat Apr 1 21:47:52 MDT 2017


Hi Alejandro,

Good but more issues (kubernetes dashboard issue “unauthorized”) . Did you encounter any issues with kubernetes dashboard ? I get unauthorized when I browse https://lvcaasmaster:6443/ui
I was looking seems the certificates seems either missing ?

Aslo were you able to deploy any applications.  I tried a simple one and its not getting deployed ?? keep looking more, but let me know if you have done and how you did , so that I can check.



For that particular issue as you asked it was the file “kubelet” on every minion to add the FQDN or IP address of the minion host.

KUBELET_HOSTNAME="lvcaasnode1"

Best Regards,

Rushi.
I MAY BE ONLY ONE PERSON, BUT I CAN BE ONE PERSON WHO MAKES A DIFFERENCE


From: Alejandro Bonilla <abonilla at suse.com>
Date: Saturday, April 1, 2017 at 8:29 PM
To: "Ns, Rushi" <rushi.ns at sap.com>
Cc: "caasp-beta at lists.suse.com" <caasp-beta at lists.suse.com>
Subject: Re: [caasp-beta] caasp-beta

Hi Rushi,

Great news.

So the certificate problem perhaps was a time mismatch of the validity due to a NTP time that is now acceptable. Then I wonder what names were in the kubelet file vs the correct hostname? Is it 'linux' or where does the mismatch come from?

Thanks for the updates...


On Apr 1, 2017 11:15 PM, " Ns, Rushi " <rushi.ns at sap.com> wrote:

Ok “kubectl get nodes” empty issue too fixed.



Basically every minion host  in file  /etc/kubernetes/kubelet value didn't match the hostname , so I had to add the hostname of the minion FQDN and restar  the services.





Now I get all my minions.



PALM00635056A:.kube i811144$ kubectl get nodes

NAME          STATUS    AGE

lvcaasnode1   Ready     13m

lvcaasnode2   Ready     5m

lvcaasnode3   Ready     2m

lvcaasnode4   Ready     1m





Best Regards,



Rushi.

I MAY BE ONLY ONE PERSON, BUT I CAN BE ONE PERSON WHO MAKES A DIFFERENCE





From: "Ns, Rushi" <rushi.ns at sap.com>
Date: Saturday, April 1, 2017 at 6:13 PM
To: Alejandro Bonilla <abonilla at suse.com>
Cc: "caasp-beta at lists.suse.com" <caasp-beta at lists.suse.com>
Subject: Re: [caasp-beta] caasp-beta



Hi Alejandro,



Ok this one too fixed.



Now Kubectl works with cluster-info but I don’t see any nodes , however in my setup we had 4 minions , so I expected to have the output with 4 nodes but its empty?



PALM00635056A:.kube i811144$ kubectl  cluster-info

Kubernetes master is running at https://lvcaasmaster:6443



To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.





PALM00635056A:.kube i811144$ kubectl  get nodes

Nothing





Best Regards,



Rushi.

I MAY BE ONLY ONE PERSON, BUT I CAN BE ONE PERSON WHO MAKES A DIFFERENCE





From: "Ns, Rushi" <rushi.ns at sap.com>
Date: Saturday, April 1, 2017 at 6:05 PM
To: Alejandro Bonilla <abonilla at suse.com>
Cc: "caasp-beta at lists.suse.com" <caasp-beta at lists.suse.com>
Subject: Re: [caasp-beta] caasp-beta



Hi Alejandro,



Ok I have fixed it, I  think it was due NTP issue. Ok its created.



Now I get the certificates error after downloading the kubectl config.





$ kubectl get nodes

Unable to connect to the server: x509: certificate is valid for lvcaasmaster.pal.sap.corp, kubernetes.default.svc.cluster.local, not lvcaasmaster







[cid:image001.png at 01D2AB24.9C6542E0]



Best Regards,



Rushi.

I MAY BE ONLY ONE PERSON, BUT I CAN BE ONE PERSON WHO MAKES A DIFFERENCE





From: "Ns, Rushi" <rushi.ns at sap.com>
Date: Saturday, April 1, 2017 at 11:40 AM
To: Alejandro Bonilla <abonilla at suse.com>
Cc: "caasp-beta at lists.suse.com" <caasp-beta at lists.suse.com>
Subject: Re: [caasp-beta] caasp-beta



Hi Alejandro,



Thank you. For my setup I have proper DNS of hostname and I pretty much did as you did and I didnt hit any issues until the bootstrap cluster , the bootstrap cluster is running errors what I sent you in my first response ,,
Do you have time to go over on share screen I can show and see if we can fix this together



Best Regards,



Rushi.

Success is not a matter of being the best & winning the race. Success is a matter of handling the worst & finishing the race



Sent from my iPhone

please excuse typos and brevity

On Apr 1, 2017, at 10:59, Alejandro Bonilla <abonilla at suse.com<mailto:abonilla at suse.com>> wrote:

Hi Rushi,



I think the autoyast expects all nodes IP/DNS proper resolution. I did not use that in my first install because I don't have a setup with proper DNS...



I deployed successfully and quite quickly by



- Created 4 VMs



- Each booted by DVD1

- - One was named admin for the Dashboard installer. I changed the 'linux' hostname at the 'Network' section as one would with yast lan under Hostname while on the installer. Used the Dashboard role, if I recall correctly the name of that role.

- - Did the same with master, worker1, worker2 and set them all as the worker role. Specified the admin IP as the controller.



- before installing the master and worker nodes, I finished the admin install and waited for it to boot up, it takes 10 seconds after the login prompt for the web Interface to be available.



- Finished the master, workers install, they quickly showed up in the Dashboard, selected master as the, well... master.



Deployment finished in less than 5 minutes.



On my laptop:

Downloaded kubeconfig, then installed

kubernetes-client and kubernetes-common (I used them from software.opensuse.org<http://software.opensuse.org>, but there may be a more official installer.)



Edit kubeconfig to ensure the master address is correct, vs a non-resolvable DN entry.



mkdir ~/.kube

cp kubeconfig ~/.kube/config



Use kubectl :-)



On Mar 31, 2017 11:22 PM, "Ns, Rushi" <rushi.ns at sap.com<mailto:rushi.ns at sap.com>> wrote:

Here is my  beta test results.



I setup total  5 systems (1 controller, 1 master, 3 workers)



1)    Setup controller works fast and very quick too.

2)    Bootstrap nodes with autoyast worked and as well  manual install by selecting nodes by specifying the controller IP worked.

3)    Install cluster with selection ran several hours and I see the following



Any other tools to see whats happening other than kubelet commands I used here.



lvcaasadmin:/var/lib # kubelet list

I0401 03:58:17.465609    8571 feature_gate.go:189] feature gates: map[]

W0401 03:58:17.465761    8571 server.go:400] No API client: no api servers specified

I0401 03:58:17.465852    8571 docker.go:356] Connecting to docker on unix:///var/run/docker.sock

I0401 03:58:17.465893    8571 docker.go:376] Start docker client with request timeout=2m0s

E0401 03:58:17.467067    8571 cni.go:163] error updating cni config: No networks found in /etc/cni/net.d

I0401 03:58:17.475120    8571 manager.go:143] cAdvisor running in container: "/user.slice"

W0401 03:58:17.481167    8571 manager.go:151] unable to connect to Rkt api service: rkt: cannot tcp Dial rkt api service: dial tcp [::1]:15441: getsockopt: connection refused

I0401 03:58:17.487164    8571 fs.go:117] Filesystem partitions: map[/dev/sda2:{mountpoint:/var/lib/docker/btrfs major:0 minor:34 fsType:btrfs blockSize:0}]

I0401 03:58:17.489428    8571 manager.go:198] Machine: {NumCores:4 CpuFrequency:2297339 MemoryCapacity:33553862656 MachineID:026dbed750d34678baa318a927beb43c SystemUUID:4212E894-D404-BC7E-12C1-104406FAC1AF BootID:4fca4f86-b8df-40aa-9a52-9f32d82a65c6 Filesystems:[{Device:/dev/sda2 Capacity:105219358720 Type:vfs Inodes:0 HasInodes:true}] DiskMap:map[2:0:{Name:fd0 Major:2 Minor:0 Size:4096 Scheduler:cfq} 8:0:{Name:sda Major:8 Minor:0 Size:107374182400 Scheduler:cfq}] NetworkDevices:[{Name:eth0 MacAddress:00:50:56:92:78:da Speed:10000 Mtu:1500}] Topology:[{Id:0 Memory:33553862656 Cores:[{Id:0 Threads:[0] Caches:[]} {Id:1 Threads:[1] Caches:[]}] Caches:[{Size:41943040 Type:Unified Level:3}]} {Id:1 Memory:0 Cores:[{Id:0 Threads:[2] Caches:[]} {Id:1 Threads:[3] Caches:[]}] Caches:[{Size:41943040 Type:Unified Level:3}]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}

I0401 03:58:17.490209    8571 manager.go:204] Version: {KernelVersion:4.4.52-1-default ContainerOsVersion:SUSE Container as a Service Platform 1.0 DockerVersion:1.12.6 CadvisorVersion: CadvisorRevision:}

I0401 03:58:17.491582    8571 cadvisor_linux.go:152] Failed to register cAdvisor on port 4194, retrying. Error: listen tcp :4194: bind: address already in use

W0401 03:58:17.494066    8571 container_manager_linux.go:205] Running with swap on is not supported, please disable swap! This will be a fatal error by default starting in K8s v1.6! In the meantime, you can opt-in to making this a fatal error by enabling --experimental-fail-swap-on.

W0401 03:58:17.494326    8571 server.go:669] No api server defined - no events will be sent to API server.

W0401 03:58:17.497638    8571 kubelet_network.go:69] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"

I0401 03:58:17.497694    8571 kubelet.go:477] Hairpin mode set to "hairpin-veth"

I0401 03:58:17.504274    8571 docker_manager.go:256] Setting dockerRoot to /var/lib/docker

I0401 03:58:17.504311    8571 docker_manager.go:259] Setting cgroupDriver to cgroupfs

I0401 03:58:17.506297    8571 server.go:770] Started kubelet v1.5.3

W0401 03:58:17.506331    8571 kubelet.go:1224] No api server defined - no node status update will be sent.

E0401 03:58:17.506400    8571 server.go:481] Starting health server failed: listen tcp 127.0.0.1:10248: bind: address already in use

E0401 03:58:17.506382    8571 kubelet.go:1145] Image garbage collection failed: unable to find data for container /

I0401 03:58:17.506527    8571 server.go:123] Starting to listen on 0.0.0.0:10250

I0401 03:58:17.506647    8571 kubelet_node_status.go:204] Setting node annotation to enable volume controller attach/detach

F0401 03:58:17.509424    8571 server.go:148] listen tcp 0.0.0.0:10255: bind: address already in use



[cid:image002.png at 01D2AB24.9C6542E0]

[cid:image003.png at 01D2AB24.9C6542E0]







from messages I see the following.





event.go:208] Unable to write event: 'Post http://127.0.0.1:8080/api/v1/namespaces/default/events: dial tcp 127.0.0.1:8080: getsockopt: connection refused' (may retry after sleeping)

2017-04-01T04:01:57.375925+00:00 lvcaasadmin hyperkube[2651]: E0401 04:01:57.375830    2651 reflector.go:188] pkg/kubelet/kubelet.go:378: Failed to list *api.Service: Get http://127.0.0.1:8080/api/v1/services?resourceVersion=0: dial tcp 127.0.0.1:8080: getsockopt: connection refused

2017-04-01T04:01:57.443964+00:00 lvcaasadmin hyperkube[2651]: E0401 04:01:57.443871    2651 reflector.go:188] pkg/kubelet/config/apiserver.go:44: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%3D127.0.0.1&resourceVersion=0: dial tcp 127.0.0.1:8080: getsockopt: connection refused

2017-04-01T04:01:57.444895+00:00 lvcaasadmin hyperkube[2651]: E0401 04:01:57.444801    2651 reflector.go:188] pkg/kubelet/kubelet.go:386: Failed to list *api.Node: Get http://127.0.0.1:8080/api/v1/nodes?fieldSelector=metadata.name%3D127.0.0.1&resourceVersion=0: dial tcp 127.0.0.1:8080: getsockopt: connection refused

2017-04-01T04:01:58.376648+00:00 lvcaasadmin hyperkube[2651]: E0401 04:01:58.376550    2651 reflector.go:188] pkg/kubelet/kubelet.go:378: Failed to list *api.Service: Get http://127.0.0.1:8080/api/v1/services?resourceVersion=0: dial tcp 127.0.0.1:8080: getsockopt: connection refused

2017-04-01T04:01:58.444697+00:00 lvcaasadmin hyperkube[2651]: E0401 04:01:58.444601    2651 reflector.go:188] pkg/kubelet/config/apiserver.go:44: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%3D127.0.0.1&resourceVersion=0: dial tcp 127.0.0.1:8080: getsockopt: connection refused

2017-04-01T04:01:58.446107+00:00 lvcaasadmin hyperkube[2651]: E0401 04:01:58.446008    2651 reflector.go:188] pkg/kubelet/kubelet.go:386: Failed to list *api.Node: Get http://127.0.0.1:8080/api/v1/nodes?fieldSelector=metadata.name%3D127.0.0.1&resourceVersion=0: dial tcp 127.0.0.1:8080: getsockopt: connection refused

2017-04-01T04:01:59.377433+00:00 lvcaasadmin hyperkube[2651]: E0401 04:01:59.377325    2651 reflector.go:188] pkg/kubelet/kubelet.go:378: Failed to list *api.Service: Get http://127.0.0.1:8080/api/v1/services?resourceVersion=0: dial tcp 127.0.0.1:8080: getsockopt: connection refused

2017-04-01T04:01:59.445470+00:00 lvcaasadmin hyperkube[2651]: E0401 04:01:59.445336    2651 reflector.go:188] pkg/kubelet/config/apiserver.go:44: F



Best Regards,



Rushi.

I MAY BE ONLY ONE PERSON, BUT I CAN BE ONE PERSON WHO MAKES A DIFFERENCE





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.suse.com/pipermail/caasp-beta/attachments/20170402/0f258555/attachment.htm>


More information about the caasp-beta mailing list