[caasp-beta] BTRFS space and quota
Paul Gonin
paul.gonin at suse.com
Thu Nov 2 04:55:05 MDT 2017
Hi Stéphane,
Thanks, we're looking into reproducing this issue.
rgdsPaul
Le jeudi 02 novembre 2017 à 10:35 +0000, Le Bihan Stéphane (AMUNDI-ITS)
a écrit :
> Hi Paul,
>
> The result of command snapper ls.
>
> # snapper ls
> Type | # | Pre # | Date | User | Cleanup
> | Description | Userdata
> -------+---+-------+---------------------------------+------+------
> ---+-----------------------+---------
> single | 0 | | | root |
> | current |
> single | 1 | | Fri 06 Oct 2017 08:47:14 AM UTC | root |
> | first root filesystem |
>
> I delete quota on /var/lb/etcd, and test balance but it’s not ok.
> I recreate quota and rescan and value is same before deletion.
>
> For information I launch du –sh on / and result is 7.8Go.
>
> # du -sh /
> du: cannot access '/proc/7982/task/7982/fd/4': No such file or
> directory
> du: cannot access '/proc/7982/task/7982/fdinfo/4': No such file or
> directory
> du: cannot access '/proc/7982/fd/3': No such file or directory
> du: cannot access '/proc/7982/fdinfo/3': No such file or directory
> 7.8G /
>
> Regards,
>
>
>
>
>
>
>
>
>
>
>
>
> Stéphane Le Bihan
>
>
>
>
> SDE/DSI/IPR/SSD/UNX
>
>
>
>
> 90, Boulevard Pasteur - 75015 Paris
>
>
>
>
> Web:
> http://www.amundi.com
>
>
>
>
> Tél: +33 1 76 32 32 08
> Equipe Unix : +33 1 76 32 02 30
>
>
>
>
> @:
> stephane.lebihan at amundi.com
> @ :
> sits.unix at amundi.com
>
>
>
>
>
>
>
>
>
> De : Paul Gonin [mailto:paul.gonin at suse.com]
>
>
> Envoyé : jeudi 2 novembre 2017 10:55
>
> À : Le Bihan Stéphane (AMUNDI-ITS); caasp-beta at lists.suse.com
>
> Objet : Re: [caasp-beta] BTRFS space and quota
>
>
>
>
> Hi Stephane,
>
>
>
>
>
> What is the output of
>
>
>
> # snapper ls
>
>
> ?
>
>
>
>
>
> I assume that since you there were no updates yet it should look like
>
>
>
>
>
> Type | # | Pre # | Date | User | Cleanup |
> Description | Userdata
>
>
> -------+---+-------+--------------------------+------+---------+-----
> ------------------+--------------
>
>
> single | 0 | | | root | |
> current |
>
>
> single | 1 | | Tue Oct 31 09:07:13 2017 | root | |
> first root filesystem |
>
>
> single | 2 | | Tue Oct 31 09:10:42 2017 | root | number |
> after installation | important=yes
>
>
>
>
>
> rgds
>
>
> Paul
>
>
>
>
>
> Le mardi 31 octobre 2017 à 13:38 +0000, Le Bihan Stéphane (AMUNDI-
> ITS) a écrit :
>
> > Hi Paul,
> >
> > We work with CaaSP2.
> >
> > Regards,
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Stéphane Le Bihan
> >
> >
> >
> >
> > SDE/DSI/IPR/SSD/UNX
> >
> >
> >
> >
> > 90, Boulevard Pasteur - 75015 Paris
> >
> >
> >
> >
> > Web:
> > http://www.amundi.com
> >
> >
> >
> >
> > Tél: +33 1 76 32 32 08
> > Equipe Unix : +33 1 76 32 02 30
> >
> >
> >
> >
> > @:
> > stephane.lebihan at amundi.com
> > @ :
> > sits.unix at amundi.com
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > De : Paul Gonin [mailto:paul.gonin at suse.com]
> >
> >
> > Envoyé : mardi 31 octobre 2017 14:34
> >
> > À : Le Bihan Stéphane (AMUNDI-ITS);
> > caasp-beta at lists.suse.com
> >
> > Objet : Re: [caasp-beta] BTRFS space and quota
> >
> >
> >
> >
> > Hi Stéphane,
> >
> >
> >
> >
> >
> > Not that it should make a difference for the issue described, what
> > version of CaaSP the cluster is running ?
> >
> >
> > Is it CaaSP2 ? RC1 ?
> >
> >
> >
> >
> >
> > thanks
> >
> >
> > Paul
> >
> >
> >
> >
> >
> > Le mardi 31 octobre 2017 à 08:35 +0000, Le Bihan Stéphane (AMUNDI-
> > ITS) a écrit :
> >
> > > Hello,
> > >
> > > We have a strange case on CAASP plateform with btrfs quota.
> > >
> > > For history, I was out of office since 3 weeks, but others
> > > colleague test kubernetes plateform.
> > > When I return, we ask me because FS is full on master and worker
> > > nodes.
> > > I don’t have cause, but I think with a bad config, subvolume
> > > /var/lib/etcd grown and after correction reduce, though quota
> > > reserved all space.
> > >
> > > When I check, I see btrfs usage and it’s really full, but balance
> > > as no effect.
> > > After search I see quota is activate, and subvolumes
> > > /var/lib/etcd reserved 90% of space. But I don’t succeed to
> > > release this space.
> > >
> > > Can you help me for release space disk ?
> > >
> > > ·
> > > On master :
> > >
> > > # btrfs filesystem usage /
> > > Overall:
> > > Device size: 30.00GiB
> > > Device allocated: 29.99GiB
> > > Device unallocated: 17.00MiB
> > > Device missing: 0.00B
> > > Used: 27.56GiB
> > > Free (estimated): 504.93MiB (min: 496.43MiB)
> > > Data ratio: 1.00
> > > Metadata ratio: 2.00
> > > Global reserve: 16.00MiB (used: 0.00B)
> > >
> > > Data,single: Size:27.97GiB, Used:27.49GiB
> > > /dev/vda6 27.97GiB
> > >
> > > Metadata,DUP: Size:1.00GiB, Used:32.64MiB
> > > /dev/vda6 2.00GiB
> > >
> > > System,DUP: Size:9.50MiB, Used:16.00KiB
> > > /dev/vda6 19.00MiB
> > >
> > > Unallocated:
> > > /dev/vda6 17.00MiB
> > >
> > > # btrfs fi df /
> > > Data, single: total=27.97GiB, used=27.50GiB
> > > System, DUP: total=9.50MiB, used=16.00KiB
> > > Metadata, DUP: total=1.00GiB, used=32.66MiB
> > > GlobalReserve, single: total=16.00MiB, used=0.00B
> > >
> > > # btrfs fi show /
> > > Label: none uuid: 1b0614eb-fc59-4841-bbc5-5318087f6432
> > > Total devices 1 FS bytes used 27.53GiB
> > > devid 1 size 30.00GiB used 29.99GiB path /dev/vda6
> > >
> > > # btrfs subvolume list /
> > > ID 257 gen 40 top level 5 path @
> > > ID 258 gen 194820 top level 257 path @/.snapshots
> > > ID 259 gen 197128 top level 258 path @/.snapshots/1/snapshot
> > > ID 260 gen 194810 top level 257 path @/boot/grub2/i386-pc
> > > ID 261 gen 194810 top level 257 path @/boot/grub2/x86_64-efi
> > > ID 262 gen 194810 top level 257 path @/cloud-init-config
> > > ID 263 gen 194810 top level 257 path @/home
> > > ID 264 gen 197081 top level 257 path @/root
> > > ID 265 gen 197111 top level 257 path @/tmp
> > > ID 266 gen 194809 top level 257 path @/var/cache
> > > ID 267 gen 194809 top level 257 path @/var/crash
> > > ID 268 gen 195783 top level 257 path @/var/lib/ca-certificates
> > > ID 269 gen 195783 top level 257 path @/var/lib/cloud
> > > ID 270 gen 24 top level 257 path @/var/lib/docker
> > > ID 271 gen 194810 top level 257 path @/var/lib/dockershim
> > > ID 272 gen 195719 top level 257 path @/var/lib/etcd
> > > ID 273 gen 194810 top level 257 path @/var/lib/kubelet
> > > ID 274 gen 194810 top level 257 path @/var/lib/machines
> > > ID 275 gen 196430 top level 257 path @/var/lib/misc
> > > ID 276 gen 194810 top level 257 path @/var/lib/mysql
> > > ID 277 gen 194810 top level 257 path @/var/lib/nfs
> > > ID 278 gen 194810 top level 257 path @/var/lib/ntp
> > > ID 279 gen 196428 top level 257 path @/var/lib/overlay
> > > ID 280 gen 194810 top level 257 path @/var/lib/rollback
> > > ID 281 gen 196427 top level 257 path @/var/lib/systemd
> > > ID 282 gen 194810 top level 257 path @/var/lib/vmware
> > > ID 283 gen 194810 top level 257 path @/var/lib/wicked
> > > ID 284 gen 197128 top level 257 path @/var/log
> > > ID 285 gen 197111 top level 257 path @/var/spool
> > > ID 286 gen 196428 top level 257 path @/var/tmp
> > >
> > > # btrfs qgroup show -pcreFf /var/lib/etcd
> > > qgroupid rfer excl max_rfer max_excl
> > > parent child
> > > -------- ---- ---- -------- -------- ----
> > > -- -----
> > > 0/272 25.14GiB 25.14GiB none none -
> > > -- ---
> > >
> > > # du -sh /var/lib/etcd/
> > > 417M /var/lib/etcd/
> > >
> > > ·
> > > On one worker
> > >
> > > # btrfs fi usage /
> > > Overall:
> > > Device size: 30.00GiB
> > > Device allocated: 30.00GiB
> > > Device unallocated: 1.00MiB
> > > Device missing: 0.00B
> > > Used: 27.94GiB
> > > Free (estimated): 135.28MiB (min: 135.28MiB)
> > > Data ratio: 1.00
> > > Metadata ratio: 2.00
> > > Global reserve: 16.00MiB (used: 0.00B)
> > >
> > > Data,single: Size:27.99GiB, Used:27.86GiB
> > > /dev/vda6 27.99GiB
> > >
> > > Metadata,DUP: Size:1.00GiB, Used:43.44MiB
> > > /dev/vda6 2.00GiB
> > >
> > > System,DUP: Size:8.00MiB, Used:16.00KiB
> > > /dev/vda6 16.00MiB
> > >
> > > Unallocated:
> > > /dev/vda6 1.00MiB
> > >
> > > # btrfs fi df /
> > > Data, single: total=27.99GiB, used=27.86GiB
> > > System, DUP: total=8.00MiB, used=16.00KiB
> > > Metadata, DUP: total=1.00GiB, used=43.44MiB
> > > GlobalReserve, single: total=16.00MiB, used=0.00B
> > >
> > > # btrfs fi show /
> > > Label: none uuid: 1d7b76f8-f91c-47e0-8be2-a3f02f90ac96
> > > Total devices 1 FS bytes used 27.90GiB
> > > devid 1 size 30.00GiB used 30.00GiB path /dev/vda6
> > >
> > > # btrfs qgroup show -pcreFf /var/lib/etcd
> > > qgroupid rfer excl max_rfer max_excl
> > > parent child
> > > -------- ---- ---- -------- -------- ----
> > > -- -----
> > > 0/272 20.99GiB 20.99GiB none none -
> > > -- ---
> > >
> > > # du -sh /var/lib/etcd/
> > > 452M /var/lib/etcd/
> > >
> > > Regards,
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > Stéphane Le Bihan
> > >
> > >
> > >
> > >
> > > SDE/DSI/IPR/SSD/UNX
> > >
> > >
> > >
> > >
> > > 90, Boulevard Pasteur - 75015 Paris
> > >
> > >
> > >
> > >
> > > Web:
> > > http://www.amundi.com
> > >
> > >
> > >
> > >
> > > Tél: +33 1 76 32 32 08
> > > Equipe Unix : +33 1 76 32 02 30
> > >
> > >
> > >
> > >
> > > @:
> > > stephane.lebihan at amundi.com
> > > @ :
> > > sits.unix at amundi.com
> > >
> > >
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > caasp-beta mailing list
> > > caasp-beta at lists.suse.com
> > > http://lists.suse.com/mailman/listinfo/caasp-beta
> >
> > _______________________________________________
> > caasp-beta mailing list
> > caasp-beta at lists.suse.com
> > http://lists.suse.com/mailman/listinfo/caasp-beta
>
>
>
>
> _______________________________________________
> caasp-beta mailing list
> caasp-beta at lists.suse.com
> http://lists.suse.com/mailman/listinfo/caasp-beta
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.suse.com/pipermail/caasp-beta/attachments/20171102/5c1cd64b/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.gif
Type: image/gif
Size: 2430 bytes
Desc: not available
URL: <http://lists.suse.com/pipermail/caasp-beta/attachments/20171102/5c1cd64b/attachment.gif>
More information about the caasp-beta
mailing list