hi all, I hope somebody can offer some advice on a xfs disk quota problem, where the soft block quota grace period is ignored. I confirmed that the problem affects at least 4 file servers that I mgirated to Rocky.
On all servers, xfs disk quotas have worked correctly for over a year with CentOS8.4. Last month, I upgraded the 4 fileservers from CentOS8.4 to CentOS8.5, rebooted, then ran the migrate2rocky.sh script. The migration script reported no errors. This week, I noticed that quotas grace period was ignored. I changed nothing on the servers except upgrading packages/migrating to Rocky. Now when a user's soft block quota is exceeded, write access is denied (even though the user is within the grace period, and below the hard quota).
Below is information from one server, but all 4 are similar: ------------------------------------------------------------- # cat /etc/redhat-release Rocky Linux release 8.5 (Green Obsidian)
# uname -a Linux fs8.priv 4.18.0-348.7.1.el8_5.x86_64 #1 SMP Tue Dec 21 19:02:23 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
# rpm -qa | grep -i quota quota-4.04-14.el8.x86_64 quotatool-1.6.2-13.el8.x86_64 quota-nls-4.04-14.el8.noarch quota-rpc-4.04-14.el8.x86_64
mount filesystem with user quotas and confirm: ---------------------------------------------- # mount -o uquota /dev/vg716f9/lv716f9 /export/716f9 # mount | grep 716f9 /dev/mapper/vg716f9-lv716f9 on /export/716f9 type xfs (rw,relatime,inode64,logbufs=8,logbsize=32k,sunit=256,swidth=3328,usrquota)
# xfs_quota -x -c "state -u" /export/716f9 User quota state on /export/716f9 (/dev/mapper/vg716f9-lv716f9) Accounting: ON Enforcement: ON Inode: #515 (42 blocks, 41 extents) Blocks grace time: [14 days] Blocks max warnings: 5 Inodes grace time: [7 days] Inodes max warnings: 5 Realtime Blocks grace time: [7 days]
set user 'julie' block soft quota to 5GB and block hard quota to 100GB: ----------------------------------------------------------------------- # xfs_quota -x -c 'limit bsoft=5G bhard=100G julie' /export/716f9
# xfs_quota -x -c 'report -h' /export/716f9 User quota on /export/716f9 (/dev/mapper/vg716f9-lv716f9) Blocks User ID Used Soft Hard Warn/Grace ---------- --------------------------------- julie 4.1G 5G 100G 00 [------]
log in as user 'julie' to create a file ------------------------------------------ [julie@fs8 ~]$ touch /export/716f9/julie/a [julie@fs8 ~]$ quota -s Disk quotas for user julie (uid 1353): Filesystem space quota limit grace files quota limit grace /dev/mapper/vg716f9-lv716f9 4240M 5120M 100G 12503 0 0
set user 'julie' block soft quota to 1GB and 100GB hard, and note 14-day grace period: --------------------------------------------------------------------------------------- # xfs_quota -x -c 'limit bsoft=1G bhard=100G julie' /export/716f9
# xfs_quota -x -c 'report -h' /export/716f9 | grep julie User quota on /export/716f9 (/dev/mapper/vg716f9-lv716f9) Blocks User ID Used Soft Hard Warn/Grace ---------- --------------------------------- julie 4.1G 1G 100G 00 [14 days]
log in as user 'julie' again confirm quota and try to write file ---------------------------------------------------------------- [julie@fs8 ~]$ quota -s Disk quotas for user julie (uid 1353): Filesystem space quota limit grace files quota limit grace /dev/mapper/vg716f9-lv716f9 4240M* 1024M 100G 13days 12504 0 0
[julie@fs8 ~]$ touch /export/716f9/julie/a touch: cannot touch '/export/716f9/julie/a': Disk quota exceeded
Can anybody offer some advice?
I tried the following without any luck:
removed the hard block disk quota upgraded all packages and rebooted downgraded quota-* rpms (from version 4.04-14 to 4.04.12 used rhel 8.3) and restarting rpc-rquotad (systemctl restart rpc-rquotad)
before downgrade # rpm -qa | grep -i quota quota-4.04-14.el8.x86_64 quotatool-1.6.2-13.el8.x86_64 quota-nls-4.04-14.el8.noarch quota-rpc-4.04-14.el8.x86_64
after downgrade # rpm -qa | grep -i quota quota-rpc-4.04-10.el8.x86_64 quota-nls-4.04-10.el8.noarch quota-4.04-10.el8.x86_64
Thanks! Julie
---end quoted text---
I am certainly no expert on XFS disk quotas, but have you tried downgrading the kernel to see if it's a regression in the kernel module?
Do keep in mind that Rocky Linux 8.4 and 8.3 kernels are not signed for secureboot if you test this theory.
Peter
On 11/01/22 10:41 am, Julie Ashworth via rocky wrote:
hi all, I hope somebody can offer some advice on a xfs disk quota problem, where the soft block quota grace period is ignored. I confirmed that the problem affects at least 4 file servers that I mgirated to Rocky.
On all servers, xfs disk quotas have worked correctly for over a year with CentOS8.4. Last month, I upgraded the 4 fileservers from CentOS8.4 to CentOS8.5, rebooted, then ran the migrate2rocky.sh script. The migration script reported no errors. This week, I noticed that quotas grace period was ignored. I changed nothing on the servers except upgrading packages/migrating to Rocky. Now when a user's soft block quota is exceeded, write access is denied (even though the user is within the grace period, and below the hard quota).
Below is information from one server, but all 4 are similar:
# cat /etc/redhat-release Rocky Linux release 8.5 (Green Obsidian)
# uname -a Linux fs8.priv 4.18.0-348.7.1.el8_5.x86_64 #1 SMP Tue Dec 21 19:02:23 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
# rpm -qa | grep -i quota quota-4.04-14.el8.x86_64 quotatool-1.6.2-13.el8.x86_64 quota-nls-4.04-14.el8.noarch quota-rpc-4.04-14.el8.x86_64
mount filesystem with user quotas and confirm:
# mount -o uquota /dev/vg716f9/lv716f9 /export/716f9 # mount | grep 716f9 /dev/mapper/vg716f9-lv716f9 on /export/716f9 type xfs (rw,relatime,inode64,logbufs=8,logbsize=32k,sunit=256,swidth=3328,usrquota)
# xfs_quota -x -c "state -u" /export/716f9 User quota state on /export/716f9 (/dev/mapper/vg716f9-lv716f9) Accounting: ON Enforcement: ON Inode: #515 (42 blocks, 41 extents) Blocks grace time: [14 days] Blocks max warnings: 5 Inodes grace time: [7 days] Inodes max warnings: 5 Realtime Blocks grace time: [7 days]
set user 'julie' block soft quota to 5GB and block hard quota to 100GB:
# xfs_quota -x -c 'limit bsoft=5G bhard=100G julie' /export/716f9
# xfs_quota -x -c 'report -h' /export/716f9 User quota on /export/716f9 (/dev/mapper/vg716f9-lv716f9) Blocks User ID Used Soft Hard Warn/Grace
julie 4.1G 5G 100G 00 [------]
log in as user 'julie' to create a file
[julie@fs8 ~]$ touch /export/716f9/julie/a [julie@fs8 ~]$ quota -s Disk quotas for user julie (uid 1353): Filesystem space quota limit grace files quota limit grace /dev/mapper/vg716f9-lv716f9 4240M 5120M 100G 12503 0 0
set user 'julie' block soft quota to 1GB and 100GB hard, and note 14-day grace period:
# xfs_quota -x -c 'limit bsoft=1G bhard=100G julie' /export/716f9
# xfs_quota -x -c 'report -h' /export/716f9 | grep julie User quota on /export/716f9 (/dev/mapper/vg716f9-lv716f9) Blocks User ID Used Soft Hard Warn/Grace
julie 4.1G 1G 100G 00 [14 days]
log in as user 'julie' again confirm quota and try to write file
[julie@fs8 ~]$ quota -s Disk quotas for user julie (uid 1353): Filesystem space quota limit grace files quota limit grace /dev/mapper/vg716f9-lv716f9 4240M* 1024M 100G 13days 12504 0 0
[julie@fs8 ~]$ touch /export/716f9/julie/a touch: cannot touch '/export/716f9/julie/a': Disk quota exceeded
Can anybody offer some advice?
I tried the following without any luck:
removed the hard block disk quota upgraded all packages and rebooted downgraded quota-* rpms (from version 4.04-14 to 4.04.12 used rhel 8.3) and restarting rpc-rquotad (systemctl restart rpc-rquotad)
before downgrade # rpm -qa | grep -i quota quota-4.04-14.el8.x86_64 quotatool-1.6.2-13.el8.x86_64 quota-nls-4.04-14.el8.noarch quota-rpc-4.04-14.el8.x86_64
after downgrade # rpm -qa | grep -i quota quota-rpc-4.04-10.el8.x86_64 quota-nls-4.04-10.el8.noarch quota-4.04-10.el8.x86_64
Thanks! Julie
---end quoted text--- _______________________________________________ rocky mailing list -- rocky@lists.resf.org To unsubscribe send an email to rocky-leave@lists.resf.org
Thanks Peter, I haven't downgraded the kernel yet, and I think it's a great suggestion if I could report the bug to Redhat. I just don't know if it's worth my time to build a RHEL server. Instead, I'll probably disable quota enforcement, and write a script to replace the functionality. That way, I can add some useful features of my own ;).
On 11-01-2022 01.16 +0000, Peter via rocky wrote:
I am certainly no expert on XFS disk quotas, but have you tried downgrading the kernel to see if it's a regression in the kernel module?
Do keep in mind that Rocky Linux 8.4 and 8.3 kernels are not signed for secureboot if you test this theory.
Peter
On 11/01/22 10:41 am, Julie Ashworth via rocky wrote:
hi all, I hope somebody can offer some advice on a xfs disk quota problem, where the soft block quota grace period is ignored. I confirmed that the problem affects at least 4 file servers that I mgirated to Rocky.
On all servers, xfs disk quotas have worked correctly for over a year with CentOS8.4. Last month, I upgraded the 4 fileservers from CentOS8.4 to CentOS8.5, rebooted, then ran the migrate2rocky.sh script. The migration script reported no errors. This week, I noticed that quotas grace period was ignored. I changed nothing on the servers except upgrading packages/migrating to Rocky. Now when a user's soft block quota is exceeded, write access is denied (even though the user is within the grace period, and below the hard quota).
Below is information from one server, but all 4 are similar:
# cat /etc/redhat-release Rocky Linux release 8.5 (Green Obsidian)
# uname -a Linux fs8.priv 4.18.0-348.7.1.el8_5.x86_64 #1 SMP Tue Dec 21 19:02:23 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
# rpm -qa | grep -i quota quota-4.04-14.el8.x86_64 quotatool-1.6.2-13.el8.x86_64 quota-nls-4.04-14.el8.noarch quota-rpc-4.04-14.el8.x86_64
mount filesystem with user quotas and confirm:
# mount -o uquota /dev/vg716f9/lv716f9 /export/716f9 # mount | grep 716f9 /dev/mapper/vg716f9-lv716f9 on /export/716f9 type xfs (rw,relatime,inode64,logbufs=8,logbsize=32k,sunit=256,swidth=3328,usrquota)
# xfs_quota -x -c "state -u" /export/716f9 User quota state on /export/716f9 (/dev/mapper/vg716f9-lv716f9) Accounting: ON Enforcement: ON Inode: #515 (42 blocks, 41 extents) Blocks grace time: [14 days] Blocks max warnings: 5 Inodes grace time: [7 days] Inodes max warnings: 5 Realtime Blocks grace time: [7 days]
set user 'julie' block soft quota to 5GB and block hard quota to 100GB:
# xfs_quota -x -c 'limit bsoft=5G bhard=100G julie' /export/716f9
# xfs_quota -x -c 'report -h' /export/716f9 User quota on /export/716f9 (/dev/mapper/vg716f9-lv716f9) Blocks User ID Used Soft Hard Warn/Grace
julie 4.1G 5G 100G 00 [------]
log in as user 'julie' to create a file
[julie@fs8 ~]$ touch /export/716f9/julie/a [julie@fs8 ~]$ quota -s Disk quotas for user julie (uid 1353): Filesystem space quota limit grace files quota limit grace /dev/mapper/vg716f9-lv716f9 4240M 5120M 100G 12503 0 0
set user 'julie' block soft quota to 1GB and 100GB hard, and note 14-day grace period:
# xfs_quota -x -c 'limit bsoft=1G bhard=100G julie' /export/716f9
# xfs_quota -x -c 'report -h' /export/716f9 | grep julie User quota on /export/716f9 (/dev/mapper/vg716f9-lv716f9) Blocks User ID Used Soft Hard Warn/Grace
julie 4.1G 1G 100G 00 [14 days]
log in as user 'julie' again confirm quota and try to write file
[julie@fs8 ~]$ quota -s Disk quotas for user julie (uid 1353): Filesystem space quota limit grace files quota limit grace /dev/mapper/vg716f9-lv716f9 4240M* 1024M 100G 13days 12504 0 0
[julie@fs8 ~]$ touch /export/716f9/julie/a touch: cannot touch '/export/716f9/julie/a': Disk quota exceeded
Can anybody offer some advice?
I tried the following without any luck:
removed the hard block disk quota upgraded all packages and rebooted downgraded quota-* rpms (from version 4.04-14 to 4.04.12 used rhel 8.3) and restarting rpc-rquotad (systemctl restart rpc-rquotad)
before downgrade # rpm -qa | grep -i quota quota-4.04-14.el8.x86_64 quotatool-1.6.2-13.el8.x86_64 quota-nls-4.04-14.el8.noarch quota-rpc-4.04-14.el8.x86_64
after downgrade # rpm -qa | grep -i quota quota-rpc-4.04-10.el8.x86_64 quota-nls-4.04-10.el8.noarch quota-4.04-10.el8.x86_64
Thanks! Julie
---end quoted text--- _______________________________________________ rocky mailing list -- rocky@lists.resf.org To unsubscribe send an email to rocky-leave@lists.resf.org
rocky mailing list -- rocky@lists.resf.org To unsubscribe send an email to rocky-leave@lists.resf.org
---end quoted text---
-
Julie Ashworth -
Peter