| View previous topic :: View next topic |
| Author |
Message |
curmudgeon
Veteran
Joined: 08 Aug 2003
Posts: 1741
|
 Posted: Sat Mar 16, 2019 12:44 pm Post subject: HELP! My /home partition keeps filling up. |
 |
|
Don't know if this belongs in this forum, but none of the others seem obvious.
I don't have a lot of extra space in the /home partition to begin with (my fault - I should have made it bigger), but I am losing about 8 MB an hour to the file /home/user/.local/share/sddm/xorg-session.log (it keeps growing at that rate). I can delete the file one time, which frees up the space, but then the free disk space starts to disappear again at the same rate, and this time, the lost space is not associated with any file. This obviously leads to many unwanted reboots.
What is going on inside of that file, you ask? Well, the vast majority of it (except for the first 400 lines or so), is filled up with the following three lines over and over again:
| Code: |
[00007f947401ec80] vdpau_chroma filter error: video mixer attributes failure: An invalid handle value was provided.
[00007f947401ec80] vdpau_chroma filter error: video mixer rendering failure: An invalid handle value was provided.
[00007f947401ec80] vdpau_chroma filter error: video mixer features failure: An invalid handle value was provided.
|
I have tried to figure out what is causing this, but it looks like every kde program under the sun (I have looked at the output without counting the number of lines) has an association with that file:
| Code: |
# lsof | grep xorg-session.log | wc -l
198
|
Any ideas on how to stop this, or at least mitigate it (I would be happy if I could run a cron job a few times a day if it could release the space)?
Thank you in advance for your help. |
|
| Back to top |
|
 |
russK
l33t
Joined: 27 Jun 2006
Posts: 665
|
| Posted: Sat Mar 16, 2019 3:37 pm Post subject: |
|
|
If you are not using LVM, next time maybe, it's good to be able to resize ... https://wiki.gentoo.org/wiki/LVM. This may not help you right now, sorry.
Other possible options ...
If you have extra space on your disk unallocated, or on a disk you could add, mount it somewhere under your home, perhaps at /home/user/.local
(you would probably need to do that without your graphical session running, ask again if you need details)
I was going to suggest using the sddm configuration value, SessionLogFile to place the file somewhere else, but the man pages state it is relative to your home.
Also, maybe find a way to disable hardware acceleration? Lots of google hits regarding vlc and VDPAU suggest that something is relentlessly wanting to use features of your video drivers that are not supported (not sure).
HTH |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
| Posted: Sat Mar 16, 2019 4:10 pm Post subject: |
|
|
A temporary workaround might be to use logrotate. You could filter the logs to exclude those messages so that you could still see other errors. Or maybe just exclude those errors, mail the results and then remove the file.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Jaglover
Watchman
Joined: 29 May 2005
Posts: 8291
Location: Saint Amant, Acadiana
|
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21607
|
| Posted: Sat Mar 16, 2019 4:49 pm Post subject: Re: HELP! My /home partition keeps filling up. |
|
|
| curmudgeon wrote: | | I can delete the file one time, which frees up the space, but then the free disk space starts to disappear again at the same rate, and this time, the lost space is not associated with any file. |
It's probably still associated with the deleted file. The space isn't free until the file is deallocated, which requires both that it have no links and that no program have it open. Deleting it eliminates the last link, but until you kill whatever is writing to it, it is still open and still allocated. | curmudgeon wrote: | | I have tried to figure out what is causing this, but it looks like every kde program under the sun (I have looked at the output without counting the number of lines) has an association with that file: |
Probably the file is opened by your startup script, then the descriptor leaks into every child process from there. This is arguably a bug in whatever opened it. The file descriptor should not be passed down to processes which have no use for it. | curmudgeon wrote: | | Code: | | # lsof | grep xorg-session.log | wc -l |
|
| Code: | | lsof | grep -c -F xorg-session.log |
| curmudgeon wrote: | | Any ideas on how to stop this, or at least mitigate it (I would be happy if I could run a cron job a few times a day if it could release the space)? |
You could try truncating the file instead of deleting it. Subsequent writes would restore the apparent size, but the file would be sparse, which should help. The best thing to do would be to fix the descriptor leak so that lsof produces useful output, then hunt down the process(es) writing that message and terminate them. | pjp wrote: | | A temporary workaround might be to use logrotate. You could filter the logs to exclude those messages so that you could still see other errors. Or maybe just exclude those errors, mail the results and then remove the file. |
Log rotation only helps if the rotation frees the space of the old log. That means either:- truncating the log file or
- deleting the old log file and
- forcing the programs that are writing to it to close and reopen it
|
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
| Posted: Sat Mar 16, 2019 5:18 pm Post subject: |
|
|
I presumed enough knowledge with the tool, or if not, the ability to read about it. And the opportunity to ask for help :)
For reference, 'man logrotate' section 'copytruncate' (too many options to list, many may be useful in this situation). Off the top of my head, rotate on size (multiple options) and/or time (multiple options) and one or more of the features that rely on 'endscript'.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Sun Mar 17, 2019 10:03 am Post subject: |
|
|
I don't think logrotate would help there, as Hu point out already, the space is not deallocated until the program release the handle over the file, that's just linux virtual fs strengh.
deleting or rotating the file would gave the feeling the space has been freed... |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6097
Location: Dallas area
|
| Posted: Sun Mar 17, 2019 10:50 am Post subject: |
|
|
set up cron to run hourly and instead of deleting the file do a "> filename" on it to truncate it. That way the filename will stay there.
There is a config option for the log file, but it's always relative to the home directory  and seemingly no way to tell it not to log
Edit to add: about the last sentence, it might be possible to make it leave the .local directory structure, but appending enough ../ to get it out of /home.
But that's something that you would have to try to see if it's feasible.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
Goverp
Veteran
Joined: 07 Mar 2007
Posts: 1999
|
| Posted: Sun Mar 17, 2019 12:06 pm Post subject: |
|
|
Would symlinking the log to /dev/null work?
_________________
Greybeard |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Sun Mar 17, 2019 1:00 pm Post subject: |
|
|
| Goverp wrote: | | Would symlinking the log to /dev/null work? |
was thinking about that too, it should ; but that's not strictly a nice solve.
user will lost valuable source of debugging by loosing whole log usage.
I think it would be better to find a way to make vdpau_chroma filter shutup as the info is filling the log and is not really helpful once you know the error exists. |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
| Posted: Mon Mar 18, 2019 5:11 am Post subject: |
|
|
| krinn wrote: | | I don't think logrotate would help there, as Hu point out already, the space is not deallocated until the program release the handle over the file |
| man logrotate wrote: | copytruncate
Truncate the original log file to zero size in place after creating a copy, instead of moving the old log file and optionally
creating a new one. It can be used when some program cannot be told to close its logfile and thus might continue writing
(appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating
it, so some logging data might be lost. When this option is used, the create option will have no effect, as the old log file
stays in place. |
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Goverp
Veteran
Joined: 07 Mar 2007
Posts: 1999
|
| Posted: Mon Mar 18, 2019 9:16 am Post subject: |
|
|
| krinn wrote: | | Goverp wrote: | | Would symlinking the log to /dev/null work? |
...
user will lost valuable source of debugging by loosing whole log usage
... |
I find the lack of timestamps render the log's contents useless. IIUC, it's just the catenation of all the system.out from stuff running in X, and those programs expected it to be read at the time of creation, not used for later forensics.
_________________
Greybeard |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Mon Mar 18, 2019 1:12 pm Post subject: |
|
|
Out of curiosity, I downloaded the tarballs for everything with vdpau in it's name and searched for the error message.
| Code: | $ ls
libva-2.4.0 libva-vdpau-driver-0.7.4 libvdpau-libvdpau-1.2 vdpauinfo-1.0
tony@MSI vdpau $ grep -r "vdpau_chroma filter error" *
tony@MSI vdpau $
|
Interesting. Because it seems to me that you want to know why that message appears and at least temporarily, patch it out, or fix the video problem.
Are you running a 9999 version?
EDIT: Interestingly, I don't even have a file named xorg-session.log | Code: | tony@MSI ~ $ locate xorg-session.log
tony@MSI ~ $
|
What package does it belong to? I have Xorg.0.log and Xorg.1.log |
|
| Back to top |
|
|
saellaven
l33t
Joined: 23 Jul 2006
Posts: 646
|
| Posted: Mon Mar 18, 2019 1:31 pm Post subject: |
|
|
| Almost everyone is focusing on the symptom... the cure is you're going to want a bigger /home partition anyway. If log files are making it impossible to keep your system running, you almost certainly need more space to store your actual files. |
|
| Back to top |
|
|
1clue
Advocate
Joined: 05 Feb 2006
Posts: 2569
|
| Posted: Mon Mar 18, 2019 3:53 pm Post subject: |
|
|
| saellaven wrote: | | Almost everyone is focusing on the symptom... the cure is you're going to want a bigger /home partition anyway. If log files are making it impossible to keep your system running, you almost certainly need more space to store your actual files. |
I agree that almost everyone is focusing on the symptom, but a bigger partition is not it. It's a separate but slightly related topic. +1 for LVM on that, BTW.
The cure is to figure out why the system is printing those three lines, and fix that problem. No more logs growing continuously.
<rant>
IMO the biggest fail in system administration is tolerating a "harmless" repeating error to spam the logs. When something is really broken, and keeping 1000 people who work by the hour from doing their jobs, you don't want to wade through 10 gigabytes of "harmless" errors to get to what's really broken. Our logs should not have errors in everyday system use.
Likewise writing code that spams logs with useless information without a 'debug' priority is even worse, because it prevents people from diagnosing the problem as well.
</rant> |
|
| Back to top |
|
|
Naib
Watchman
Joined: 21 May 2004
Posts: 6051
Location: Removed by Neddy
|
| Posted: Mon Mar 18, 2019 5:27 pm Post subject: |
|
|
Why is this log growing at a rate of 8meg an hour?
Is the log telling you something is wrong ? If so a better approach would be to resolve the cause of the log entries
If it is normal operation (and I really would question that), do you need the log file or is it just a byproduct of what you use? You could symlink against /dev/null
This however is another frig around /home being too small for your usage BUT I would always target the root cause and in this instance the root cause is the logging while growing /home is an architectural decision which would still produce a large log
_________________
| Quote: | | Removed by Chiitoo |
|
|
| Back to top |
|
|
ct85711
Veteran
Joined: 27 Sep 2005
Posts: 1791
|
| Posted: Mon Mar 18, 2019 6:45 pm Post subject: |
|
|
Searching the error, it seems this "vdpau_chroma filter error" message is associated with vlc only (I haven't seen anything from any searches except those that is directly involved with vlc). At the same time, it seems this message is also primarily a thing of your graphics hardware (drivers) do not support vdpau.
I do recall seeing a couple references of a corrupted vdpau driver also causing this problem too. |
|
| Back to top |
|
|
saellaven
l33t
Joined: 23 Jul 2006
Posts: 646
|
| Posted: Mon Mar 18, 2019 8:27 pm Post subject: |
|
|
| 1clue wrote: | | saellaven wrote: | | Almost everyone is focusing on the symptom... the cure is you're going to want a bigger /home partition anyway. If log files are making it impossible to keep your system running, you almost certainly need more space to store your actual files. |
I agree that almost everyone is focusing on the symptom, but a bigger partition is not it. It's a separate but slightly related topic. +1 for LVM on that, BTW.
The cure is to figure out why the system is printing those three lines, and fix that problem. No more logs growing continuously. |
Right now, it's X filling up the partition, tomorrow it's going to be something else, so there are two problems:
1) the partition is too small if logging alone is going to cause it to fill up and require a reboot/restarting X (this is actually one of the advantages in breaking up your system into multiple partitions (/var or /var/log in particular) - you can ensure that your system will remain in a usable state even if something goes poorly elsewhere)
2) the X error needs to be tracked down and fixed
Fixing one alone isn't going to cure the problems of other. |
|
| Back to top |
|
|
1clue
Advocate
Joined: 05 Feb 2006
Posts: 2569
|
| Posted: Mon Mar 18, 2019 9:28 pm Post subject: |
|
|
| saellaven wrote: | | 1clue wrote: | | saellaven wrote: | | Almost everyone is focusing on the symptom... the cure is you're going to want a bigger /home partition anyway. If log files are making it impossible to keep your system running, you almost certainly need more space to store your actual files. |
I agree that almost everyone is focusing on the symptom, but a bigger partition is not it. It's a separate but slightly related topic. +1 for LVM on that, BTW.
The cure is to figure out why the system is printing those three lines, and fix that problem. No more logs growing continuously. |
Right now, it's X filling up the partition, tomorrow it's going to be something else, so there are two problems:
1) the partition is too small if logging alone is going to cause it to fill up and require a reboot/restarting X (this is actually one of the advantages in breaking up your system into multiple partitions (/var or /var/log in particular) - you can ensure that your system will remain in a usable state even if something goes poorly elsewhere)
2) the X error needs to be tracked down and fixed
Fixing one alone isn't going to cure the problems of other. |
Exactly.
If you resize the partition only then you simply delay and magnify the problem with logs growing.
If you fix the logs only, then you will inevitably run out of space for some other reason, especially since this is the user's home partition.
This is, in my opinion, the beauty of LVM. You can resize volumes on-the-fly either bigger or smaller. |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Mon Mar 18, 2019 11:28 pm Post subject: |
|
|
| 1clue, +10000 on your rant! People tolerate things even on servers that an embedded programmer never would, at least if he/she didn't want airplanes falling out of the sky or nuclear power plants exploding. |
|
| Back to top |
|
|
curmudgeon
Veteran
Joined: 08 Aug 2003
Posts: 1741
|
| Posted: Sat Mar 23, 2019 11:31 pm Post subject: |
|
|
I am aware of the benefits of LVM, and certainly use it for a lot of things (especially crypto-related), I am one of these dinosaurs who still feel uncomfortable resizing a live filesystem (not the LVM container itself, though).
| Hu wrote: |
You could try truncating the file instead of deleting it. Subsequent writes would restore the apparent size, but the file would be sparse, which should help. The best thing to do would be to fix the descriptor leak so that lsof produces useful output, then hunt down the process(es) writing that message and terminate them.
|
Thanks. I will pursue this.
| Anon-E-moose wrote: |
set up cron to run hourly and instead of deleting the file do a "> filename" on it to truncate it. That way the filename will stay there.
|
Thanks. Will give that a try, as well.
| Tony0945 wrote: |
Out of curiosity, I downloaded the tarballs for everything with vdpau in it's name and searched for the error message.
| Code: | $ ls
libva-2.4.0 libva-vdpau-driver-0.7.4 libvdpau-libvdpau-1.2 vdpauinfo-1.0
tony@MSI vdpau $ grep -r "vdpau_chroma filter error" *
tony@MSI vdpau $
|
Interesting. Because it seems to me that you want to know why that message appears and at least temporarily, patch it out, or fix the video problem.
Are you running a 9999 version?
EDIT: Interestingly, I don't even have a file named xorg-session.log | Code: | tony@MSI ~ $ locate xorg-session.log
tony@MSI ~ $
|
What package does it belong to? I have Xorg.0.log and Xorg.1.log
|
No 9999 versions of anything.
Since this file (under the /home directory, which on my machines is a separate partition) is named /home/user/.local/share/sddm/xorg-session.log, I am assuming that sddm is the app that is writing to it (and I am not really sure why - I thought that display managers mostly did greeting and logon, and then got out of the way).
| saellaven wrote: |
Almost everyone is focusing on the symptom... the cure is you're going to want a bigger /home partition anyway. If log files are making it impossible to keep your system running, you almost certainly need more space to store your actual files.
|
The size of the /home partition is adequate for its intended use (which is only to store user settings and caches).
Something growing endlessly will eventually consume any arbitrarily large (but fixed) amount of storage space. More practically, 8 MB in one hour comes to 8 GB in about six weeks, and my uptimes are usually multiples of that.
| 1clue wrote: |
<rant>
IMO the biggest fail in system administration is tolerating a "harmless" repeating error to spam the logs. When something is really broken, and keeping 1000 people who work by the hour from doing their jobs, you don't want to wade through 10 gigabytes of "harmless" errors to get to what's really broken. Our logs should not have errors in everyday system use.
Likewise writing code that spams logs with useless information without a 'debug' priority is even worse, because it prevents people from diagnosing the problem as well.
</rant>
|
Completely agree.
| ct85711 wrote: |
Searching the error, it seems this "vdpau_chroma filter error" message is associated with vlc only (I haven't seen anything from any searches except those that is directly involved with vlc). At the same time, it seems this message is also primarily a thing of your graphics hardware (drivers) do not support vdpau.
|
It does appear vlc-related. I am using nvidia-drivers (although the 340 series since this is an older card). I would have thought that even far older drivers than I am running support vdpau.
| Code: |
$ equery depends libvdpau
* These packages depend on libvdpau:
media-libs/mesa-18.2.8 (vdpau ? >=x11-libs/libvdpau-1.1[abi_x86_32(-)?,abi_x86_64(-)?,abi_x86_x32(-)?,abi_mips_n32(-)?,abi_mips_n64(-)?,abi_mips_o32(-)?,abi_ppc_32(-)?,abi_ppc_64(-)?,abi_s390_32(-)?,abi_s390_64(-)?])
media-video/ffmpeg-3.4.5 (vdpau ? >=x11-libs/libvdpau-0.7[abi_x86_32(-)?,abi_x86_64(-)?,abi_x86_x32(-)?,abi_mips_n32(-)?,abi_mips_n64(-)?,abi_mips_o32(-)?,abi_ppc_32(-)?,abi_ppc_64(-)?,abi_s390_32(-)?,abi_s390_64(-)?])
media-video/mplayer-1.3.0-r5 (vdpau ? x11-libs/libvdpau)
media-video/mpv-0.27.2 (x11-libs/libvdpau)
x11-drivers/nvidia-drivers-340.107 (X ? >=x11-libs/libvdpau-0.3-r1)
|
I notice that only mpv (my main media player, of course) absolutely requires libvdpau. I am wondering if rebuilding ffmpeg without the vdpau USE flag would help. |
|
| Back to top |
|
|
Tony0945
Watchman
Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA
|
| Posted: Sun Mar 24, 2019 12:14 am Post subject: |
|
|
| curmudgeon wrote: | | It does appear vlc-related. I am using nvidia-drivers (although the 340 series since this is an older card). I would have thought that even far older drivers than I am running support vdpau. |
Don't know if it will help, but build the nouveau module and try it instead of nvidia. If the log spam stops, you know the root cause. And the display may be acceptable for vlc. I'm assuming you are not gaming. Even if you are, try it as an experiment.
I switched to nouveau when my old card fell out of nvidia-drivers and it runs fine. I'm not picky about the display though, I have cataracts and presbyopia. |
|
| Back to top |
|
|
1clue
Advocate
Joined: 05 Feb 2006
Posts: 2569
|
| Posted: Sun Mar 24, 2019 12:59 am Post subject: |
|
|
| curmudgeon wrote: | | I am aware of the benefits of LVM, and certainly use it for a lot of things (especially crypto-related), I am one of these dinosaurs who still feel uncomfortable resizing a live filesystem (not the LVM container itself, though). |
In this particular case LVM would be especially handy. You could resize the partition to something way too large, to give yourself time to debug the logging issue. Then when you figure it out, you could resize it back to whatever you feel is reasonable.
Back in the day, around 20 years ago, I worked at IBM and was trying to learn to maintain RS/6000s, which ran IBM's UNIX. AIX. AIX used JFS and whatever else necessary to be very similar to LVM. The volumes were resizable on the fly as well, and it was common practice to shrink infrequently written volumes to be just barely larger than the data it held. Sometimes within 1%.
I used to rail about this, and the people who actually knew what they were doing would come by and listen, and then go away after that. It must have been pretty entertaining. Now, I find myself treating Linux volumes similarly, although I don't shrink them down so far. |
|
| Back to top |
|
|
ct85711
Veteran
Joined: 27 Sep 2005
Posts: 1791
|
| Posted: Sun Mar 24, 2019 7:19 am Post subject: |
|
|
| Quote: | | I notice that only mpv (my main media player, of course) absolutely requires libvdpau. |
Just double checking, mpv also has the vdpau use flag which controls the libvdpau dependency...
The applicable portion of mpv-0.27.ebuild
| Code: | vaapi? (
!libav? ( >=media-video/ffmpeg-3.3:0 )
libav? ( >=media-video/libav-13_pre20171219:0 )
x11-libs/libva:=[drm?,X?,wayland?]
)
vdpau? (
!libav? ( >=media-video/ffmpeg-3.3:0 )
libav? ( >=media-video/libav-13_pre20171219:0 )
x11-libs/libvdpau
)
|
Going by that, simply turning off vdpau use flag, would remove the dependency on libvdpau (of course, recompiling the package so the change will take effect). |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6097
Location: Dallas area
|
| Posted: Sun Mar 24, 2019 10:43 am Post subject: |
|
|
| curmudgeon wrote: |
Since this file (under the /home directory, which on my machines is a separate partition) is named /home/user/.local/share/sddm/xorg-session.log, I am assuming that sddm is the app that is writing to it (and I am not really sure why - I thought that display managers mostly did greeting and logon, and then got out of the way).
|
It's not that sddm is writing to the file, sddm starts X and that's the file it tells X to output it's runtime messages to.
On my system it's ~/.xsession-errors which is the default if not overridden, it seems sddm puts it in the .local subdir and as I mention earlier the location can be changed, but not turned off.
Post your .config/mpv/mpv.conf file.
You can usually turn off vdpau there.
Edit to add:
I'm not sure why you say mpv "requires" vdpau, it's an option flag, if on, then it pulls in libvdpau, but nothing seems to make it be on.
as ct85711 says turn off the vdpau use flag
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Other Things Gentoo
