Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Status during USB file transfers
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
ALF__
Apprentice
Apprentice


Joined: 30 Nov 2003
Posts: 252

PostPosted: Tue Oct 13, 2020 8:58 pm    Post subject: Status during USB file transfers Reply with quote

So, i have this strange problem that ive never encountered before.

When i mount a USB-stick, and transfer files to it, the "status" is finished almost instantly. But the transfer is still going in the background.
This is systemwide, so its not only true for a specific application.

For example, 400mb takes a while, but the status says it is finished in about 2 seconds. However, the LED on the stick keeps blinking for the right amount of time.
It is also noticable because when unmounting it wont release before its finished in the background, so that works as expected.


This is not just for a perticular stick or as stated before; application.


Any ideas?
Back to top
View user's profile Send private message
Hu
Administrator
Administrator


Joined: 06 Mar 2007
Posts: 23411

PostPosted: Tue Oct 13, 2020 10:25 pm    Post subject: Reply with quote

The file pages were quickly and successfully passed to the kernel to be written to the device. The kernel will write them to the device in the background as time and resources permit. If you need to block until the data is written, call fsync after calling write. This behavior is expected, and occurs on all devices. It is more noticeable on devices that are slow to write.
Back to top
View user's profile Send private message
ALF__
Apprentice
Apprentice


Joined: 30 Nov 2003
Posts: 252

PostPosted: Wed Oct 14, 2020 12:00 pm    Post subject: Reply with quote

Hu wrote:
The file pages were quickly and successfully passed to the kernel to be written to the device. The kernel will write them to the device in the background as time and resources permit. If you need to block until the data is written, call fsync after calling write. This behavior is expected, and occurs on all devices. It is more noticeable on devices that are slow to write.


Oh wow.

So i.e the progress bar of MC is not for the actual file-transfer but from harddrive -> RAM then? That's not very meaningful, and im surprised i only noticing this now.

Well! Thanks for the heads up.
Back to top
View user's profile Send private message
Hu
Administrator
Administrator


Joined: 06 Mar 2007
Posts: 23411

PostPosted: Wed Oct 14, 2020 3:54 pm    Post subject: Reply with quote

I cannot say how the progress bar is judged. If the application is routinely asking the kernel to fsync the data, then after fsync returns, the application can assume that the device has reported that the data was written. Some devices are known to lie about this and merely report it as written when it is transferred to the device's internal cache (as opposed to still being in system RAM). I do not recall how common that behavior is.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 10019
Location: almost Mile High in the USA

PostPosted: Wed Oct 14, 2020 4:48 pm    Post subject: Reply with quote

Fortunately most devices internal write cache is fairly small, though for like hardware RAID this can be fairly large and who knows when a particular write completes.

I suspect the copy (write) routines can call open() with synchronous writes and thus its status bar should know exactly when a write completes. For the most part writing large files sequentially should be safe, but writing lots of random blocks will limit the disk schedulers what to write first for best performance. Of course this most benefits mechanical disks, but can affect solid state disks as well - the OS can affect fragmentation and metadata writes on SSDs, which can turn into reduced write amplification down the road when it comes time to garbage collect.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum