Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Disk I/O the advantage of bcache on a hardware raid
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
jpsollie
n00b
n00b


Joined: 17 Aug 2013
Posts: 39

PostPosted: Tue Apr 24, 2018 6:51 am    Post subject: Disk I/O the advantage of bcache on a hardware raid Reply with quote

so, here's the situation:
I have a huge hardware raid5 disk (32TB) which has 16+1 2TB drives. It will be used mainly as a fileserver disk (other options too, but 90% fileserver
I also have a spare 500GB SSD.
read performance of the disk:
Code:

linuxserver janpieter # dd if=/dev/sde of=/dev/null bs=512k
76758+0 records in
76757+0 records out
40242774016 bytes (40 GB) copied, 24.3736 s, 1.7 GB/s

performance of the SSD:
Code:

linuxserver janpieter # dd if=/dev/sdg of=/dev/null bs=512k
20125+0 records in
20124+0 records out
10550771712 bytes (11 GB) copied, 20.2758 s, 520 MB/s

now, my question:
I thought adding the SSD through bcache would help me to add some responsivity through the cache: the array is shut down after 20 mins of inactivity (eg: during the night), and starting up all those disks takes some time.
Would the array become more responsive when using bcache? I was mainly thinking of a situation where some content is streamed, no disk IO is present because the server has 128GB of RAM and has read some part already in memory, and after 20 mins, the array needs to boot again, causing disruptions in the streaming process.
Would it help? or would it merely involve a perfomance loss?
_________________
I am using gentoo for over 10 years now. yet seen its possibilities, I still feel like a n00b...
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 41333
Location: 56N 3W

PostPosted: Tue Apr 24, 2018 8:56 am    Post subject: Reply with quote

jpsollie,

You have two issues here and they need to be separated.

1) The throughput.
2) The latency.

The throughput is how fast you can get an your data once the raid set is awake, that's 1.7 GB/s for sequential reads.
The latency is however long it takes to spin up your raid set.

The SSD will not affect your read throughput as its cache.
It probably won't help the latency either as if data is served from the SSD, the raid won't spin up.

Letting the drives spin down like that is usually bad for their life expectancy unless they are laptop drives, which are designed for for intermittent operation to maximise battery life.

The only fix for the latency is to force the raid set to spin up before its actually needed for streaming.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
jpsollie
n00b
n00b


Joined: 17 Aug 2013
Posts: 39

PostPosted: Tue Apr 24, 2018 9:22 am    Post subject: Reply with quote

NeddySeagoon wrote:
jpsollie,
It probably won't help the latency either as if data is served from the SSD, the raid won't spin up.


Isn't that exactly what I am looking for? when a file is used, and it's available on the SSD, the SSD can serve the file instead of the disk
_________________
I am using gentoo for over 10 years now. yet seen its possibilities, I still feel like a n00b...
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 41333
Location: 56N 3W

PostPosted: Tue Apr 24, 2018 1:25 pm    Post subject: Reply with quote

jpsollie,

It depends on exactly what is on the SSD.
If its a fraction of the file you want, you will need to wait for the raid to spin up when that fraction has been used.
If none of the file is there, you have to wait for the raid to spin up at the start.
You don't control whats in the cache.

Your data throughput is fine, its the latency that needs to be addressed.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
jpsollie
n00b
n00b


Joined: 17 Aug 2013
Posts: 39

PostPosted: Wed Apr 25, 2018 6:59 am    Post subject: Reply with quote

I understand. But it isn't possible to reduce the latency by copying the whole file into RAM/ on the SSD when the raid spins up for the first time?
_________________
I am using gentoo for over 10 years now. yet seen its possibilities, I still feel like a n00b...
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 6817

PostPosted: Wed Apr 25, 2018 12:02 pm    Post subject: Reply with quote

as i read it, i think you're looking for a way to keep your raid off, but caching have a huge problem, you need to make sure the cache item remains valid, which mean, you must use a way to compare the cache item with the real item.

so no, you will never have your array kept sleeping with bcache, because it will always need to bring it up to check the cache value is not dirty.
the caching is there to solve the equation (time to read on array > time to check cache validity + read from cache), what you are trying to solve is (time to awake array + read the file on array > time to awake the array + check cache validity + read from cache).
In equation number two, you cannot remove "awake the array", because it is certainly needed for "check cache validity".
Your only way to solve this is to remove "awake the array" from the equation, which mean, making it always running.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 41333
Location: 56N 3W

PostPosted: Wed Apr 25, 2018 7:28 pm    Post subject: Reply with quote

jpsollie,

jpsollie wrote:
I understand. But it isn't possible to reduce the latency by copying the whole file into RAM/ on the SSD when the raid spins up for the first time?


The latency does not start until you run out of data in the cache, or get to the point of triggering the raid to spin up because the read ahead wants to fetch more data.
You may be able to tune the read ahead so that it triggers the raid spin up in time to keep the join between cache and data read from the raid invisible.
That may not be practical too.

e.g. You stream data at 1MB/sec. Your raid takes 60 sec to spin up.
You need to force read ahead when you have 60MB in the cache.

The parameter you need to tune is /sys/block/sdX/queue/read_ahead_kb and on my system its set to 128.
That's 128kB

Don't play with that on a production system because the read ahead value will apply to every open file on the drive.

Far safer and better for the drives to run them all the time.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum