View previous topic :: View next topic |
Author |
Message |
Uncle_Psychosis Guru
Joined: 31 Jan 2004 Posts: 387
|
Posted: Tue Dec 28, 2010 12:12 am Post subject: migrating to a new HDD |
|
|
Hi guys
The time has come to upgrade to a bigger, hopefully faster hard drive.
I have had my current gentoo install for a good five years so I'm a little bit nervous about what I need to do to move to a new hard drive. Do any of you have any good tips for doing so? Should I just follow the install guide again or is there an easy way to copy all my stuff to a new (and not necessarily the same) file system layout?
Also, what file systems would you recommend for / and /home? (I put them on separate partitions so they don't necessarily have to be the same)
Cheers
Sam |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54028 Location: 56N 3W
|
Posted: Tue Dec 28, 2010 12:31 am Post subject: |
|
|
Uncle_Psychosis,
Make a mount point for your new drive, say /mnt/new
partition it to your liking and make filesystems. I suggest ext4 but rieser and ext3 work too.
Assemble the partitions into a filesystem tree. under /mnt/new
Copy everything over. This is best done with a liveCD as you must not copy /dev /proc, /sys and there is no point in copying /tmp
You can use rsync, cp -a, tar etc for the copy.
After the copy install grub onto the MBR. What you do next depends what you will do with the old drive.
You may need to change /etc/fstab and grub.conf to get the new drive to boot.
Oh, be sure your kernel supports any new filesystems you have elected to use. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
d2_racing Bodhisattva
Joined: 25 Apr 2005 Posts: 13047 Location: Ste-Foy,Canada
|
Posted: Tue Dec 28, 2010 4:41 am Post subject: |
|
|
Hi, I use rsync all the time to backup or migrate my stuff.
I use this command :
Code: |
# time rsync -aHA --del --force --stats --progress /mnt/backup/gentoo/ /mnt/gentoo
|
Where /mnt/backup/gentoo is my old hdd and /mnt/gentoo is my new hdd.
Clean and simple. |
|
Back to top |
|
|
Uncle_Psychosis Guru
Joined: 31 Jan 2004 Posts: 387
|
Posted: Tue Dec 28, 2010 8:50 am Post subject: |
|
|
Ahha...now that seems like the kind of thing I need!
The only problem I can forsee is that I also need to put windows onto the new drive. My plan is to:
1. remove exisiting drives
2. plug in new one
3. partition new drive
4. install windows
5. plug in existing drives again
6. boot from live cd
7. copy my gentoo root folder to the new drive using rsync (thanks for the command)
8. install grub on the new disk and modify fstab as necessary
9. boot into new system
10. sort out the horrible mess that is my existing disks...
Also, I have a question about swap space. How much do you actually need for a modern PC? I've got 8Gb of RAM---do I really need to waste a partition on swap? |
|
Back to top |
|
|
XQYZ Apprentice
Joined: 19 Jul 2009 Posts: 231 Location: Europe
|
Posted: Tue Dec 28, 2010 12:56 pm Post subject: |
|
|
Uncle_Psychosis wrote: | Also, I have a question about swap space. How much do you actually need for a modern PC? I've got 8Gb of RAM---do I really need to waste a partition on swap? |
Hibernation would be an argument. also: can you really not spare the 8gb on your hard drive for a swap partition? |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54028 Location: 56N 3W
|
Posted: Tue Dec 28, 2010 1:06 pm Post subject: |
|
|
Uncle_Psychosis,
If you need more than 512Mb for swapping your really don't have enough RAM.
However, swap a little larger than your RAM is needed for hibernation, if thats important to you.
You should have a swap, as not having swap space does not stop the kernel from swapping, you just disable one of the mechanisims for swapping that is available to the kernel.
Without swap, there is no home on disc for dynamically allocated RAM. The kernel will still flush/reload code and data that belongs to a disk file when RAM is short. Thats a form of swapping.
Note that code/data that have a real home in a disk file are never written to swap. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
Fitzcarraldo Advocate
Joined: 30 Aug 2008 Posts: 2034 Location: United Kingdom
|
Posted: Tue Dec 28, 2010 4:36 pm Post subject: Size of swap if you want the PC to be able to hibernate |
|
|
NeddySeagoon wrote: | However, swap a little larger than your RAM is needed for hibernation, if thats important to you. |
I've read in the past that swap can be slightly smaller, not larger, than your RAM if you want Linux to be able to hibernate. The reason for this was stated as being because the contents of RAM are compressed when saved to disk.
Does anyone have a definitive reference for the amount of swap required for hibernation (suspend-to-disk)? |
|
Back to top |
|
|
XQYZ Apprentice
Joined: 19 Jul 2009 Posts: 231 Location: Europe
|
Posted: Tue Dec 28, 2010 5:01 pm Post subject: |
|
|
AFAIK at least the default configuration of tuxonice with hibernate-scripts doesn't compress the ram when it's written to disk. I'm not sure compression would make that much sense anyway as it takes quite a bit to do it and most of the time you wouldn't even get a good ratio out of it. Also I don't think you really can give an accurate number for this. I mean the safest bet is make it at least as big as your ram is, so even if it's not compressible at all, it still works. I mean personally I'd hate it if it wouldn't fit at some point just because I'm out of luck.
http://tuxonice.net/ could have some info on it, but apparently you need to register yourself first for wiki access. |
|
Back to top |
|
|
Fitzcarraldo Advocate
Joined: 30 Aug 2008 Posts: 2034 Location: United Kingdom
|
Posted: Tue Dec 28, 2010 8:20 pm Post subject: Size of swap for hibernation |
|
|
Had a hunt for the post I remembered reading, and found it in KernelTrap: http://kerneltrap.org/node/16317#comment-303704
Quote: | Anonymous (not verified) on June 23, 2008 - 5:10pm
Isn't swap necessary for Suspend To Disk/Hibernate? And if so, how much? |
Quote: | Anonymous (not verified) on June 24, 2008 - 5:45am
I think for suspend to disk you need a storage space an large as your ram. |
Quote: | Anonymous (not verified) on June 24, 2008 - 12:57pm
suspend-to-disk uses compressed images, so you can do with less swap than RAM (if it compresses well). It also only stores actual program images, so caches are likely to get evicted, the kernel is mostly unloaded, etc. |
Is that actually the case?
EDIT (21:03 GMT): Just found this post from TuxOnIce developer Nigel Cunningham (http://lists.tuxonice.net/pipermail/tuxonice-devel/2009-June/005423.html):
Nigel Cunningham wrote: | > * How the installer of a Linux distribution should decide on the size of
> the hibernation file/partition? Worst case seems to be the installed
> memory size + swap partition but given that LZO compresses the image
> well enough is it possible to assume a size for the image file based on
> some proportions or something else?
LZO compression almost always achieves greater than 50% compression, so
you should be able to fit all images in swapspace equal to half the
amount of memory you have. Of course you might want to add a margin for
normal swapping and variations in the compression that's achieved. |
So it seems that making swap the same size as RAM is more than enough. |
|
Back to top |
|
|
Fitzcarraldo Advocate
Joined: 30 Aug 2008 Posts: 2034 Location: United Kingdom
|
Posted: Wed Dec 29, 2010 4:40 pm Post subject: Size of swap required for hibernation |
|
|
I did a little experiment to see what TuxOnIce does.
Here are my RAM and swap sizes in bytes:
Code: | # free -bt
total used free shared buffers cached
Mem: 4142669824 2399014912 1743654912 0 175312896 1153126400
-/+ buffers/cache: 1070575616 3072094208
Swap: 4293558272 0 4293558272
Total: 8436228096 2399014912 6037213184 |
My (2.6.36) kernel config was as follows w.r.t. LZO compression:
Code: | $ cat /usr/src/linux/.config | grep LZO
CONFIG_HAVE_KERNEL_LZO=y
# CONFIG_KERNEL_LZO is not set <---- Kernel Compression Mode was selected to be LZMA rather than LZO
CONFIG_RD_LZO=y
CONFIG_JFFS2_LZO=y
# CONFIG_JFFS2_CMODE_FAVOURLZO is not set
CONFIG_UBIFS_FS_LZO=y
CONFIG_SQUASHFS_LZO=y
CONFIG_CRYPTO_LZO=m
CONFIG_LZO_COMPRESS=m
CONFIG_LZO_DECOMPRESS=y
CONFIG_DECOMPRESS_LZO=y |
and TuxOnIce did not work when I tried to hibernate my laptop:
Code: | Dec 29 12:01:06 meshedgedx kernel: [ 272.829994] TuxOnIce: Failed to initialise the lzo compression transform.
Dec 29 12:01:06 meshedgedx kernel: [ 272.829995] Failed to initialise compression algorithm.
Dec 29 12:01:06 meshedgedx kernel: [ 272.829996] Continuing without compressing the image.
.
.
.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908327] TuxOnIce debugging info:
Dec 29 12:01:06 meshedgedx kernel: [ 316.908330] - TuxOnIce core : 3.2-rc2
Dec 29 12:01:06 meshedgedx kernel: [ 316.908331] - Kernel Version : 2.6.36-sabayon
Dec 29 12:01:06 meshedgedx kernel: [ 316.908333] - Compiler vers. : 4.4
Dec 29 12:01:06 meshedgedx kernel: [ 316.908335] - Attempt number : 1
Dec 29 12:01:06 meshedgedx kernel: [ 316.908336] - Parameters : 32777 667648 0 0 0 5
Dec 29 12:01:06 meshedgedx kernel: [ 316.908338] - Overall expected compression percentage: 0.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908340] - Checksum method is 'md4'.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908344] 0 pages resaved in atomic copy.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908345] - Block I/O active.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908346] Used 0 pages from swap on /dev/sda5.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908347] - Max outstanding reads 1052. Max writes 20353.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908348] Memory_needed: 1024 x (4096 + 344 + 112) = 4661248 bytes.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908349] Free mem throttle point reached 0.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908350] - Swap Allocator enabled.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908351] Swap available for image: 1048232 pages.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908352] - File Allocator active.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908352] Storage available for image: 0 pages.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908353] - I/O speed: Write 54180 KB/s.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908354] - Extra pages : 5179 used/500.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908355] - Result : Hibernation was aborted.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908356] : Insufficient storage was available.
Dec 29 12:01:06 meshedgedx kernel: [ 316.908357] : I/O errors were encountered. |
As you can see above, with LZO compression configured to be a module instead of in-kernel, TuxOnIce did not compress RAM, and the 4293558272 bytes of swap was insufficient for hibernation when I have 4142669824 bytes RAM (of which 2399014912 was in use). So I used the kernel configuration options specified in the Gentoo Wiki article on TuxOnIce, and rebuilt my kernel to perform in-kernel LZO compression:
Code: | $ cat /usr/src/linux/.config | grep LZO
CONFIG_HAVE_KERNEL_LZO=y
# CONFIG_KERNEL_LZO is not set <---- I left Kernel Compression Mode as LZMA rather than LZO
CONFIG_RD_LZO=y
CONFIG_JFFS2_LZO=y
# CONFIG_JFFS2_CMODE_FAVOURLZO is not set
CONFIG_UBIFS_FS_LZO=y
CONFIG_SQUASHFS_LZO=y
CONFIG_CRYPTO_LZO=y
CONFIG_LZO_COMPRESS=y
CONFIG_LZO_DECOMPRESS=y
CONFIG_DECOMPRESS_LZO=y |
and now the laptop hibernates successfully:
Code: | Dec 29 15:22:45 meshedgedx kernel: [ 1205.711101] TuxOnIce debugging info:
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711102] - TuxOnIce core : 3.2-rc2
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711103] - Kernel Version : 2.6.36-sabayon
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711103] - Compiler vers. : 4.4
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711104] - Attempt number : 1
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711104] - Parameters : 0 667648 0 0 0 5
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711105] - Overall expected compression percentage: 0.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711105] - Checksum method is 'md4'.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711106] 0 pages resaved in atomic copy.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711107] - Compressor is 'lzo'.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711107] Compressed 2180071424 bytes into 914681193 (58 percent compression).
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711108] - Block I/O active.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711108] Used 225238 pages from swap on /dev/sda5.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711109] - Max outstanding reads 1338. Max writes 10377.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711110] Memory_needed: 1024 x (4096 + 344 + 112) = 4661248 bytes.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711110] Free mem throttle point reached 0.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711111] - Swap Allocator enabled.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711112] Swap available for image: 1048232 pages.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711112] - File Allocator active.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711113] Storage available for image: 0 pages.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711113] - I/O speed: Write 130 MB/s, Read 153 MB/s.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711114] - Extra pages : 5214 used/500.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711115] - Result : Succeeded. |
If I understand the above TuxOnIce output correctly, with in-kernel LZO compression enabled my hibernating laptop with 3950 KB RAM occupied 225238/1048232 pages of swap = 22 per cent of my laptop's 4094 KB swap. The amount of swap used for hibernation was 225238x4096/4142669824 = 23 per cent of my laptop's RAM.
So, making the swap size the same as the RAM size is more than enough. There is certainly no need to make the swap size larger than the RAM size, and one could get away with making the swap size smaller than the RAM size. |
|
Back to top |
|
|
Uncle_Psychosis Guru
Joined: 31 Jan 2004 Posts: 387
|
Posted: Thu Jan 27, 2011 5:47 pm Post subject: |
|
|
d2_racing wrote: | Hi, I use rsync all the time to backup or migrate my stuff.
I use this command :
Code: |
# time rsync -aHA --del --force --stats --progress /mnt/backup/gentoo/ /mnt/gentoo
|
Where /mnt/backup/gentoo is my old hdd and /mnt/gentoo is my new hdd.
Clean and simple. |
Right, I'm finally getting round to doing this. Will your command copy everything thats under / ?
Thanks
Sam |
|
Back to top |
|
|
d2_racing Bodhisattva
Joined: 25 Apr 2005 Posts: 13047 Location: Ste-Foy,Canada
|
Posted: Sun Jan 30, 2011 4:53 am Post subject: |
|
|
Yes |
|
Back to top |
|
|
Uncle_Psychosis Guru
Joined: 31 Jan 2004 Posts: 387
|
Posted: Sun Mar 13, 2011 3:40 pm Post subject: |
|
|
Right, I've finally found the time to do this.
d2_racing, I'm going to use your command to copy my / partition to the new drive.
However, I currently have /home on its own partition and I dont want to back that up too---I want to back that up separately (plan is to partition the new drive the same as the old one, only each partition will be bigger)
Thanks
Sam |
|
Back to top |
|
|
d2_racing Bodhisattva
Joined: 25 Apr 2005 Posts: 13047 Location: Ste-Foy,Canada
|
Posted: Sun Mar 13, 2011 4:04 pm Post subject: |
|
|
Use the switch exclude then :
--exclude /home' |
|
Back to top |
|
|
davidm Guru
Joined: 26 Apr 2009 Posts: 557 Location: US
|
Posted: Sun Mar 13, 2011 4:06 pm Post subject: Re: Size of swap for hibernation |
|
|
Also see:
http://www.mjmwired.net/kernel/Documentation/power/interface.txt
Quote: |
50 /sys/power/image_size controls the size of the image created by
51 the suspend-to-disk mechanism. It can be written a string
52 representing a non-negative integer that will be used as an upper
53 limit of the image size, in bytes. The suspend-to-disk mechanism will
54 do its best to ensure the image size will not exceed that number. However,
55 if this turns out to be impossible, it will try to suspend anyway using the
56 smallest image possible. In particular, if "0" is written to this file, the
57 suspend image will be as small as possible.
58
59 Reading from this file will display the current image size limit, which
60 is set to 500 MB by default.
61
|
Note: On mine it actually seems to be set to "847683584" by default so I think the default might have changed or there is some other process at work which I am not aware of that sets it. My RAM is 2 GB and I have also created a 2 GB swap. |
|
Back to top |
|
|
Uncle_Psychosis Guru
Joined: 31 Jan 2004 Posts: 387
|
Posted: Tue Mar 15, 2011 9:40 pm Post subject: |
|
|
Just out of interest... can I do this with a windows partition? Or does windows actually need to be installed "properly"? |
|
Back to top |
|
|
Hu Moderator
Joined: 06 Mar 2007 Posts: 21431
|
Posted: Wed Mar 16, 2011 2:00 am Post subject: |
|
|
Copying an NTFS volume using rsync may not produce the desired results. If you need to copy the entire filesystem, use ntfsclone for a filesystem-aware copy or dd for a bit-for-bit copy. NTFS volumes can be resized using ntfsresize if needed. |
|
Back to top |
|
|
d2_racing Bodhisattva
Joined: 25 Apr 2005 Posts: 13047 Location: Ste-Foy,Canada
|
Posted: Wed Mar 16, 2011 2:21 am Post subject: |
|
|
I think that partimage can do the trick too. |
|
Back to top |
|
|
cach0rr0 Bodhisattva
Joined: 13 Nov 2008 Posts: 4123 Location: Houston, Republic of Texas
|
Posted: Wed Mar 16, 2011 4:49 am Post subject: |
|
|
hey neat, ntfsclone even handles junctions (think symlinks) correctly.
trick ive had to use in the past for clients sending me their db backup with the drive hard-coded (MSSQL) _________________ Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash |
|
Back to top |
|
|
|