Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
migrating to a new HDD
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
Uncle_Psychosis
Guru
Guru


Joined: 31 Jan 2004
Posts: 387

PostPosted: Tue Dec 28, 2010 12:12 am    Post subject: migrating to a new HDD Reply with quote

Hi guys

The time has come to upgrade to a bigger, hopefully faster hard drive.

I have had my current gentoo install for a good five years so I'm a little bit nervous about what I need to do to move to a new hard drive. Do any of you have any good tips for doing so? Should I just follow the install guide again or is there an easy way to copy all my stuff to a new (and not necessarily the same) file system layout?

Also, what file systems would you recommend for / and /home? (I put them on separate partitions so they don't necessarily have to be the same)

Cheers

Sam
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54028
Location: 56N 3W

PostPosted: Tue Dec 28, 2010 12:31 am    Post subject: Reply with quote

Uncle_Psychosis,

Make a mount point for your new drive, say /mnt/new
partition it to your liking and make filesystems. I suggest ext4 but rieser and ext3 work too.

Assemble the partitions into a filesystem tree. under /mnt/new
Copy everything over. This is best done with a liveCD as you must not copy /dev /proc, /sys and there is no point in copying /tmp

You can use rsync, cp -a, tar etc for the copy.

After the copy install grub onto the MBR. What you do next depends what you will do with the old drive.
You may need to change /etc/fstab and grub.conf to get the new drive to boot.

Oh, be sure your kernel supports any new filesystems you have elected to use.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
d2_racing
Bodhisattva
Bodhisattva


Joined: 25 Apr 2005
Posts: 13047
Location: Ste-Foy,Canada

PostPosted: Tue Dec 28, 2010 4:41 am    Post subject: Reply with quote

Hi, I use rsync all the time to backup or migrate my stuff.

I use this command :

Code:

# time rsync -aHA --del --force --stats --progress /mnt/backup/gentoo/ /mnt/gentoo


Where /mnt/backup/gentoo is my old hdd and /mnt/gentoo is my new hdd.

Clean and simple.
Back to top
View user's profile Send private message
Uncle_Psychosis
Guru
Guru


Joined: 31 Jan 2004
Posts: 387

PostPosted: Tue Dec 28, 2010 8:50 am    Post subject: Reply with quote

Ahha...now that seems like the kind of thing I need!

The only problem I can forsee is that I also need to put windows onto the new drive. My plan is to:

1. remove exisiting drives
2. plug in new one
3. partition new drive
4. install windows
5. plug in existing drives again
6. boot from live cd
7. copy my gentoo root folder to the new drive using rsync (thanks for the command)
8. install grub on the new disk and modify fstab as necessary
9. boot into new system
10. sort out the horrible mess that is my existing disks...


Also, I have a question about swap space. How much do you actually need for a modern PC? I've got 8Gb of RAM---do I really need to waste a partition on swap?
Back to top
View user's profile Send private message
XQYZ
Apprentice
Apprentice


Joined: 19 Jul 2009
Posts: 231
Location: Europe

PostPosted: Tue Dec 28, 2010 12:56 pm    Post subject: Reply with quote

Uncle_Psychosis wrote:
Also, I have a question about swap space. How much do you actually need for a modern PC? I've got 8Gb of RAM---do I really need to waste a partition on swap?


Hibernation would be an argument. also: can you really not spare the 8gb on your hard drive for a swap partition?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54028
Location: 56N 3W

PostPosted: Tue Dec 28, 2010 1:06 pm    Post subject: Reply with quote

Uncle_Psychosis,

If you need more than 512Mb for swapping your really don't have enough RAM.
However, swap a little larger than your RAM is needed for hibernation, if thats important to you.

You should have a swap, as not having swap space does not stop the kernel from swapping, you just disable one of the mechanisims for swapping that is available to the kernel.

Without swap, there is no home on disc for dynamically allocated RAM. The kernel will still flush/reload code and data that belongs to a disk file when RAM is short. Thats a form of swapping.
Note that code/data that have a real home in a disk file are never written to swap.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Fitzcarraldo
Advocate
Advocate


Joined: 30 Aug 2008
Posts: 2034
Location: United Kingdom

PostPosted: Tue Dec 28, 2010 4:36 pm    Post subject: Size of swap if you want the PC to be able to hibernate Reply with quote

NeddySeagoon wrote:
However, swap a little larger than your RAM is needed for hibernation, if thats important to you.

I've read in the past that swap can be slightly smaller, not larger, than your RAM if you want Linux to be able to hibernate. The reason for this was stated as being because the contents of RAM are compressed when saved to disk.

Does anyone have a definitive reference for the amount of swap required for hibernation (suspend-to-disk)?
Back to top
View user's profile Send private message
XQYZ
Apprentice
Apprentice


Joined: 19 Jul 2009
Posts: 231
Location: Europe

PostPosted: Tue Dec 28, 2010 5:01 pm    Post subject: Reply with quote

AFAIK at least the default configuration of tuxonice with hibernate-scripts doesn't compress the ram when it's written to disk. I'm not sure compression would make that much sense anyway as it takes quite a bit to do it and most of the time you wouldn't even get a good ratio out of it. Also I don't think you really can give an accurate number for this. I mean the safest bet is make it at least as big as your ram is, so even if it's not compressible at all, it still works. I mean personally I'd hate it if it wouldn't fit at some point just because I'm out of luck.

http://tuxonice.net/ could have some info on it, but apparently you need to register yourself first for wiki access.
Back to top
View user's profile Send private message
Fitzcarraldo
Advocate
Advocate


Joined: 30 Aug 2008
Posts: 2034
Location: United Kingdom

PostPosted: Tue Dec 28, 2010 8:20 pm    Post subject: Size of swap for hibernation Reply with quote

Had a hunt for the post I remembered reading, and found it in KernelTrap: http://kerneltrap.org/node/16317#comment-303704

Quote:
Anonymous (not verified) on June 23, 2008 - 5:10pm
Isn't swap necessary for Suspend To Disk/Hibernate? And if so, how much?

Quote:
Anonymous (not verified) on June 24, 2008 - 5:45am
I think for suspend to disk you need a storage space an large as your ram.

Quote:
Anonymous (not verified) on June 24, 2008 - 12:57pm
suspend-to-disk uses compressed images, so you can do with less swap than RAM (if it compresses well). It also only stores actual program images, so caches are likely to get evicted, the kernel is mostly unloaded, etc.

Is that actually the case?

EDIT (21:03 GMT): Just found this post from TuxOnIce developer Nigel Cunningham (http://lists.tuxonice.net/pipermail/tuxonice-devel/2009-June/005423.html):

Nigel Cunningham wrote:
> * How the installer of a Linux distribution should decide on the size of
> the hibernation file/partition? Worst case seems to be the installed
> memory size + swap partition but given that LZO compresses the image
> well enough is it possible to assume a size for the image file based on
> some proportions or something else?

LZO compression almost always achieves greater than 50% compression, so
you should be able to fit all images in swapspace equal to half the
amount of memory you have. Of course you might want to add a margin for
normal swapping and variations in the compression that's achieved
.

So it seems that making swap the same size as RAM is more than enough.
Back to top
View user's profile Send private message
Fitzcarraldo
Advocate
Advocate


Joined: 30 Aug 2008
Posts: 2034
Location: United Kingdom

PostPosted: Wed Dec 29, 2010 4:40 pm    Post subject: Size of swap required for hibernation Reply with quote

I did a little experiment to see what TuxOnIce does.

Here are my RAM and swap sizes in bytes:
Code:
# free -bt
             total       used       free     shared    buffers     cached
Mem:    4142669824 2399014912 1743654912          0  175312896 1153126400
-/+ buffers/cache: 1070575616 3072094208
Swap:   4293558272          0 4293558272
Total:  8436228096 2399014912 6037213184

My (2.6.36) kernel config was as follows w.r.t. LZO compression:
Code:
$ cat /usr/src/linux/.config | grep LZO
CONFIG_HAVE_KERNEL_LZO=y
# CONFIG_KERNEL_LZO is not set  <---- Kernel Compression Mode was selected to be LZMA rather than LZO
CONFIG_RD_LZO=y
CONFIG_JFFS2_LZO=y
# CONFIG_JFFS2_CMODE_FAVOURLZO is not set
CONFIG_UBIFS_FS_LZO=y
CONFIG_SQUASHFS_LZO=y
CONFIG_CRYPTO_LZO=m
CONFIG_LZO_COMPRESS=m
CONFIG_LZO_DECOMPRESS=y
CONFIG_DECOMPRESS_LZO=y

and TuxOnIce did not work when I tried to hibernate my laptop:
Code:
Dec 29 12:01:06 meshedgedx kernel: [  272.829994] TuxOnIce: Failed to initialise the lzo compression transform.
Dec 29 12:01:06 meshedgedx kernel: [  272.829995] Failed to initialise compression algorithm.
Dec 29 12:01:06 meshedgedx kernel: [  272.829996] Continuing without compressing the image.
.
.
.
Dec 29 12:01:06 meshedgedx kernel: [  316.908327] TuxOnIce debugging info:
Dec 29 12:01:06 meshedgedx kernel: [  316.908330] - TuxOnIce core  : 3.2-rc2
Dec 29 12:01:06 meshedgedx kernel: [  316.908331] - Kernel Version : 2.6.36-sabayon
Dec 29 12:01:06 meshedgedx kernel: [  316.908333] - Compiler vers. : 4.4
Dec 29 12:01:06 meshedgedx kernel: [  316.908335] - Attempt number : 1
Dec 29 12:01:06 meshedgedx kernel: [  316.908336] - Parameters     : 32777 667648 0 0 0 5
Dec 29 12:01:06 meshedgedx kernel: [  316.908338] - Overall expected compression percentage: 0.
Dec 29 12:01:06 meshedgedx kernel: [  316.908340] - Checksum method is 'md4'.
Dec 29 12:01:06 meshedgedx kernel: [  316.908344]   0 pages resaved in atomic copy.
Dec 29 12:01:06 meshedgedx kernel: [  316.908345] - Block I/O active.
Dec 29 12:01:06 meshedgedx kernel: [  316.908346]   Used 0 pages from swap on /dev/sda5.
Dec 29 12:01:06 meshedgedx kernel: [  316.908347] - Max outstanding reads 1052. Max writes 20353.
Dec 29 12:01:06 meshedgedx kernel: [  316.908348]   Memory_needed: 1024 x (4096 + 344 + 112) = 4661248 bytes.
Dec 29 12:01:06 meshedgedx kernel: [  316.908349]   Free mem throttle point reached 0.
Dec 29 12:01:06 meshedgedx kernel: [  316.908350] - Swap Allocator enabled.
Dec 29 12:01:06 meshedgedx kernel: [  316.908351]   Swap available for image: 1048232 pages.
Dec 29 12:01:06 meshedgedx kernel: [  316.908352] - File Allocator active.
Dec 29 12:01:06 meshedgedx kernel: [  316.908352]   Storage available for image: 0 pages.
Dec 29 12:01:06 meshedgedx kernel: [  316.908353] - I/O speed: Write 54180 KB/s.
Dec 29 12:01:06 meshedgedx kernel: [  316.908354] - Extra pages    : 5179 used/500.
Dec 29 12:01:06 meshedgedx kernel: [  316.908355] - Result         : Hibernation was aborted.
Dec 29 12:01:06 meshedgedx kernel: [  316.908356]                  : Insufficient storage was available.
Dec 29 12:01:06 meshedgedx kernel: [  316.908357]                  : I/O errors were encountered.

As you can see above, with LZO compression configured to be a module instead of in-kernel, TuxOnIce did not compress RAM, and the 4293558272 bytes of swap was insufficient for hibernation when I have 4142669824 bytes RAM (of which 2399014912 was in use). So I used the kernel configuration options specified in the Gentoo Wiki article on TuxOnIce, and rebuilt my kernel to perform in-kernel LZO compression:
Code:
$ cat /usr/src/linux/.config | grep LZO
CONFIG_HAVE_KERNEL_LZO=y
# CONFIG_KERNEL_LZO is not set  <---- I left Kernel Compression Mode as LZMA rather than LZO
CONFIG_RD_LZO=y
CONFIG_JFFS2_LZO=y
# CONFIG_JFFS2_CMODE_FAVOURLZO is not set
CONFIG_UBIFS_FS_LZO=y
CONFIG_SQUASHFS_LZO=y
CONFIG_CRYPTO_LZO=y
CONFIG_LZO_COMPRESS=y
CONFIG_LZO_DECOMPRESS=y
CONFIG_DECOMPRESS_LZO=y

and now the laptop hibernates successfully:
Code:
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711101] TuxOnIce debugging info:
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711102] - TuxOnIce core  : 3.2-rc2
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711103] - Kernel Version : 2.6.36-sabayon
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711103] - Compiler vers. : 4.4
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711104] - Attempt number : 1
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711104] - Parameters     : 0 667648 0 0 0 5
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711105] - Overall expected compression percentage: 0.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711105] - Checksum method is 'md4'.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711106]   0 pages resaved in atomic copy.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711107] - Compressor is 'lzo'.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711107]   Compressed 2180071424 bytes into 914681193 (58 percent compression).
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711108] - Block I/O active.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711108]   Used 225238 pages from swap on /dev/sda5.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711109] - Max outstanding reads 1338. Max writes 10377.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711110]   Memory_needed: 1024 x (4096 + 344 + 112) = 4661248 bytes.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711110]   Free mem throttle point reached 0.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711111] - Swap Allocator enabled.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711112]   Swap available for image: 1048232 pages.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711112] - File Allocator active.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711113]   Storage available for image: 0 pages.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711113] - I/O speed: Write 130 MB/s, Read 153 MB/s.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711114] - Extra pages    : 5214 used/500.
Dec 29 15:22:45 meshedgedx kernel: [ 1205.711115] - Result         : Succeeded.

If I understand the above TuxOnIce output correctly, with in-kernel LZO compression enabled my hibernating laptop with 3950 KB RAM occupied 225238/1048232 pages of swap = 22 per cent of my laptop's 4094 KB swap. The amount of swap used for hibernation was 225238x4096/4142669824 = 23 per cent of my laptop's RAM.

So, making the swap size the same as the RAM size is more than enough. There is certainly no need to make the swap size larger than the RAM size, and one could get away with making the swap size smaller than the RAM size.
Back to top
View user's profile Send private message
Uncle_Psychosis
Guru
Guru


Joined: 31 Jan 2004
Posts: 387

PostPosted: Thu Jan 27, 2011 5:47 pm    Post subject: Reply with quote

d2_racing wrote:
Hi, I use rsync all the time to backup or migrate my stuff.

I use this command :

Code:

# time rsync -aHA --del --force --stats --progress /mnt/backup/gentoo/ /mnt/gentoo


Where /mnt/backup/gentoo is my old hdd and /mnt/gentoo is my new hdd.

Clean and simple.


Right, I'm finally getting round to doing this. Will your command copy everything thats under / ?

Thanks

Sam
Back to top
View user's profile Send private message
d2_racing
Bodhisattva
Bodhisattva


Joined: 25 Apr 2005
Posts: 13047
Location: Ste-Foy,Canada

PostPosted: Sun Jan 30, 2011 4:53 am    Post subject: Reply with quote

Yes :)
Back to top
View user's profile Send private message
Uncle_Psychosis
Guru
Guru


Joined: 31 Jan 2004
Posts: 387

PostPosted: Sun Mar 13, 2011 3:40 pm    Post subject: Reply with quote

Right, I've finally found the time to do this.

d2_racing, I'm going to use your command to copy my / partition to the new drive.

However, I currently have /home on its own partition and I dont want to back that up too---I want to back that up separately (plan is to partition the new drive the same as the old one, only each partition will be bigger)

Thanks

Sam
Back to top
View user's profile Send private message
d2_racing
Bodhisattva
Bodhisattva


Joined: 25 Apr 2005
Posts: 13047
Location: Ste-Foy,Canada

PostPosted: Sun Mar 13, 2011 4:04 pm    Post subject: Reply with quote

Use the switch exclude then :

--exclude /home'
Back to top
View user's profile Send private message
davidm
Guru
Guru


Joined: 26 Apr 2009
Posts: 557
Location: US

PostPosted: Sun Mar 13, 2011 4:06 pm    Post subject: Re: Size of swap for hibernation Reply with quote

Fitzcarraldo wrote:
Had a hunt for the post I remembered reading, and found it in KernelTrap: http://kerneltrap.org/node/16317#comment-303704
So it seems that making swap the same size as RAM is more than enough.


Also see:

http://www.mjmwired.net/kernel/Documentation/power/interface.txt

Quote:

50 /sys/power/image_size controls the size of the image created by
51 the suspend-to-disk mechanism. It can be written a string
52 representing a non-negative integer that will be used as an upper
53 limit of the image size, in bytes. The suspend-to-disk mechanism will
54 do its best to ensure the image size will not exceed that number. However,
55 if this turns out to be impossible, it will try to suspend anyway using the
56 smallest image possible. In particular, if "0" is written to this file, the
57 suspend image will be as small as possible.
58
59 Reading from this file will display the current image size limit, which
60 is set to 500 MB by default.
61


Note: On mine it actually seems to be set to "847683584" by default so I think the default might have changed or there is some other process at work which I am not aware of that sets it. My RAM is 2 GB and I have also created a 2 GB swap.
Back to top
View user's profile Send private message
Uncle_Psychosis
Guru
Guru


Joined: 31 Jan 2004
Posts: 387

PostPosted: Tue Mar 15, 2011 9:40 pm    Post subject: Reply with quote

Just out of interest... can I do this with a windows partition? Or does windows actually need to be installed "properly"?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21431

PostPosted: Wed Mar 16, 2011 2:00 am    Post subject: Reply with quote

Copying an NTFS volume using rsync may not produce the desired results. If you need to copy the entire filesystem, use ntfsclone for a filesystem-aware copy or dd for a bit-for-bit copy. NTFS volumes can be resized using ntfsresize if needed.
Back to top
View user's profile Send private message
d2_racing
Bodhisattva
Bodhisattva


Joined: 25 Apr 2005
Posts: 13047
Location: Ste-Foy,Canada

PostPosted: Wed Mar 16, 2011 2:21 am    Post subject: Reply with quote

I think that partimage can do the trick too.
Back to top
View user's profile Send private message
cach0rr0
Bodhisattva
Bodhisattva


Joined: 13 Nov 2008
Posts: 4123
Location: Houston, Republic of Texas

PostPosted: Wed Mar 16, 2011 4:49 am    Post subject: Reply with quote

hey neat, ntfsclone even handles junctions (think symlinks) correctly.
trick ive had to use in the past for clients sending me their db backup with the drive hard-coded (MSSQL)
_________________
Lost configuring your system?
dump lspci -n here | see Pappy's guide | Link Stash
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum