demerge - emerge the other way around

Message

ian! · Post by **ian!** » Wed Apr 11, 2007 10:25 pm

"Gentoo is about choice", guess you heard about it at least once. That's why most of us give a lot of packages a try and probably also remove some of them afterwards. Most of us most likely will not. When I became a dev I faced one problem testing packages: 'Keeping my installation clean.' As a developer you often install a lot of packages. Lots of them are still in testing. Your day often looks like:

Code: Select all

# emerge $this
($this comes with deps $bar and $baz that also get installed)
# emerge $that
(bump $this and $that version)
# emerge $this
# emerge $that
# emerge $some_deps
# USE="foo" emerge $that

In the end you installed a lot of stuff and your world file grows. - Wouldn't it be nice to revert all your changes with one simple command? - With demerge you can. - Lemme give a short example:

Code: Select all

# emerge demerge
# demerge --record
# emerge gentoo
# demerge --record
# emerge -C DateManip

Calling demerge now gives:

Code: Select all

# demerge
demerge version 0.029

  Use this program carefully - otherwise you might run into problems.
  You are root. You are responsible for your actions.
  Bugs and requests go to ian <ian@gentoo.org>.


  Found previous states:
    1172352506 (2007-02-24 22:28:26)
          -app-misc/gentoo-0.11.55 USE="nls -gnome -fam"
          +dev-perl/DateManip-5.44 USE=""

    1172352564 (2007-02-24 22:29:24)
          +dev-perl/DateManip-5.44 USE=""

  To revert to one of the previous system-states run 'demerge --restore timestamp'.

So if you want to go back to the state where 'gentoo' was not installed but 'DateManip' was type:

Code: Select all

# demerge --restore 1172352506
 ..
 Packages that will be uninstalled:
  app-misc/gentoo-0.11.55 USE="nls -gnome -fam"
  
 Packages that will be installed:
  dev-perl/DateManip-5.44 USE=""

 Proceed? (y/n):

Type 'y' and watch demerge revert to the given system state. - Now type `demerge` again..

Code: Select all

# demerge
 ...
 
 1172352506 (2007-02-24 22:28:26)
  No differences found.
 
 ...

Just what you expected, right? Let's see what happens if we do something different:

Code: Select all

# USE="gnome" emerge gentoo
# demerge
 ...
  1172352564 (2007-02-24 22:29:24)
         -app-misc/gentoo-0.11.55 USE="nls gnome -fam"
         +app-misc/gentoo-0.11.55 USE="nls -gnome -fam"

See? demerge is useflag aware.
Ok great. Now you decided to like demerge and want it to record the system-state after every --sync. You can do that by:

Code: Select all

# chmod +x /etc/portage/postsync.d/demerge-record

Now after a `emerge --sync` the current system-state will be recorded by demerge automatically so that you always have states at hand to revert to.

After some time you will have quite some stuff on your harddisk. That is when the option --wipe-older comes in handy:

Code: Select all

# demerge --wipe-older 1172352506

Executing this will remove all stuff recorded older than the given timestamp.

Enjoy! And don't forget about `man demerge`

purpler · Post by **purpler** » Thu Apr 12, 2007 12:59 am

wowwwww
nice oneeee

ian! · Post by **ian!** » Thu Apr 12, 2007 8:13 pm

Just released demerge-0.040 which will hit the mirrors in just a few.

Changelog wrote:0.040
- Speedups/Caching (in memory)
- New storage format (uses less disk space; melts down ~13MB to 60kb per state)
- Converter which converts old states (version >=0.027) to the new format
- Overall performance: up to 5 times faster than 0.035
- Do not call GetTerminalSize() from Term::ReadKey when --postsync is called.
(Problems occured while testing on ppc/efika. Reported by Wernfried Haas <amne@gentoo.org>)
- Moved configfile to /etc/demerge.conf (converter takes care about moving your configfile to the new location)

Enjoy!

likewhoa · Post by **likewhoa** » Thu Apr 12, 2007 8:29 pm

thanks for this fine package. will come in handy with my bloated test system.

swimmer · Post by **swimmer** » Fri Apr 13, 2007 1:41 am

Will this converter be called automagically with the update? Or do we have to do this manually?

And thanks for the good work!!!

Greetz
swimmer

mudrii · Post by **mudrii** » Fri Apr 13, 2007 2:33 am

ian!

Big Thank you for this one

Regards

ian! · Post by **ian!** » Fri Apr 13, 2007 6:37 am

swimmer wrote:Will this converter be called automagically with the update?

Yes. It will check for old versions and convert them automatically. (Versions recorded with <demerge-0.0.27 cannot be converted btw because these are missing useflag data. The converter will tell you about it in case you have such.)

swimmer · Post by **swimmer** » Fri Apr 13, 2007 8:28 am

Thank you

salam · Post by **salam** » Tue Apr 17, 2007 3:43 pm

does it also record the dependecies which are pulled in with emerging a single package?

ian! · Post by **ian!** » Tue Apr 17, 2007 7:18 pm

salam wrote:does it also record the dependecies which are pulled in with emerging a single package?

Sure.

It records all package changes, which useflags are set (on a per package base) and what packages are recorded in the worldfile and which are not. When restoring a state demerge will take care of restoring all that.

sonicbhoc · Post by **sonicbhoc** » Tue Apr 17, 2007 7:20 pm

Woah. Now that sounds interesting. I might give it a shot if I ever get adventurous.

ian! · Post by **ian!** » Wed Apr 18, 2007 8:11 pm

demerge-0.042 is on the way..

Changelog wrote:0.042
- New option --comment which allows users to leave comments on each state.
- Autocomment on states being recorded before restoring a state.
- Autocomment on states being recorded in postsync hook.
- Option --wipe can now also be used to just delete a single state.

So - just to give a simple example - you now can do something like:

Code: Select all

# demerge --record --comment='Before fixing GLSA 200703-26 [N] file: Integer underflow ( sys-apps/file )'
..
# glsa-check -f 200703-26
..
# demerge
..
 * Found previous states:

   1176926919 (2007-04-18 22:08:39) - Before fixing GLSA 200703-26 [N] file: Integer underflow ( sys-apps/file )
        -sys-apps/file-4.20 USE="python"
        +sys-apps/file-4.19 USE="python"
..

plut0 · Post by **plut0** » Thu Apr 19, 2007 1:44 pm

Curious, why is this needed when depclean is available? Wouldn't it be better to oneshot your dependencies instead of putting them in the world file?

Kalin · Post by **Kalin** » Thu Apr 19, 2007 2:14 pm

Curious, why is this needed when depclean is available?

Well, one example was given above: it is useflag aware.
2nd, it remembers (ian!, thanks for --comment option) states for you, so you can revert several steps at a time.

Wouldn't it be better to oneshot your dependencies instead of putting them in the world file?

AFAIK, nothing is put in the world file, everything is stored in ~/.demerge/

ian! · Post by **ian!** » Thu Apr 19, 2007 4:34 pm

plut0 wrote:Curious, why is this needed when depclean is available?

demerge does way more then depclean does.

Examples:
1.) Before being able to use depclean you would need to know which packages were installed, unmerge them manually and then call depclean.

2.) depclean relies on the dependency information provided by ebuilds. demerge does not. This makes it a lot more safe and sane in cases where ebuilds are missing vital dependency information.

3.) demerge does recall multiple states.

4.) With demerge you simply can go back in time and switch to previous states, revert them again, copy them on other machines and clone installations etc.

demerge was written with a different goal in mind than just being a better --depclean. (See my first post in this thread.)

plut0 wrote:Wouldn't it be better to oneshot your dependencies instead of putting them in the world file?

Only packages that were in world before will be recorded in world when restoring a state.

avx · Post by **avx** » Thu Apr 19, 2007 9:50 pm

Now that's a nice one, thanks! Will this be part of regular emerge at some time?

jesnow · Post by **jesnow** » Thu Apr 19, 2007 10:28 pm

ph030 wrote:Now that's a nice one, thanks! Will this be part of regular emerge at some time?

Second that.

J.

plut0 · Post by **plut0** » Mon Apr 23, 2007 2:54 pm

Another question...If you install application 'abc' that depended on say apache, which was not already installed, take a snapshot after this install. Then you install application 'xyz' that also depended on apache and took another snapshot. Now you want to remove application 'abc' using demerge. Does apache get removed also?

ian! · Post by **ian!** » Mon Apr 23, 2007 8:24 pm

plut0 wrote:Now you want to remove application 'abc' using demerge.

You can't. demerges purpose is to revert to previous recorded states.

demerge != depclean

plut0 · Post by **plut0** » Mon Apr 23, 2007 8:52 pm

Thanks, I got it now

Looks like a great tool!

biatch0 · Post by **biatch0** » Sat Apr 28, 2007 1:46 pm

How does demerge deal with deprecated and/or masked packages?

ian! · Post by **ian!** » Sat Apr 28, 2007 2:49 pm

biatch0 wrote:How does demerge deal with deprecated and/or masked packages?

Currently it does not care about that at all but upcoming versions will.

kfiadeg · Post by **kfiadeg** » Sat Apr 28, 2007 10:12 pm

This is a great piece of good stuff! Thank you!
Hope this will be a part of portage utils soon.

Extintor · Post by **Extintor** » Sat Apr 28, 2007 10:28 pm

Looks promising. Thanks a bunch!

jurrie · Post by **jurrie** » Sun May 06, 2007 10:41 am

Thanks a lot for this. This program has saved me a great deal of pain from reverting back to my 'stable' state after doing some experiments ^_^