Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
update svn/git packages script
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Sun Feb 10, 2008 2:45 pm    Post subject: update svn/git packages script Reply with quote

Hi!

Here is a new version of my updatekde script.

It has now a config file which is located under /etc/updatesvn/updatesvn.conf.
There you will fill in all the packages which you would like to check for updates.
They must be exactly named like what you see in /usr/portage/distfiles/svn-src/.

The sub-directorie KDE will be regarded, so just put the names without directory.

Please feel free to improve it.

Tobi

UPDATED: Sun Mar 9 16:46:00 CET 2008

ToDo:

- First safe packages which should be emerged (binpackages)
- Writing an ebuild

updatesvn:
http://finswimmer.de/gentoo/updatesvn

/etc/updatesvn/updatesvn.conf
http://finswimmer.de/gentoo/updatesvn.conf

UPDATED
http://finswimmer.de/gentoo/UPDATED
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke


Last edited by Finswimmer on Sun Mar 09, 2008 3:46 pm; edited 18 times in total
Back to top
View user's profile Send private message
LXj
Tux's lil' helper
Tux's lil' helper


Joined: 03 Dec 2005
Posts: 107

PostPosted: Mon Feb 11, 2008 9:51 am    Post subject: Reply with quote

Thanks for the scripts! I didn't test it yet, but I already have feature requests :)

1. Is it possible to add a mode to update all SVN packages? I also use berkano overlay to install mplayer and x264, and I think it would be a good idea to update all my packages, without setting up $CHECK
2. Some packages in portage/overlays are actually using CVS (emacs-cvs) or GIT (wine). I don't actually use them, but I think their users would be grateful if you add support of these packages
3. SVN can be changed at any time, and sometimes these changes can brake things. So I want to have a quick way to revert to previous version. And portage has the feature to do it — binary packages! So what should be added to your script is...
Code:
DATE=`date +"%m%d"`
mkdir $PKGDIRROOT/$DATE
export PKGDIR="$PKGDIRROOT/$DATE"
export FEATURES="buildpkg"
...

...so that new (maybe broken) binary packages with the same package version (from emerge's POV) will not replace old ones. Now of course it would be neat to have a set of scripts to prune old binary packages and to install set of binary packages from set date :)
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Mon Feb 11, 2008 11:33 pm    Post subject: Reply with quote

LXj wrote:
Thanks for the scripts! I didn't test it yet, but I already have feature requests :)

1. Is it possible to add a mode to update all SVN packages? I also use berkano overlay to install mplayer and x264, and I think it would be a good idea to update all my packages, without setting up $CHECK

Done. Just add "all" in $CHECK.
LXj wrote:

2. Some packages in portage/overlays are actually using CVS (emacs-cvs) or GIT (wine). I don't actually use them, but I think their users would be grateful if you add support of these packages

GIT is working - please test. CVS work in progress ;)
LXj wrote:

3. SVN can be changed at any time, and sometimes these changes can brake things. So I want to have a quick way to revert to previous version. And portage has the feature to do it — binary packages! So what should be added to your script is...
Code:
DATE=`date +"%m%d"`
mkdir $PKGDIRROOT/$DATE
export PKGDIR="$PKGDIRROOT/$DATE"
export FEATURES="buildpkg"
...

...so that new (maybe broken) binary packages with the same package version (from emerge's POV) will not replace old ones. Now of course it would be neat to have a set of scripts to prune old binary packages and to install set of binary packages from set date :)


On Todo list.

Tobi
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
hron
n00b
n00b


Joined: 04 May 2006
Posts: 65
Location: Budapest, Hungary

PostPosted: Tue Feb 12, 2008 11:36 am    Post subject: Reply with quote

Why don't ask an exactly distdir folder name?

Code:

# ....
DISTDIR="$(portageq distdir)"
svn --quiet st -u "${DISTDIR}/svn-src/$line/$line" |sed '/^$/d'|egrep -v "external|Status against revision| working copy|X" |tee  /tmp/svn-tmp
# ...

_________________
Garami Gabor Gyorgy
translator
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Tue Feb 12, 2008 8:57 pm    Post subject: Reply with quote

hron wrote:
Why don't ask an exactly distdir folder name?

Code:

# ....
DISTDIR="$(portageq distdir)"
svn --quiet st -u "${DISTDIR}/svn-src/$line/$line" |sed '/^$/d'|egrep -v "external|Status against revision| working copy|X" |tee  /tmp/svn-tmp
# ...


Done :)
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
drizzt
Guru
Guru


Joined: 21 Jul 2002
Posts: 424

PostPosted: Wed Feb 13, 2008 7:54 am    Post subject: Reply with quote

Doesn't work.
Code:

sh ./updatesvn.sh
kdelibs
kdepimlibs
kdebase
kdegraphics
kdeadmin
kdeaccessibility
kdeartwork
kdenetwork
kdemultimedia
kdesdk
kdepim
kdeutils
extragear-plasma
./updatesvn.sh: line 62: cd: /usr/portage/distfiles/svn-src/KDE/extragear-plasma/extragear-plasma: Datei oder Verzeichnis nicht gefunden
kdelibs
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdelibs/kdelibs: Datei oder Verzeichnis nicht gefunden
kdepimlibs
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdepimlibs/kdepimlibs: Datei oder Verzeichnis nicht gefunden
kdebase
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdebase/kdebase: Datei oder Verzeichnis nicht gefunden
kdegraphics
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdegraphics/kdegraphics: Datei oder Verzeichnis nicht gefunden
kdeadmin
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdeadmin/kdeadmin: Datei oder Verzeichnis nicht gefunden


Why is KDE discarded ?

updatesvn.conf:
Code:

##Emerge Command? Useful if you use nightmerge etc.
EMCMD="emerge"
##Subdirectories like KDE go here:
SUB="KDE"
##Which packages should be updated?
CHECK="kdelibs
kdepimlibs
kdebase
kdegraphics
kdeadmin
kdeaccessibility
kdeartwork
kdenetwork
kdemultimedia
kdesdk
kdepim
kdeutils
extragear-plasma"
##If svn-directory differs from ebuild packagen name, please specifiy: /Svn-directory/ebuild-name/
##seperated by space
DIFFER="/extragear/extragear-plasma/"
##Specify which packages MUST be emerged before all others. Obey the correct order in this variable!
BEFORE="strigi soprano kdelibs kdepimlibs kdebase"


What am I doing wrong here ?
_________________
People don't have to earn my respect. I offer my respect to them, but be careful to lose my respect...
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Wed Feb 13, 2008 8:04 am    Post subject: Reply with quote

drizzt wrote:
Doesn't work.
Code:

sh ./updatesvn.sh
kdelibs
kdepimlibs
kdebase
kdegraphics
kdeadmin
kdeaccessibility
kdeartwork
kdenetwork
kdemultimedia
kdesdk
kdepim
kdeutils
extragear-plasma
./updatesvn.sh: line 62: cd: /usr/portage/distfiles/svn-src/KDE/extragear-plasma/extragear-plasma: Datei oder Verzeichnis nicht gefunden
kdelibs
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdelibs/kdelibs: Datei oder Verzeichnis nicht gefunden
kdepimlibs
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdepimlibs/kdepimlibs: Datei oder Verzeichnis nicht gefunden
kdebase
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdebase/kdebase: Datei oder Verzeichnis nicht gefunden
kdegraphics
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdegraphics/kdegraphics: Datei oder Verzeichnis nicht gefunden
kdeadmin
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdeadmin/kdeadmin: Datei oder Verzeichnis nicht gefunden


Why is KDE discarded ?

updatesvn.conf:
Code:

##Emerge Command? Useful if you use nightmerge etc.
EMCMD="emerge"
##Subdirectories like KDE go here:
SUB="KDE"
##Which packages should be updated?
CHECK="kdelibs
kdepimlibs
kdebase
kdegraphics
kdeadmin
kdeaccessibility
kdeartwork
kdenetwork
kdemultimedia
kdesdk
kdepim
kdeutils
extragear-plasma"
##If svn-directory differs from ebuild packagen name, please specifiy: /Svn-directory/ebuild-name/
##seperated by space
DIFFER="/extragear/extragear-plasma/"
##Specify which packages MUST be emerged before all others. Obey the correct order in this variable!
BEFORE="strigi soprano kdelibs kdepimlibs kdebase"


What am I doing wrong here ?


It only works if you have already installed the packages. Maybe teher is no /usr/portage/distfiles/svn-src/kdelibs/kdelibs?
Please print ls /usr/portage/distfiles/svn-src/

Tobi
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
drizzt
Guru
Guru


Joined: 21 Jul 2002
Posts: 424

PostPosted: Wed Feb 13, 2008 8:30 am    Post subject: Reply with quote

I'm writing from inside KDE-SVN :wink:

Code:

AMD64 drizzt # ls -l /usr/portage/distfiles/svn-src/
insgesamt 2
drwxr-sr-x  2 root portage 1024 11. Dez 17:50 einit
drwxr-sr-x 18 root portage 1024  8. Feb 17:22 KDE

Code:

AMD64 drizzt # ls -l /usr/portage/distfiles/svn-src/KDE/
insgesamt 15
drwxr-sr-x 4 root portage 1024 20. Jan 17:17 extragear
drwxr-sr-x 3 root portage 1024 19. Jan 18:00 kdeaccessibility
drwxr-sr-x 3 root portage 1024 19. Jan 17:23 kdeadmin
drwxr-sr-x 3 root portage 1024 19. Jan 17:26 kdeartwork
drwxr-sr-x 3 root portage 1024 19. Jan 02:06 kdebase
drwxr-sr-x 3 root portage 1024 19. Jan 17:40 kdegraphics
drwxr-sr-x 3 root portage 1024 18. Jan 22:17 kdelibs
drwxr-sr-x 3 root portage 1024 26. Jan 10:55 kdemultimedia
drwxr-sr-x 3 root portage 1024 19. Jan 18:05 kdenetwork
drwxr-sr-x 3 root portage 1024 19. Jan 19:30 kdepim
drwxr-sr-x 3 root portage 1024 19. Jan 01:57 kdepimlibs
drwxr-sr-x 3 root portage 1024 20. Jan 10:25 kdesdk
drwxr-sr-x 3 root portage 1024 19. Jan 20:39 kdeutils
drwxr-sr-x 3 root portage 1024  8. Feb 17:22 kdevelop
drwxr-sr-x 3 root portage 1024  8. Feb 17:16 kdevplatform

Code:

---snip---
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdelibs/kdelibs: Datei oder Verzeichnis nicht gefunden
---snip---

As far as I see the script discards the "KDE" subdirectory at some level, but I have no clue why...
_________________
People don't have to earn my respect. I offer my respect to them, but be careful to lose my respect...
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Wed Feb 13, 2008 6:03 pm    Post subject: Reply with quote

In updatesvn.conf is variable called SUB: SUB="kde"
or
CHECK="all "
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
drizzt
Guru
Guru


Joined: 21 Jul 2002
Posts: 424

PostPosted: Wed Feb 13, 2008 7:10 pm    Post subject: Reply with quote

drizzt wrote:
Doesn't work.
Code:

sh ./updatesvn.sh
kdelibs
kdepimlibs
kdebase
kdegraphics
kdeadmin
kdeaccessibility
kdeartwork
kdenetwork
kdemultimedia
kdesdk
kdepim
kdeutils
extragear-plasma
./updatesvn.sh: line 62: cd: /usr/portage/distfiles/svn-src/KDE/extragear-plasma/extragear-plasma: Datei oder Verzeichnis nicht gefunden
kdelibs
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdelibs/kdelibs: Datei oder Verzeichnis nicht gefunden
kdepimlibs
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdepimlibs/kdepimlibs: Datei oder Verzeichnis nicht gefunden
kdebase
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdebase/kdebase: Datei oder Verzeichnis nicht gefunden
kdegraphics
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdegraphics/kdegraphics: Datei oder Verzeichnis nicht gefunden
kdeadmin
./updatesvn.sh: line 78: cd: /usr/portage/distfiles/svn-src/kdeadmin/kdeadmin: Datei oder Verzeichnis nicht gefunden


Why is KDE discarded ?

updatesvn.conf:
Code:

##Emerge Command? Useful if you use nightmerge etc.
EMCMD="emerge"
##Subdirectories like KDE go here:
SUB="KDE"
##Which packages should be updated?
CHECK="kdelibs
kdepimlibs
kdebase
kdegraphics
kdeadmin
kdeaccessibility
kdeartwork
kdenetwork
kdemultimedia
kdesdk
kdepim
kdeutils
extragear-plasma"
##If svn-directory differs from ebuild packagen name, please specifiy: /Svn-directory/ebuild-name/
##seperated by space
DIFFER="/extragear/extragear-plasma/"
##Specify which packages MUST be emerged before all others. Obey the correct order in this variable!
BEFORE="strigi soprano kdelibs kdepimlibs kdebase"


What am I doing wrong here ?


As you can see in my updatesvn.conf the subdir-Variable is set and as the first two errors from the script shows used for the first two "cd"s but not afterwards....
_________________
People don't have to earn my respect. I offer my respect to them, but be careful to lose my respect...
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Wed Feb 13, 2008 9:29 pm    Post subject: Reply with quote

Does it work at all?

The script is checking in subdirectory KDE and in svn-src for kdelibs...
So, if it wants to emerge something all should be fine.

I will add a check if the directory exists...

Tobi
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
drizzt
Guru
Guru


Joined: 21 Jul 2002
Posts: 424

PostPosted: Wed Feb 13, 2008 10:07 pm    Post subject: Reply with quote

Ah, ok. I tried to understand the script and now this makes sense.
But it didn't emerge anything :(
I'll try again in one or two days as I compiled kde manually today...
_________________
People don't have to earn my respect. I offer my respect to them, but be careful to lose my respect...
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Thu Feb 14, 2008 10:04 pm    Post subject: Reply with quote

UPDATE:

1)
Now you have a new option to specify the packages you want do emerge.

./updatesvn all --> checks all svn and git packages
./update KDE --> checks all packages specified in updatesvn.conf:

updatesvn.conf wrote:

KDE="kdelibs\
kdebase"

Note that $1 must be *exactly* written like the variable in updatesvn.conf.
Example: KDE="kdelibs" and ./updatesvn kde
will not work

2)
Maybe you have packages that you want to emerge everytime before other packages are emerged.
These belong in variable $EVERY.

Please test it.

Tobi
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Fri Feb 15, 2008 7:30 am    Post subject: Reply with quote

1) Right now: Bugfixed

2) If $2 is "rebuild" in combination with a set of package defined in conf the script will rebuild these packages without check.

Tobi
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
drizzt
Guru
Guru


Joined: 21 Jul 2002
Posts: 424

PostPosted: Fri Feb 15, 2008 7:59 am    Post subject: Reply with quote

Two things to keep attention to:

    a)BEFORE variable is still needed in config file (throws error without it)
    b)all variables must be written in one line with space separation

because of b) I think the script failed on my workstation before
_________________
People don't have to earn my respect. I offer my respect to them, but be careful to lose my respect...
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Fri Feb 15, 2008 5:40 pm    Post subject: Reply with quote

drizzt wrote:
Two things to keep attention to:

    a)BEFORE variable is still needed in config file (throws error without it)
    b)all variables must be written in one line with space separation

because of b) I think the script failed on my workstation before


b) is just like make.conf.
You can use this:

KDE="kdelibs\
kdebase\
kdepim"

a) I will fix this.

Tobi
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Sat Feb 16, 2008 2:28 pm    Post subject: Reply with quote

Update:

Code:
updatesvn all rebuild
rebuilds now all available svn/git packages, in correct order.

Tobi
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
LXj
Tux's lil' helper
Tux's lil' helper


Joined: 03 Dec 2005
Posts: 107

PostPosted: Sat Mar 08, 2008 7:42 am    Post subject: Reply with quote

I have a problem with x264-svn package from Berkano overlay: for some reason updatesvn searches for x264 package, not x264-svn.

Also is it possible to skip packages which are no longer installed? (Though of course deleting their directories in /usr/portage/distfiles/svn-src is good idea anyway)
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Sat Mar 08, 2008 10:07 am    Post subject: Reply with quote

LXj wrote:
I have a problem with x264-svn package from Berkano overlay: for some reason updatesvn searches for x264 package, not x264-svn.

There is something in the conf:
Code:
##If svn-directory differs from ebuild packagen name, please specifiy: /Svn-directory/ebuild-name/
##seperated by space

DIFFER="/extragear/extragear-plasma/"

So, for your problem it should be:
Code:

DIFFER="/extragear/extragear-plasma/
/x264/x264-svn/"

LXj wrote:

Also is it possible to skip packages which are no longer installed? (Though of course deleting their directories in /usr/portage/distfiles/svn-src is good idea anyway)


I will add an option.

Tobi
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Sun Mar 09, 2008 2:29 pm    Post subject: Reply with quote

- Option to skip first preview of packages which should be emerged (useful for automatic updates)

Done.
In updatesvn.conf:

Code:
 
##Should all packages be displayed before real emerge?
DISP="true"


If set to false, the script will directly start emerging.

Tobi
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
Finswimmer
Moderator
Moderator


Joined: 02 Sep 2004
Posts: 5467
Location: Langen (Hessen), Germany

PostPosted: Sun Mar 09, 2008 3:45 pm    Post subject: Reply with quote

Quote:
- Re-emerge only installed packages (optional)


Done.
updatesvn.conf:

Code:
##Set INS to true, to re-emerge only packages that are already installed
INS="true"


Tobi
_________________
Bitte auf Rechtschreibung, korrekte Formatierung und Höflichkeit achten!
Danke
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum