Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
updatoo (new version 0.4)
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
asmith
n00b
n00b


Joined: 29 Jun 2008
Posts: 14
Location: Brazil

PostPosted: Fri Dec 05, 2008 12:35 am    Post subject: updatoo (new version 0.4) Reply with quote

updatoo is a bash script that performing a simple full (silent if you want) update in a Gentoo System.
By default updatoo will synchronize your portage tree with eix-sync, check if your system is update and for bad packages, create a pretend list of packages, try to install all the packages from the pretend list, clean up the system and run revdep-rebuild command.
If occur any problem updatoo will abort with code 1 so you can combine with && or || operator.
Everything is loged in /root/.updatoo/ where you can check anytime.
Please report any bug to ricardo.iramar@gmail.com or https://forums.gentoo.org/viewtopic-t-717092.html .

ChangeLog
Version 0.1 (29/11/2008)
- First version. No bugs yet.

Version 0.2 (07/12/2008)
- Gentoo look output (/etc/init.d/functions.sh").
- Some code fix.

Version 0.3 (02/02/2009)
- Code improvement.
- Changed force mode by ask mode.
- Multiple independent options.
- Error check.
- Detect masked packages.
- Prepare mode added.

Version 0.4 (15/03/2009)
- Parallel fetch.

ToDo
- None.

Code:

#!/bin/bash
#
# updatoo
# Agent Smith (Ricardo Iramar dos Santos)
# ricardo.iramar@gmail.com
#
# updatoo is a bash script that performing a simple full (silent if you want) update in a Gentoo System.
# By default updatoo will synchronize your portage tree with eix-sync, check if your system is update and for bad packages, create a pretend list of packages, try to install all the packages from the pretend list, clean up the system and run revdep-rebuild command.
# If occur any problem updatoo will abort with code 1 so you can combine with && or || operator.
# Everything is loged in /root/.updatoo/ where you can check anytime.
# Please report any bug to ricardo.iramar@gmail.com or http://forums.gentoo.org/viewtopic-t-717092.html.
#
# ChangeLog
#   Version 0.1 (29/11/2008)
#   - First version. No bugs yet.
#
#   Version 0.2 (07/12/2008)
#   - Gentoo look output (/etc/init.d/functions.sh").
#   - Some code fix.
#
#   Version 0.3 (02/02/2009)
#   - Code improvement.
#   - Changed force mode by ask mode.
#   - Multiple independent options.
#   - Error check.
#   - Detect masked packages.
#   - Prepare mode added.
#
#   Version 0.4 (15/03/2009)
#   - Parallel fetch.
#
# ToDo
#   - None.
#

### Variables ###
StrVersion="v0.4"
StrHomeDir="$HOME/.updatoo"
StrWorkDir="$StrHomeDir/`date +%F`"
StrFuncFile="/etc/init.d/functions.sh"
StrSyncCmd="/usr/bin/eix-sync"
StrEmergeWorld="/usr/bin/emerge --verbose --pretend --update --deep --newuse world"
StrEmergeFetch="/usr/bin/emerge --fetchonly --nodeps"
StrEmergeOnePkg="/usr/bin/emerge --oneshot --nodeps"
StrEmergeDepClean="/usr/bin/emerge --depclean"
StrEclean="/usr/bin/eclean distfiles"
StrRmRfClean="rm -rf /var/tmp/portage/*"
StrRevdep="/usr/bin/revdep-rebuild"
### End Variables ###

### Begin SubHelp ###
SubHelp()
{
echo "updatoo [ options ]
Options:
--help (-h)      Print this help.
--ask (-a)      Ask me to confirm each step.
--sync (-s)      Synchronize the portage tree with eix-sync.
--prepare (-p)      Don't emerge anything, only create the lists and check for bad packages.
--fetch (-f)      Fetch the packages from the last list of the day in parallel.
--execute (-e)      Don't create the lists, only update the system using the last lists of the day.
--clean (-c)      Clean up the entire system.
--revdep (-r)      Run revdep-rebuild command after all.
You can combine the options.
The default operation is the same that \"updatoo -spfecr\"."
}
### End SubHelp ###

### Begin SubAbort ###
SubAbort()
{
   eend "Aborted!"
   exit 1
}
### End SubAbort ###

### Begin Script ###
if [ ! -f "$StrFuncFile" ]
then
   echo "Could not found $StrFuncFile. Please emerge baselayout first."
   echo "Aborted!"
   exit 1
fi
source "$StrFuncFile"
if [ "$USER" != "root" ]
then
   ewarn "You need to be root to run updatoo."
   SubAbort
elif [ ! -f "/usr/bin/eix" ]
then
   ewarn "Please install eix first (emerge eix)."
   SubAbort
elif [ ! -f "/usr/bin/revdep-rebuild" ]
then
   ewarn "Please emerge gentoolkit first (emerge gentoolkit)."
   SubAbort
fi

if [[ -z "$@" || "$@" =~ ^(-a|--ask)$ ]]
then
   StrDefault="Y"
   if [[ "$@" =~ ^(-a|--ask)$ ]]; then StrAsk="Y"; fi
else
   for StrOpt in "$@"
   do
      if [[ "$StrOpt" =~ ^-([^-].*)$ ]]
      then
         for StrOpt in `echo ${BASH_REMATCH[1]} | sed 's/\(.\)/\1 /g'`
         do
            if [ "$StrOpt" = "h" -a "$StrHelp" != "Y" -a "$StrAsk" != "Y" -a "$StrPrepare" != "Y" -a "$StrFetch" != "Y" -a "$StrExecute" != "Y" ]
            then
               StrHelp="Y"
            elif [ "$StrOpt" = "a" -a "$StrAsk" != "Y" -a "$StrHelp" != "Y" ]
            then
               StrAsk="Y"
            elif [ "$StrOpt" = "s" -a "$StrSync" != "Y" -a "$StrHelp" != "Y" ]
            then
               StrSync="Y"
            elif [ "$StrOpt" = "p" -a "$StrPrepare" != "Y" -a "$StrHelp" != "Y" ]
            then
               StrPrepare="Y"
            elif [ "$StrOpt" = "f" -a "$StrFetch" != "Y" -a "$StrHelp" != "Y" ]
            then
               StrFetch="Y"
            elif [ "$StrOpt" = "e" -a "$StrExecute" != "Y" -a "$StrHelp" != "Y" ]
            then
               StrExecute="Y"
            elif [ "$StrOpt" = "c" -a "$StrClean" != "Y" -a "$StrHelp" != "Y" ]
            then
               StrClean="Y"
            elif [ "$StrOpt" = "r" -a "$StrRev" != "Y" -a "$StrHelp" != "Y" ]
            then
               StrRev="Y"
            else
               eerror "Invalid option!"
               SubHelp
               exit 1
            fi
         done
      elif [[ "$StrOpt" =~ ^--([^-].*)$ ]]
      then
         if [ "${BASH_REMATCH[1]}" = "help" -a "$StrHelp" != "Y" -a "$StrAsk" != "Y" -a "$StrPrepare" != "Y" -a "$StrFetch" != "Y" -a "$StrExecute" != "Y" ]
         then
            StrHelp="Y"
         elif [ "${BASH_REMATCH[1]}" = "ask" -a "$StrAsk" != "Y" -a "$StrHelp" != "Y" ]
         then
            StrAsk="Y"
         elif [ "${BASH_REMATCH[1]}" = "sync" -a "$StrSync" != "Y" -a "$StrHelp" != "Y" ]
         then
            StrSync="Y"
         elif [ "${BASH_REMATCH[1]}" = "prepare" -a "$StrPrepare" != "Y" -a "$StrHelp" != "Y" ]
         then
            StrPrepare="Y"
         elif [ "${BASH_REMATCH[1]}" = "fetch" -a "$StrFetch" != "Y" -a "$StrHelp" != "Y" ]
         then
            StrFetch="Y"
         elif [ "${BASH_REMATCH[1]}" = "execute" -a "$StrExecute" != "Y" -a "$StrHelp" != "Y" ]
         then
            StrExecute="Y"
         elif [ "${BASH_REMATCH[1]}" = "clean" -a "$StrClean" != "Y" -a "$StrHelp" != "Y" ]
         then
            StrClean="Y"
         elif [ "${BASH_REMATCH[1]}" = "revdep" -a "$StrRev" != "Y" -a "$StrHelp" != "Y" ]
         then
            StrRev="Y"
         else
            eerror "Invalid option!"
            SubHelp
            exit 1
         fi

      else
         eerror "Invalid option!"
         SubHelp
         exit 1
      fi
   done
fi

if [ "$StrHelp" = "Y" ]
then
   SubHelp
   exit 0
fi

if [ ! -d "$StrHomeDir" ]; then mkdir "$StrHomeDir"; fi
if [ ! -d "$StrWorkDir" ]; then mkdir "$StrWorkDir"; fi
StrError="N"

if [ "$StrSync" = "Y" -o "$StrDefault" = "Y" ]
then
   StrAnswer=""
   if [ -e "$StrWorkDir/eix-sync.log" -a "$StrAsk" = "Y" ]
   then
      ewarn "The portage tree has already synchronized today."
      ewarn "Would you like to synchronize again with eix-sync? (Y/n)"
      read StrAnswer
   fi
   if [[ "$StrAnswer" =~ ^([yY]([eE][sS])?)?$ ]]
   then
      ebegin "Synchronizing the portage tree with eix-sync"
      $StrSyncCmd &> "$StrWorkDir/eix-sync.log"
      if [ "$?" -eq 0 ]
      then
         eend
      else
         eerror "Failed, please fix the errors describe in $StrWorkDir/eix-sync.log first and run updantoo late."
         SubAbort
      fi
   else
      ewarn "The portage tree was not synchronized."
   fi
fi

if [ "$StrPrepare" = "Y" -o "$StrDefault" = "Y" ]
then
   StrAnswer=""
   if [ -e "$StrWorkDir/emerge_world.out" -a -e "$StrWorkDir/blocked.lst" -a -e "$StrWorkDir/fetched.lst" -a -e "$StrWorkDir/masked.lst" -a -e "$StrWorkDir/pretend.lst" -a "$StrAsk" = "Y" ]
   then
      ewarn "updatoo has alredy prepared today."
      ewarn "Would you like to prepare again? (Y/n)"
      read StrAnswer
   fi
   if [[ "$StrAnswer" =~ ^([yY]([eE][sS])?)?$ ]]
   then
      einfo "Preparing your system"
      eindent
      ebegin "Checking if your system is already updated"
      $StrEmergeWorld &> "$StrWorkDir/emerge_world.out"
      if [ "`tail -1 $StrWorkDir/emerge_world.out`" = "Total: 0 packages, Size of downloads: 0 kB" ]
      then
         einfo "Your system is alredy updated!"
         eoutdent
         einfo "Finished successfully!"
         exit 0
      else
         ebegin "Checking for blocks packages"
         grep '^\[blocks' "$StrWorkDir/emerge_world.out" &> "$StrWorkDir/blocked.lst"
         if [ -s "$StrWorkDir/blocked.lst" ]
         then
            eerror "There are the follows blocks packages."
            cat "$StrWorkDir/blocked.lst"
            eerror "Please fix them first and run updantoo late."
            eoutdent
            SubAbort
         else
            eend
         fi
         ebegin "Checking for fetched packages"
         grep '^\[ebuild[^]F]*F' "$StrWorkDir/emerge_world.out" &> "$StrWorkDir/fetched.lst"
         if [ -s "$StrWorkDir/fetched.lst" ]
         then
            eerror "There are the follows fetch packages."
            cat "$StrWorkDir/fetched.lst"
            eerror "Please download them manually first and run updantoo late."
            einfo "Trying to emerge these packages in order to get the download URL."
            while read -r StrLine
            do
               StrPackage="`echo $StrLine | sed -e 's/^[^]]*\] //g' -e 's/ .*$//g'`"
               $StrEmergeOnePkg "=$StrPackage"
            done < "$StrWorkDir/fetched.lst"
            eoutdent
            SubAbort
         else
            eend
         fi
         ebegin "Checking for masked packages"
         grep '^!!! All ebuilds that could satisfy.*have been masked.' "$StrWorkDir/emerge_world.out" &> "$StrWorkDir/masked.lst"
         if [ -s "$StrWorkDir/masked.lst" ]
         then
            eerror "There are the follows masked packages."
            cat "$StrWorkDir/emerge_world.out"
            eerror "Please fix them first and run updantoo late."
            eoutdent
            SubAbort
         else
            eend
         fi
         ebegin "Creating pretend list"
         grep '^\[' "$StrWorkDir/emerge_world.out" &> "$StrWorkDir/pretend.lst"
         if [ ! -s "$StrWorkDir/pretend.lst" ]
         then
            eerror "Failed, please fix the errors below and run updantoo late."
            cat "$StrWorkDir/emerge_world.out"
            eoutdent
            SubAbort
         else
            eend
         fi
      fi
      eoutdent
   else
      ewarn "updatoo was not prepared."
   fi
fi

if [ "$StrFetch" = "Y" -o "$StrDefault" = "Y" ]
then
   if [ ! -e "$StrWorkDir/pretend.lst" ]
   then
      ewarn "You need to prepare first in order to fetch."
      SubAbort
   fi
   StrAnswer=""
   if [ -e "$StrWorkDir/fetch.log" -a "$StrAsk" = "Y" ]
   then
      ewarn "updatoo has alredy fetched packages from the last list today."
      ewarn "Would you like to fetch in parallel again? (Y/n)"
      read StrAnswer
   fi
   if [[ "$StrAnswer" =~ ^([yY]([eE][sS])?)?$ ]]
   then
      ebegin "Fetching packages in parallel from pretend list"
      while read -r StrLine
      do
         StrPackage="`echo $StrLine | sed -e 's/^[^]]*\] //g' -e 's/ .*$//g'`"
         StrPackages="$StrPackages =$StrPackage"
      done < "$StrWorkDir/pretend.lst"
      $StrEmergeFetch $StrPackages &> "$StrWorkDir/fetch.log" &
   else
      ewarn "updatoo didn't fetch packages in parallel."
   fi
fi

if [ "$StrExecute" = "Y" -o "$StrDefault" = "Y" ]
then
   StrAnswer=""
   if [ -e "$StrWorkDir/emerged.lst" -a -e "$StrWorkDir/emerge.log" -a "$StrAsk" = "Y" ]
   then
      ewarn "updatoo has alredy executed today."
      ewarn "Would you like to execute again? (Y/n)"
      read StrAnswer
   fi
   if [[ "$StrAnswer" =~ ^([yY]([eE][sS])?)?$ ]]
   then
      rm -f "$StrWorkDir/emerged.lst" "$StrWorkDir/failed.lst" "$StrWorkDir/emerge.log"
      touch "$StrWorkDir/emerged.lst" "$StrWorkDir/failed.lst" "$StrWorkDir/emerge.log"
      NumPackages="`wc -l $StrWorkDir/pretend.lst | cut -d' ' -f1`"
      NumPackage="0"
      ebegin "Emerging $NumPackages packages from pretend list"
      eindent
      while read -r StrLine
      do
         let NumPackage++
         StrPackage="`echo $StrLine | sed -e 's/^[^]]*\] //g' -e 's/ .*$//g'`"
         ebegin "Emerging $StrPackage ($NumPackage of $NumPackages)"
         $StrEmergeOnePkg "=$StrPackage" >> "$StrWorkDir/emerge.log" 2>&1
         if [ "$?" -eq "0" ]
         then
            echo "$StrPackage" >> "$StrWorkDir/emerged.lst" 2>&1
            eend
         else
            echo "$StrPackage" >> "$StrWorkDir/failed.lst" 2>&1
            eend "Fail emerging $StrPackage!"
         fi
      done < "$StrWorkDir/pretend.lst"
      eoutdent
      if [ -s "$StrWorkDir/failed.lst" ]
      then
         StrError="Y"
      else
         einfo "All packages were emerged successfully!"; eend
      fi
   else
      ewarn "updatoo was not executed."
   fi
fi

if [ "$StrClean" = "Y" -o "$StrDefault" = "Y" ]
then
   StrAnswer=""
   if [ -e "$StrWorkDir/cleanup.log" -a "$StrAsk" = "Y" ]
   then
      ewarn "updatoo has alredy cleaned up your system today."
      ewarn "Would you like to clean up again? (Y/n)"
      read StrAnswer
   fi
   if [[ "$StrAnswer" =~ ^([yY]([eE][sS])?)?$ ]]
   then
      ebegin "Cleaning up the system"
      $StrEmergeDepClean &> "$StrWorkDir/cleanup.log" && $StrEclean >> "$StrWorkDir/cleanup.log" 2>&1 && $StrRmRfClean >> "$StrWorkDir/cleanup.log" 2>&1
      if [ "$?" -eq 0 ]
      then
         eend
      else
         eerror "Failed, please fix the errors describe in $StrWorkDir/cleanup.log first and run updantoo late."
         SubAbort
      fi
   fi
fi

if [ "$StrRev" = "Y" -o "$StrDefault" = "Y" ]
then
   StrAnswer=""
   if [ -e "$StrWorkDir/revdep-rebuild.log" -a "$StrAsk" = "Y" ]
   then
      ewarn "updatoo has alredy run revdep-rebuild today."
      ewarn "Would you like to run again? (Y/n)"
      read StrAnswer
   fi
   if [[ "$StrAnswer" =~ ^([yY]([eE][sS])?)?$ ]]
   then
      ebegin "Running revdep-rebuild"
      $StrRevdep &> "$StrWorkDir/revdep-rebuild.log"
      if [ "$?" -eq 0 ]
      then
         eend
      else
         eerror "Failed, please fix the errors describe in $StrWorkDir/revdep-rebuild.log first and run updantoo late."
         SubAbort
      fi
   fi
fi

if [ "$StrError" = "Y" ]
then
   eerror "Failed, some packages could not be emerged. Please fix the errors describe in $StrWorkDir/emerge.log for all packages in $StrWorkDir/failed.lst and run updantoo late."
   eend "Finished with errors!"
   exit 1
else
   einfo "Finished successfully!"
   eend
   exit 0
fi
### End Script ###

_________________
Human beings are a disease, a cancer of this planet, you are a plague, and we are the cure.


Last edited by asmith on Mon Mar 23, 2009 11:23 pm; edited 5 times in total
Back to top
View user's profile Send private message
poly_poly-man
Advocate
Advocate


Joined: 06 Dec 2006
Posts: 2477
Location: RIT, NY, US

PostPosted: Fri Dec 05, 2008 1:36 am    Post subject: Reply with quote

Code:
 # crontab -l
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.XXXXCqmN3K installed on Thu Nov 27 08:44:47 2008)
# (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $)
00 02 * * * eix-sync
30 02 * * * emerge -uDNv @world && emerge @preserved-rebuild && revdep-rebuild && echo "no need to worry" | mail -s "success" mkern@penguin

_________________
iVBORw0KGgoAAAANSUhEUgAAA

avatar: new version of logo - see topic 838248. Potentially still a WiP.
Back to top
View user's profile Send private message
asmith
n00b
n00b


Joined: 29 Jun 2008
Posts: 14
Location: Brazil

PostPosted: Fri Dec 05, 2008 1:44 am    Post subject: Reply with quote

poly_poly-man wrote:
Code:
 # crontab -l
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.XXXXCqmN3K installed on Thu Nov 27 08:44:47 2008)
# (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $)
00 02 * * * eix-sync
30 02 * * * emerge -uDNv @world && emerge @preserved-rebuild && revdep-rebuild && echo "no need to worry" | mail -s "success" mkern@penguin


The problem is if "emerge -uDNv @world" fail in the midle you can't continue. In my script every package(even deps) is installed even if the package before failed.
_________________
Human beings are a disease, a cancer of this planet, you are a plague, and we are the cure.
Back to top
View user's profile Send private message
poly_poly-man
Advocate
Advocate


Joined: 06 Dec 2006
Posts: 2477
Location: RIT, NY, US

PostPosted: Fri Dec 05, 2008 1:50 am    Post subject: Reply with quote

asmith wrote:
poly_poly-man wrote:
Code:
 # crontab -l
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.XXXXCqmN3K installed on Thu Nov 27 08:44:47 2008)
# (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $)
00 02 * * * eix-sync
30 02 * * * emerge -uDNv @world && emerge @preserved-rebuild && revdep-rebuild && echo "no need to worry" | mail -s "success" mkern@penguin


The problem is if "emerge -uDNv @world" fail in the midle you can't continue. In my script every package(even deps) is installed even if the package before failed.
true... but there's also the fact that I can fix it the next day if there's a problem :P

simplicity is the way to go
_________________
iVBORw0KGgoAAAANSUhEUgAAA

avatar: new version of logo - see topic 838248. Potentially still a WiP.
Back to top
View user's profile Send private message
asmith
n00b
n00b


Joined: 29 Jun 2008
Posts: 14
Location: Brazil

PostPosted: Fri Dec 05, 2008 8:58 am    Post subject: Reply with quote

poly_poly-man wrote:
asmith wrote:
poly_poly-man wrote:
Code:
 # crontab -l
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (/tmp/crontab.XXXXCqmN3K installed on Thu Nov 27 08:44:47 2008)
# (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $)
00 02 * * * eix-sync
30 02 * * * emerge -uDNv @world && emerge @preserved-rebuild && revdep-rebuild && echo "no need to worry" | mail -s "success" mkern@penguin


The problem is if "emerge -uDNv @world" fail in the midle you can't continue. In my script every package(even deps) is installed even if the package before failed.
true... but there's also the fact that I can fix it the next day if there's a problem :P

simplicity is the way to go


That's the point. You will spent one day for each failed package. In my case in the end of the day I will know every failed package excluding rdeps of course. By the way I will include e-mail advise in my roadmap. :D
_________________
Human beings are a disease, a cancer of this planet, you are a plague, and we are the cure.
Back to top
View user's profile Send private message
ToeiRei
Veteran
Veteran


Joined: 03 Jan 2005
Posts: 1191
Location: Austria

PostPosted: Fri Dec 05, 2008 9:10 am    Post subject: Reply with quote

just a little sidenote:

usually just updating packages may breaks things.
--buildpkgonly is my trick to prepare the packages for merging them later using emerge -K
_________________
Please stand by - The mailer daemon is busy burning your messages in hell...
Back to top
View user's profile Send private message
asmith
n00b
n00b


Joined: 29 Jun 2008
Posts: 14
Location: Brazil

PostPosted: Sat Dec 06, 2008 3:14 am    Post subject: Reply with quote

ToeiRei wrote:
just a little sidenote:

usually just updating packages may breaks things.
--buildpkgonly is my trick to prepare the packages for merging them later using emerge -K


I'm not sure but I think the idea to create and install binary packages with emerge is to standardize which means you can install in a lot of machines and keep them up-to-date.
_________________
Human beings are a disease, a cancer of this planet, you are a plague, and we are the cure.
Back to top
View user's profile Send private message
desultory
Bodhisattva
Bodhisattva


Joined: 04 Nov 2005
Posts: 9410

PostPosted: Sat Dec 06, 2008 3:38 am    Post subject: Reply with quote

That is one use, though it is certainly not the only one.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Sat Dec 06, 2008 3:46 am    Post subject: Reply with quote

Heh this is a weird post: update was originally subject-titled "YAF Update script" (it's still called that in my bookmarks;)

In the Gentoo spirit, I've pasted the earliest backup I have of it at: http://phpfi.com/385434 (note the lack of quoting on line 408 tsk;)

Major point: don't use cat like that; it's called a uuoc
Code:
while read -r foo; do
   crap -with "$foo"
done < "$file"

NB: this works on POSIX sh too. (Bookmark that link if you value your time and that of your users. BASH is a lot more portable than sed -i at least as far as a shellscripter is concerned.)

Major shellscripting point (ie more important in the longer run) USE MORE QUOTES until you know when not to: in a nutshell foo=$bar is fine in bash and no quotes needed in [[ unless you're doing something you shouldn't. ;) Other than that, quote every parameter or variable expansion (or feel the wrath of greycat which is a terrible thing to behold..;p)
Code:
emerge --oneshot --nodeps =$StrPackage >> $StrWorkDir/emerge.log 2>&1
          if [ $? -eq 0 ]

if COMMAND; then COMMAND; else COMMAND; fi
All a shell ever does is run commands. (OK you have control constructs, but it's a good way to think of it.)
Check out these links:
http://bash-hackers.org/wiki/doku.php?id=scripting:basics | http://wooledge.org/mywiki/BashGuide | http://www.grymoire.com/Unix/index.html | http://wooledge.org/mywiki/BashFAQ | http://wooledge.org/mywiki/BashPitfalls | http://www.shelldorado.com/ | and of course | http://forum.bash-hackers.org/
#bash are your friends (they just don't sound like it;)
If you're stuck on bash in cli, and you can't stand vim NOR emacs (I can't;) try this cp it to /usr/share/nano and add it to /etc/nanorc

asmith wrote:
ToeiRei wrote:
just a little sidenote:

usually just updating packages may breaks things.
--buildpkgonly is my trick to prepare the packages for merging them later using emerge -K


I'm not sure but I think the idea to create and install binary packages with emerge is to standardize which means you can install in a lot of machines and keep them up-to-date.

Yeah but building binpkgs before you install them is also a good way of testing on your own setup.
HTH.
_________________
creaker wrote:
systemd. It is a really ass pain

update - "a most excellent portage wrapper"

#friendly-coders -- We're still here for you™ ;)
Back to top
View user's profile Send private message
asmith
n00b
n00b


Joined: 29 Jun 2008
Posts: 14
Location: Brazil

PostPosted: Sun Dec 07, 2008 5:28 pm    Post subject: Reply with quote

Hi steveL!

Thanks a lot for the tips.
I've changed my script with your suggestions. Please take a look below the version 0.2.
What's your suggestion for e-mail notification?

Code:

#!/bin/bash
#
# updatoo
# Agent Smith (Ricardo Iramar dos Santos)
# ricardo.iramar@gmail.com
#
# updatoo is a bash script that performing a simple full update in a Gentoo System.
# You can run updatoo in two ways:
# 1. Without parameter (default) that it will ask you for confirmation in every step.
# 2. With --force (-f) that it won't ask you for anything.
# Basically updatoo will synchronize your portage tree with eix-sync, create a pretend list of packages and install them.
# If occur a problem the updatoo is aborted with code 1.
# Everything is loged in /root/.updatoo/ where you can check anytime.
# Please report any bug to ricardo.iramar@gmail.com or http://forums.gentoo.org/viewtopic-t-717092.html.
#
# ChangeLog
# Version 0.1 (29/11/2008)
# - First version. No bugs yet.
#
# Version 0.2 (07/12/2008)
# - Gentoo look output (/etc/init.d/functions.sh").
# - Some code fix.
#
# ToDo
# - Detect masked packages.
# - Test mode (do nothing, just create the lists).
# - E-mail notification.
#

### Begin Declare Variables ###
StrVersion="v0.2"
StrForce="no"
StrHomeDir="$HOME/.updatoo"
StrWorkDir="$StrHomeDir/`date +%F`"
StrAnswer=""
StrLine=""
StrPackage=""
NumPackages="0"
NumPackage="1"
StrFuncFile="/etc/init.d/functions.sh"
### End Declare Variables ###

### Begin SubHelp ###
SubHelp()
{
   echo "updatoo $StrVersion
updatoo --help | --force
--help (-h)   Print this help.
--force (-f)   Don't ask me anything."
}
### End SubHelp ###

### Begin SubAbort ###
SubAbort()
{
   eend "Aborted"
   exit 1
}
### End SubAbort ###

### Begin Script ###
source "$StrFuncFile"
if [ ! -d "$StrHomeDir" ]; then mkdir "$StrHomeDir"; fi
if [ ! -d "$StrWorkDir" ]; then mkdir "$StrWorkDir"; fi
if [ "$USER" != "root" ]
then
   ewarn "You need to be root to run updatoo."
   SubAbort
elif [ ! -f "/usr/bin/eix" ]
then
   ewarn "Please install eix first (emerge eix)."
   SubAbort
elif [ ! -f "/usr/bin/revdep-rebuild" ]
then
   ewarn "Please install gentoolkit first (emerge gentoolkit)."
   SubAbort
fi

if [[ "$@" =~ ^-(f|-force)$ ]]
then
   ebegin "Starting in force mode"
   StrForce="yes"
fi

if [[ "$@" =~ ^-(h|-help)$ ]]
then
   SubHelp
   exit 0
elif [[ "$@" =~ ^(-(f|-force))?$ ]]
then
   if [ -f "$StrWorkDir/eix-sync.log" -a $StrForce = "no" ]
   then
      ewarn "The portage tree has already synchronized today."
      einfo "Would you like to synchronize again? (Y/n)"
      read StrAnswer
   fi
   if [[ "$StrAnswer" =~ ^([yY]([eE][sS])?)?$ ]]
   then
      ebegin "Updating the portage tree with eix-sync"
      eix-sync &> "$StrWorkDir/eix-sync.log"
   fi
   ebegin "Creating package lists"
   if [ -f "$StrWorkDir/pretend.lst" -a $StrForce = "no" ]
   then
      ewarn "The pretend list already exits and will be overwritten."
      einfo "Would you like to continue anyway? (Y/n)"
      read StrAnswer
   fi
   if [[ "$StrAnswer" =~ ^([yY]([eE][sS])?)?$ ]]
   then
      ebegin "Creating pretend list"
      emerge --verbose --pretend --update --deep --newuse world | grep '^\[' &> "$StrWorkDir/pretend.lst"
      eend
      if [ "`cat $StrWorkDir/pretend.lst`" = "" ]
      then
         einfo "Your system is alredy updated!"
         exit 0
      fi
   else
      SubAbort
   fi
   ebegin "Checking for blocks packages"
   grep '^\[blocks' "$StrWorkDir/pretend.lst" &> "$StrWorkDir/blocked.lst"
   if [ "`cat $StrWorkDir/blocked.lst`" != "" ]
   then
      eerror "There are the follows blocks packages."
      cat "$StrWorkDir/blocked.lst"
      eerror "Please fix them first and run updantoo late."
      SubAbort
   else
      eend
   fi
   ebegin "Checking for fetch packages"
   grep '^\[ebuild[^]F]*F' "$StrWorkDir/pretend.lst" &> "$StrWorkDir/fetched.lst"
   if [ "`cat $StrWorkDir/fetched.lst`" != "" ]
   then
      eerror "There are the follows fetch packages."
      cat "$StrWorkDir/fetched.lst"
      eerror "Please fix them first and run updantoo late."
      einfo "Trying to emerge these packages in order to get the download URL."
      while read -r StrLine
      do
         StrPackage=`echo "$StrLine" | sed -e 's/^[^]]*\] //g' -e 's/ .*$//g'`
         emerge --oneshot --nodeps "=$StrPackage"
      done < "$StrWorkDir/fetched.lst"
      SubAbort
   else
      eend
   fi
   rm -f "$StrWorkDir/emerged.lst" "$StrWorkDir/failed.lst" "$StrWorkDir/emerge.log"
   touch "$StrWorkDir/emerged.lst" "$StrWorkDir/failed.lst" "$StrWorkDir/emerge.log"
   NumPackages=`wc -l "$StrWorkDir/pretend.lst" | cut -d' ' -f1`
   ebegin "Emerging $NumPackages packages from pretend list"
   while read -r StrLine
   do
      StrPackage=`echo "$StrLine" | sed -e 's/^[^]]*\] //g' -e 's/ .*$//g'`
      ebegin "Emerging $StrPackage ($NumPackage of $NumPackages)"
      let NumPackage++
      emerge --oneshot --nodeps "=$StrPackage" >> "$StrWorkDir/emerge.log" 2>&1
      if [ "$?" -eq "0" ]
      then
         echo "$StrPackage" >> "$StrWorkDir/emerged.lst" 2>&1
         eend
      else
         echo "$StrPackage" >> "$StrWorkDir/failed.lst" 2>&1
         eend "Fail emerging $StrPackage!"
      fi
   done < "$StrWorkDir/pretend.lst"
   if [ "`cat $StrWorkDir/failed.lst`" = "" ]
   then
      einfo "All packages were emerged successfully!"; eend
      if [ "$StrForce" = "no" ]
      then
         einfo "Would you like clean up the system? (Y/n)"
         read StrAnswer
      fi
      if [[ "$StrAnswer" =~ ^([yY]([eE][sS])?)?$ ]]
      then
         ebegin "Cleaning up the system"
         emerge --depclean &> $StrWorkDir/cleanup.log
         rm -rf /var/tmp/portage/* >> $StrWorkDir/cleanup.log 2>&1
         eclean distfiles >> $StrWorkDir/cleanup.log 2>&1
         eend
      fi
      if [ "$StrForce" = "no" ]
      then
         einfo "Would you like to run revdep-rebuild? (Y/n)"
         read StrAnswer
      fi
      if [[ $StrAnswer =~ ^([yY]([eE][sS])?)?$ ]]
      then
         ebegin "Runing revdep-rebuild"
         revdep-rebuild &> "$StrWorkDir/revdep-rebuild.log"
         eend
      fi
   else
      mv "$StrWorkDir/pretend.lst" "$StrWorkDir/failed_pretend.lst"
      eerror "These packages couldn't be merged:"
      cat "$StrWorkDir/failed.lst"
      eerror "Please fix them manually and run updatoo again."
      SubAbort
   fi
else
   eerror "Invalid option."
   SubHelp
   exit 1
fi
einfo "Finished!"
exit 0
### End Script ###

_________________
Human beings are a disease, a cancer of this planet, you are a plague, and we are the cure.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Mon Dec 08, 2008 5:57 am    Post subject: Reply with quote

asmith wrote:
Thanks a lot for the tips.

You're welcome :-)
Quote:
I've changed my script with your suggestions. Please take a look below the version 0.2.

OK first thing, this isn't Visual Basic. (Yes I used to code in it too, you're fine;) Every variable is a string. Even when you declare -i it. (That just means it can only ever be assigned a 64-bit integer.)
First place to check for anything is help, eg help declare or help test (you need to print that one out.)
Don't add Sub to the beginning of functions. You have complete control over what is and isn't in the script namespace. Just don't use the same name as a POSIX command (as in the link above) and if you want to be quick, use Kate as its syntax highlighting rocks (it has loads of standard Unix and Linux commands as well.)
Code:

StrVersion="v0.2"
declare -r version=0.2

StrHomeDir="$HOME/.updatoo"
declare -r homeDir=~/.updatoo # $HOME can be set in env. ~ can't afaik (it's in POSIX)
# Of course, sometimes you want the user to be able to override.
# A system update script isn't one of those times.

StrWorkDir="$StrHomeDir/`date +%F`"
workDir="$homeDir/$(date +%F)"
# $( is much easier to deal with than ` (nesting.) It's also specified for POSIX SH.
StrAnswer=""
# there's no need to declare empty variables
# if you reference a non-existent var, you get ''
NumPackages="0"
declare -i numPkg=0
# use whatever name you want, they're your fingers ;)
NumPackage="1"
declare -i currPkg=1
StrFuncFile="/etc/init.d/functions.sh"
funcFile=/etc/init.d/functions.sh
declare -i force=0
### End Declare Variables ###

### Begin SubHelp ###
#There's really no need for these; a comment before the function is fine
#and you can expand that as and when
### End SubHelp ###

if [ ! -d "$StrHomeDir" ]; then mkdir "$StrHomeDir"; fi
[[ -d "$homeDir" ]] || mkdir "$homeDir"

if [ "$USER" != "root" ]
if ((EUID)); then
   ewarn "You need to be root to run ${0##*/}."
   SubAbort # making a generic die() function is a lot easier
elif [ ! -f "/usr/bin/eix" ]
# The only characters that matter in terms of quoting are: ? * [ ( { ` $ ' " (and # obviously)
# Be safe -- just use [[ in BASH; it's quicker too.

if [[ "$@" =~ ^-(f|-force)$ ]]
# If you want compatibility across bash 3 you need to use variables to hold regexen.
# This is _not_ the way to check parameters; I only do that in update for stuff I *have* to know before
# the main command processing function can be called.
# See: http://wooledge.org/mywiki/BashFAQ/035
   ebegin "Starting in force mode"
# Again, no need to use " when you don't want any variable expansion.
   StrForce="yes"
        force=1

if [[ "$@" =~ ^-(h|-help)$ ]]
then
   SubHelp
   exit 0 # why can't helpMsg just exit 0? (or usage)
   if [ -f "$StrWorkDir/eix-sync.log" -a $StrForce = "no" ]
   if [[ -f $workDir/eix-sync.log ]] && ((!force)) # doing it like this means
# you don't lose out if the variable hasn't been declared as it'll be zero in ((
# (( is also very useful, so use it where appropriate
# see http://wooledge.org/mywiki/ArithmeticExpression
   if [[ "$StrAnswer" =~ ^([yY]([eE][sS])?)?$ ]] # EW
        if [[ $answer = [Yy][Ee][Ss] ]]
   then
      ebegin "Updating the portage tree with eix-sync"
      eix-sync &> "$StrWorkDir/eix-sync.log" # what happens if this errors?
      eix-sync &> "$workDir/eix-sync.log" || abort 'Unable to eix-sync'

      read StrAnswer
   if [[ "$StrAnswer" =~ ^([yY]([eE][sS])?)?$ ]]
# might want to make this generic (ie use a function)
      if [ "`cat $StrWorkDir/pretend.lst`" = "" ] # *shudder*
      if [[ ! -s $workDir/pretend.lst ]] # help test!!

   rm -f "$StrWorkDir/emerged.lst" "$StrWorkDir/failed.lst" "$StrWorkDir/emerge.log"
   touch "$StrWorkDir/emerged.lst" "$StrWorkDir/failed.lst" "$StrWorkDir/emerge.log"
   for f in emerged.lst failed.lst emerge.log; do
   declare -r installFiles='emerged.lst failed.lst emerge.log' # earlier ofc
   for f in $installFiles; do # we _want_ word splitting so we don't quote
      rm -f "$f" && touch "$f"
   done

      emerge --oneshot --nodeps "=$StrPackage" >> "$StrWorkDir/emerge.log" 2>&1
      if [ "$?" -eq "0" ]
      if emerge --oneshot --nodeps "=$StrPackage" >> "$workDir/emerge.log" 2>&1; then

Quote:
What's your suggestion for e-mail notification?

I'd say forget about it; if someone wants to use your script from cron, all output will be emailed to them anyhow ;)

edit: added # to list of chars that matter to quoting ;)


Last edited by steveL on Tue Dec 09, 2008 3:35 pm; edited 1 time in total
Back to top
View user's profile Send private message
asmith
n00b
n00b


Joined: 29 Jun 2008
Posts: 14
Location: Brazil

PostPosted: Mon Dec 08, 2008 10:14 pm    Post subject: Reply with quote

Thanks again steveL.
I know that every var is a string. The names of vars and functions its just for my control and turn the script more readable for the others n00bs like me. :D
_________________
Human beings are a disease, a cancer of this planet, you are a plague, and we are the cure.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Tue Dec 09, 2008 3:00 pm    Post subject: Reply with quote

asmith wrote:
The names of vars and functions its just for my control and turn the script more readable for the others n00bs like me. :D

I understand, but I'm trying to get you into good habits now, before it's too late ;) In your case, that means unlearning the VB habits you have so you can approach shell as a shellscripter would.

There's simply no need to indicate the type of a variable in its name, since they're all string. (Many people disagree with it even when you do have many types; I've used it in VB/A in the past but I prefer simple names nowadays. Maybe it's just age;) Remember that you have full control of all the variables in your script. You can use what's in the environment, or set your own values (and use defaults, alternatives etc) and make them read-only with declare -r or readonly (both builtin so accessible from help in terminal.) I recommend you read this as well: http://wooledge.org/mywiki/BashFAQ/073 wrt what you can do with variables.

WRT function naming, the whole point, in shellscript, is that they are used interchangeably with commands, and you can indeed 'decorate' an existing command (or make it nicer to use) by using a function with the same name. I recommend you do NOT do that in the general case, but it can be used, eg where the underlying implementation is not POSIX-compliant. It's better to be in the habit of knowing how the underlying commands work by using them as-is. This is the same argument against lots of custom aliases of an existing command: when you use another machine, in whatever context, you may well not have them available, but you will still be expected to get work done.

A much better reason for a function (apart from the standard coding ones of minimising code-duplication/maximising reuse) is to wrap a command (not using the same name as explained) to make it nicer to use. Here's a function which does just that, from update:
Code:
eed() {
   (($#>1)) || abort
   local f=$1; shift
   printf '%s\n' H "$@" |ed -s "$f"
}

ed reads its commands from standard input, separated by newlines; the -s tells it to run in script mode. To get proper error messages (as opposed to a simple ?) we issue it the H command.
In console I might do:
Code:
printf '%s\n' H 'g/foo/s/bar/baz/g' w | ed -s someFile

Typing this same stuff over and over gets tiresome (even with alias print="printf '%s\n'" which I love to bits ;) and doesn't look nice in a script. By using the function (which stands for 'easy ed') in the script (it's in my bashrc as well;) the code is much more just about the stuff I'm interested in; eg to display the libErrCheck code to the user (update -h error), I use:
Code:
eed /sbin/update '/^# libErrCheck/;/^}/p'

I like having the filename upfront, and then being able to type in whatever I want; it's more natural and easier to maintain. The only thing I'm reading is stuff that's relevant to what I'm doing; the stuff I might want to edit.

I could use ed -s file <<< "blah" but it's BASH specific and the escaping required is very annoying; we already have to escape regexen. You can find 'eed' by searching for it as a (case-sensitive) 'whole word' in Kate; you'll see it's used more via another wrapper in update,eg deleteSearch or edit. The first is so I can escape a search term once and then reuse it, the second is for more convenience.

What I'm getting at, in my rambling manner, is that you need to get into the headspace of shellscripting, which will only happen after you've let go of VB. Other people starting out, who haven't coded anything but have used the terminal a lot (which practically any Gentoo user has) are actually in a better position, in that they're in that headspace already. (OFC they then need to learn the basics of coding..;)

"At its base, a shell is simply a macro processor that executes commands. The term macro processor means functionality where text and symbols are expanded to create larger expressions." http://tiswww.tis.case.edu/~chet/bash/bashref.html
Back to top
View user's profile Send private message
asmith
n00b
n00b


Joined: 29 Jun 2008
Posts: 14
Location: Brazil

PostPosted: Tue Feb 03, 2009 12:48 am    Post subject: updatoo (new version) Reply with quote

Posted new version.
_________________
Human beings are a disease, a cancer of this planet, you are a plague, and we are the cure.
Back to top
View user's profile Send private message
asmith
n00b
n00b


Joined: 29 Jun 2008
Posts: 14
Location: Brazil

PostPosted: Mon Mar 23, 2009 11:20 pm    Post subject: New version! Reply with quote

Posted new version 0.4.
Please take a look in the first post.
_________________
Human beings are a disease, a cancer of this planet, you are a plague, and we are the cure.
Back to top
View user's profile Send private message
asmith
n00b
n00b


Joined: 29 Jun 2008
Posts: 14
Location: Brazil

PostPosted: Mon Apr 06, 2009 9:20 pm    Post subject: Added updatoo into Google Code site Reply with quote

Now you can help me to improve updatoo or download via svn.
Please take a look at http://code.google.com/p/updatoo/.
_________________
Human beings are a disease, a cancer of this planet, you are a plague, and we are the cure.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum