Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Creating own emul-packages
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo on AMD64
View previous topic :: View next topic  
Author Message
SeeksTheMoon
Apprentice
Apprentice


Joined: 24 Sep 2003
Posts: 158

PostPosted: Mon Aug 08, 2011 4:29 pm    Post subject: Creating own emul-packages Reply with quote

Hi,

I want to use 32bit mesa from git on my ~amd64 multilib system and want to find out how I can make my own emul-package containing this package, i.e. my own replacement for emul-linux-x86-opengl or as an acceptable workaround replace all/some of it's files by myself (e.g. compile 32bit mesa somehow and replace libGL.so).
There are some problems and conditions:

  1. I do not want to use the multilib-overlay as I have to downgrade many packages and compile 32bit packages that I don't need
  2. I want to use the mesa-9999 ebuild from x11-overlay, which is the ebuild I use for 64bit too
  3. I most certainly won't do this http://www.gentoo.org/proj/en/base/amd64/howtos/chroot.xml as I am forced to execute my programs from within the chroot and compile a complete new system there (X and wine and whatever), which is utterly redundant
  4. http://www.gentoo.org/proj/en/base/amd64/emul/ looks nice, but is deprecated and doesn't work either. I tried it and many packages won't compile (e.g. in the emerge -e world step) so I cannot prepare a proper chrootsystem. There is no obvious replacement documentation available (I hope that the 32 bit chroot is NOT the replacement for that)

So... what to do?
Back to top
View user's profile Send private message
m0p
Apprentice
Apprentice


Joined: 20 Jun 2005
Posts: 205
Location: en_GB

PostPosted: Tue Aug 09, 2011 4:49 pm    Post subject: Reply with quote

I used to do something like this when I still had 32-bit packages installed and wanted SSP/FORTIFY_SOURCE/PIE across the board:
Code:
ABI=x86 USE="-disable -flags -that -make -builds -fail -like -python" emerge --buildpkgonly --nodeps -1 package-name


A 32-bit chroot is unnecessary for most packages. Any ebuilds that use Python will fail because Portage tries looking for it in lib32 because the Python eclass isn't aware of multilib and just follows the variables set by the ABI switch, but you could probably play around with the slotting/eselect-python to get a makeshift multilib Python (emerge from another slot with ABI=x86), mess around with the appropriate eclass to make it look in the right place, or maybe even symlink the lib64 Python stuff into lib32 (not for daily use, just for doing these builds) and hope that Portage doesn't care about finding 64-bit stuff in lib32.

Some builds will fail with weird compiler/linker errors too, in which case you might have to set CBUILD/CTARGET/CHOST/etc to 'i686-pc-linux-gnu' (on the command line, I'm not sure if you need all of them but different packages need different combinations so you should just set all of them). You need to do that anyway though on packages like Pango that use the build CHOST to find their configs (otherwise it will load the x86_64-pc-linux-gnu config and look for it's modules in lib64 then refuse to load them because they're not 32-bit), but if you do it on some builds, you'll get linking errors or sometimes TEXTRELS (usually in MMX/SSE code, which is usually tough to fix and best disabled).

You might have to modify a few ebuilds as well (ones that use Python mainly, which includes mesa, though you only have to remove 1 line and modify another in a local overlay and it will build just fine), and you'll need to install some libs before you can build others so the first time you do it, it will be time consuming, so you might want to split out the emul packages even into smaller chunks and have the bigger packages depend on them so you don't have to keep updating the same package over and over again.

There's a listing of the contents of the different packages here:
http://www.gentoo.org/proj/en/base/amd64/emul/emul-linux-x86-20110722.xml

When I still had 32-bit stuff on my system I used to look at the linkage for all of my 32-bit binaries and only put the required packages in my emul-libs, instead of installing everything on those lists (e.g. all the LDAP rubbish, cups, hopelessly old versions of non-turbo jpeg, GTK+ 1.x, libnotify, gstreamer, etc) and wasting loads of time manually keeping track of security updates for libraries that I don't even need.

I was going to write a script at some point to automatically update the packages along with normal updates, unpack them, strip out everything but the libs on the 'linkage' list of all the 32-bit packages I'd installed, repack it and stick it in distfiles, bump the local ebuild, digest and emerge, but then I finally upgraded to GRUB2 and started using the Flash 11 beta so there's no need now. If I ever start using multilib again, I'll post the script up on the forum :)

edit: fixed horrible grammar, wrote this while very intoxicated and then fixed it twice while intoxicated before realising I'd made it worse...
Back to top
View user's profile Send private message
Shining Arcanine
Veteran
Veteran


Joined: 24 Sep 2009
Posts: 1110

PostPosted: Sun Oct 02, 2011 3:23 am    Post subject: Reply with quote

It would be nice if these were source based packages. Then changes could be made fairly easily.
Back to top
View user's profile Send private message
mark_alec
Bodhisattva
Bodhisattva


Joined: 11 Sep 2004
Posts: 6066
Location: Melbourne, Australia

PostPosted: Sun Oct 02, 2011 4:18 am    Post subject: Reply with quote

There is a multilib portage variation floating around somewhere - see https://bugs.gentoo.org/show_bug.cgi?id=portage-multilib for more details.

I believe it has the potential to allow for building 32-bit libraries on the 64-bit with ease.
_________________
www.gentoo.org.au || #gentoo-au
Back to top
View user's profile Send private message
SeeksTheMoon
Apprentice
Apprentice


Joined: 24 Sep 2003
Posts: 158

PostPosted: Sat Oct 22, 2011 11:34 pm    Post subject: Reply with quote

I found out how to do it, basically you create a 32bit chroot, compile mesa and your drivers after installing layman and the x11-overlay, where I use the 9999-ebuilds from. After emerging everything within the chroot, just copy some files from there into your lib32 folders within the 64bit system and restart X.

Here are the commands; maybe there are some you don't need, like the gcc emerge but it worked this way on my system:

Code:

mkdir /mnt/emul-chroot
mkdir /mnt/emul-chroot/mnt/ramdisk
cd /mnt/emul-chroot
wget ftp://ftp.informatik.rwth-aachen.de/pub/linux/gentoo/releases/x86/current-stage3/stage3-i686-20111018.tar.bz2
tar xfjp stage3-i686-20111018.tar.bz2 && rm stage3-i686-20111018.tar.bz2
wget ftp://ftp.informatik.rwth-aachen.de/pub/linux/gentoo/releases/snapshots/current/portage-latest.tar.xz
tar xfJp portage-latest.tar.xz -C usr && rm portage-latest.tar.xz
mount -o bind /dev /mnt/emul-chroot/dev/
mount -t proc none /mnt/emul-chroot/proc/
mount -o bind /mnt/ramdisk/ /mnt/emul-chroot/mnt/ramdisk/
mkdir /mnt/emul-chroot/etc/portage

echo "media-libs/mesa d3d gles llvm g3dvl\nmedia-libs/mesa gles llvm\napp-portage/layman git" > /mnt/emul-chroot/etc/portage
echo "x11-libs/libdrm\nx11-drivers/xf86-video-ati\nmedia-libs/mesa" > /mnt/emul-chroot/etc/portage/package.unmask

cat > /mnt/emul-chroot/etc/make.conf << EOF
USE="-fortran -ipv6 -openmp 3dnow smp threads vdpau xvmc"
VIDEO_CARDS="radeon"
CFLAGS="-march=athlon-mp -O2 -pipe"
CXXFLAGS="${CFLAGS}"
CHOST="i686-pc-linux-gnu"
ACCEPT_KEYWORDS="~x86"
MAKEOPTS="-j4"
PORTAGE_NICENESS=0
PORTAGE_TMPDIR=/mnt/ramdisk
EOF

chroot /mnt/emul-chroot/ /bin/bash
emerge layman
layman -L
layman -a x11
echo "source /var/lib/layman/make.conf" >> /mnt/emul-chroot/etc/make.conf
emerge gcc
gcc-config -l
gcc-config (version)
. /etc/profile
emerge libdrm mesa xf86-video-ati
exit
cp /mnt/emul-chroot/usr/lib/mesa/* /usr/lib32/mesa/
cp /mnt/emul-chroot/usr/lib/libgl* /usr/lib32/
cp /mnt/emul-chroot/usr/lib/libdrm* /usr/lib32/
quickpkg emul-linux-x86-opengl
eselect mesa set 32bit r600 gallium

/etc/init.d/xdm restart


If someone wants to put this into the gentoo-wiki and write some text, be my guest... I am too lazy to write a complete howto in another language again
Back to top
View user's profile Send private message
Shining Arcanine
Veteran
Veteran


Joined: 24 Sep 2009
Posts: 1110

PostPosted: Sun Oct 23, 2011 5:31 am    Post subject: Reply with quote

That can cause security problems because the files will not be updated by portage. Someone should open a bug report requesting that portage automate this.
Back to top
View user's profile Send private message
SeeksTheMoon
Apprentice
Apprentice


Joined: 24 Sep 2003
Posts: 158

PostPosted: Sun Oct 23, 2011 8:57 am    Post subject: Reply with quote

I don't think that this is a big security risk; here is why:

  • The files you built are newer than the files from portage; if you update daily or weekly (which is the intention for 9999-packages), possible flaws are fixed faster by you than ever possibly by gentoo and malware-writers who customize malware for you (wow, you have some serious enemies out there :wink:) have to be very speedy to catch up
  • I signed a bug report to update the packages and I mailed to the (only!) emul-packages-developer and had to wait weeks for an update (I think he only made one, because some X11 devs from gentoo seconded my bug) and more for an answer and then he used the stable release of mesa, which is not sufficient for me. I bet, the next update will come spring or summer 2012.
  • The mesa guys cross-sign their patches and don't check in new patches directly; they are being discussed some days or weeks on their mailinglist (you can check the patches too, if you are paranoid)
  • Only 32bit applications could use security problems directly, for 64bit apps I guess the malware has to be tailored for your system ("let's include a search for 32bit libs in this malware as I know this guy uses some custom files in /usr/lib32 which he never updates and I found a flaw in")
  • If you bought your win32 game and play it with these files and the game is 5 years old or so (or otherwise unmaintained), there are potentionally more security risks in your game itself than in your opengl files
  • No system will ever be safe; as emerge could be easily tricked by a man in the middle attack, everyone could give you some custom ebuilds and tarballs with some malware inside (do they finally use gpg in portage?)


I thought about an overlay though, which has daily ebuilds and daily builds. It should be fairly easy to create, but for me it is sufficient to update manually.
Back to top
View user's profile Send private message
Shining Arcanine
Veteran
Veteran


Joined: 24 Sep 2009
Posts: 1110

PostPosted: Sun Oct 23, 2011 6:30 pm    Post subject: Reply with quote

RHEL/CentOS users install software outside of their package managers all the time and when security patches are promulgated, they never apply them because they have no idea that it applies to them. If you are able to keep up with updates, then that is fine, but this is not a good practice.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo on AMD64 All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum