Unofficial minimal 2008.0 for x86/amd64 w/reiser4+truecrypt

Message

zxy · Post by **zxy** » Mon Apr 28, 2008 2:05 am

Neo2 thanks, cd like this was needed for some time.

Subscribing to the thread.

Dottout · Post by **Dottout** » Wed Apr 30, 2008 4:25 pm

hi, what about ext4 support? is it included? (both in kernel and e2fsck?)

Neo2 · Post by **Neo2** » Wed Apr 30, 2008 6:32 pm

Yes, ext4 support is included in the kernel. However I don't know if e2fsck supports ext4. Reading /usr/src/linux/Documentation/filesystems/ext4.txt tells:

Code: Select all

1. Quick usage instructions:
===========================

  - Grab updated e2fsprogs from
    ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs-interim/
    This is a patchset on top of e2fsprogs-1.39, which can be found at
    ftp://ftp.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/

  - It's still mke2fs -j /dev/hda1

  - mount /dev/hda1 /wherever -t ext4dev

  - To enable extents,

        mount /dev/hda1 /wherever -t ext4dev -o extents

  - The filesystem is compatible with the ext3 driver until you add a file
    which has extents (ie: `mount -o extents', then create a file).

    NOTE: The "extents" mount flag is temporary.  It will soon go away and
    extents will be enabled by the "-o extents" flag to mke2fs or tune2fs

  - When comparing performance with other filesystems, remember that
    ext3/4 by default offers higher data integrity guarantees than most.  So
    when comparing with a metadata-only journalling filesystem, use `mount -o
    data=writeback'.  And you might as well use `mount -o nobh' too along
    with it.  Making the journal larger than the mke2fs default often helps
    performance with metadata-intensive workloads.

I've done some quick tests. The command mke2fs -j /dev/part works, and mount correctly mounts the partition as ext4 (without -j option mount fails with "filesystem has no journal"). Looking at e2fsprogs release notes suggests that ext4 support is already included. I tested e2fsck with version 1.40.6 and this was the result on a loopback filesystem:

Code: Select all

fabio@gentoo-node1 ~ $ sudo e2fsck -pvfD -C 0 prova
e2fsck: Filesystem has unsupported feature(s) while trying to open prova
prova:
The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>

Honestly I don't know how to fix this except creating a custom ov. ebuild to use instead of the mainstream one. I looked at the link posted above for ext4 patches for e2fsprogs, but the latest commit is 2008/04/03 for version 1.39. Hence I don't know where to look for working/correct patches... sorry

I think you'll have to wait some time to get decent/mainstream ext4 support in e2fsprogs.

About the next release: I'm thinking of waiting for a 2.6.25-rx release to be in portage for some time or the official 2008.0 to come out. After that I'll probably rebuild the CDs.

Cheers,
Neo2

Dottout · Post by **Dottout** » Wed Apr 30, 2008 6:38 pm

thank you for your full reply

official e2fsprogs releases dont support ext4 yet, the working (and yes, unstable) code of e2fsprogs, ss and com_err can be found at git://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git (branch 'next'). if you are interested in ext4 support you can add my mail to msn (Im italian) or join #zen-sources @ freenode, some devs there are into ext4

Neo2 · Post by **Neo2** » Wed Apr 30, 2008 7:40 pm

Hey, I'm italian too!

Anyway, I'll try to work on some git-based overlay ebuild for e2fsprogs to include it in the next release. Actually I'm quite busy on university exams until the 11th and I'm still waiting for some ati-driver ebuild to compile against 2.6.25

Will keep you updated

Cheers,
Neo2

sPHERE911 · Post by **sPHERE911** » Mon May 05, 2008 2:09 pm

So there is actually going to be updates to the installcd? It's very, VERY tempting to try the installcd's you've made, but if you'r going to release a new version in the near future I'll gladly wait for that!

wrc1944 · Post by **wrc1944** » Mon May 05, 2008 3:08 pm

Neo2,
If you update the kdernel to 2.6.25 and rebuild the cd, add the ath5k driver as a module for the madwifi wireless chipset cards- it works very well. I and countless others had been struggling for years with madwifi wireless cards both with ndiswrapper and many madwifi driver versions, but the new ath5k worked out of the box, first time. Module autoloaded, and just a "dhcpcd ath0" connected, no problems.
What a change!

Neo2 · Post by **Neo2** » Mon May 05, 2008 3:43 pm

Well, I think I'll build a new release in 3-5 weeks time, not less. This is because 2.6.25 kernel is under active development and bugs get fixed almost every day, leading to a minor release every 1-2 week(s). It is thus worthless to upgrade and see a new release come out the day after. As of now I don't recommend waiting for a new release, if you want to try them it would be better to download the current one

ath5k driver will be obviously included it as one of the goals of these CDs is to support as much hardware as possible

But I have to remind you that as of now ath5k works only with AR5211/AR5212 chipsets. Madwifi-ng is actually not included in the current release (I just discovered the fact), sorry. When I built this release I was completely unaware of madwifi, until I bought my new laptop. Will include them as well in the next release since many laptops out there have integrated atheros chipsets.

Cheers,
Neo2

Need4Speed · Post by **Need4Speed** » Thu May 15, 2008 8:32 pm

Thanks for reiser4 livecd! You should get a link to this on the the reiser4 wiki so more people know about it. Oh and nice hosting, I was getting it at 1MB/sec.

yngwin · Post by **yngwin** » Thu May 15, 2008 10:09 pm

Wait - there is a reiser4 wiki?

Neo2 · Post by **Neo2** » Thu Jun 05, 2008 6:32 pm

Sorry, been away for a while. I think Need4Speed meant the wiki page for reiser4: http://en.wikipedia.org/wiki/Reiser4
Anyway, good news for all the supporters

Since gcc 4.2.4 has made it into the official tree and is marked unstable (currently using it on everyday compiles, no problems seen so far) and 2.6.25.4 has been around for quite a while now (3 weeks), I'm rebuilding the CDs.
I'm currently rebuilding stages 1, 2, 3 for both i686 and amd64. I should be able to upload the i686 architecture (tarballs are ready) by this evening, and the amd64 architecture by tomorrow (stage3 in progress). Since no packages can be added, I'm thinking of basing the following two stages (installcd-stage1 and installcd-stage2) only on those three. That means that the stages you'll find from tomorrow on won't be rebuilt anymore (less work for me ^^).
They feature updated package set + brand new gcc 4.2.4. Last release was based on 2008.0 profile, this release features 2008.0 as well, the only difference being that 2008.0 is no longer in /dev branch.
I'll rework a bit the directory tree on the server, so that previous releases will be available as well. This should take just few seconds, but to avoid chaos when downloading I'll ask you not to download anything until tomorrow 11.00 (GMT+1). Before that time I should have uploaded all the new stages to the server. Basically you'll just find the release date in american format after the "releases" directory (eg: unofficial/releases/20080605/...). Latest release will be marked "current" until final liveCDs will be uploaded. After that I'll assume the final release date as the name for that release. The rest of the directory hierarchy will remain untouched.
If you've got anything you wish to have integrated, please ask before saturday evening (19.00 GMT+1). Currently pending additions are:

+madwifi-ng (*)
+madwifi-ng-tools (*)
+latencytop
+iftop

* = I'll go straight for the latest unstable version for the package.
I'll also correct a bug I noticed when it was too late to rebuild that affects the kernel ide/sata detection: ide-scsi feature was built-in, leading to spurious /dev/sda entries 8Mb-wide that had nothing to do with real discs, obviously shifting the whole real device tree a position ahead, real "sda" being shown as "sdb". I'll simply port ide-scsi back to module.
I'll use the latest unstable version for genkernel, and that means that busybox will be upgraded to 1.7.4, hopefully leading to faster startup.
Closing requests on saturday evening means that in 4-5 days I should be able to address all the problems that may arise during installcd-stage1/2 compile.
Thanks for your support and appreciation, hope you'll find the new release more useful and complete, once it's done.
Oh, was about to forget: one of these days I'll update the first post to have a changelog/feature list on the first page, so that people won't have to read the whole thread every time.

Cheers

[edit]As of today, at 10.20, updated stages for both architectures are available for download and folder hierarchy has been reworked. Waiting for saturday to begin building installcd-stage1 and installcd-stage2[/edit]

sPHERE911 · Post by **sPHERE911** » Fri Jun 06, 2008 9:25 pm

Great! Looking forward to test the updated cd's!

Neo2 · Post by **Neo2** » Sat Jun 07, 2008 5:38 pm

Integration requests closed! Seems like there's nothing to add...

I'll be working on the the two final stages from tomorrow on.

Cheers

Eckos · Post by **Eckos** » Mon Jun 09, 2008 7:03 am

Can you bump the compiler up to 4.3.1 since it's in portage now?

Need4Speed · Post by **Need4Speed** » Mon Jun 09, 2008 3:38 pm

yngwin wrote:Wait - there is a reiser4 wiki?

I was referring to the gentoo reiser4 howto: http://gentoo-wiki.com/HOWTO_Reiser4_Wi ... oo-Sources

Neo2 · Post by **Neo2** » Mon Jun 09, 2008 4:56 pm

@Need4Speed: Thanks for correction

will ask about linking on the discussion page
@Eckos: Thanks for the update. I'm currently building gcc-4.3.1 on both my machines. If it compiles fine, I'll switch to gcc-4.3.1 permanently and subsequently update all the stages (sigh, I hoped I didn't have to do it again this time). According to gcc changelogs, 4.3.0 has 1691 fixed bugs and 104 for the 4.3.1 release, leading to a grandtotal of 1795 fixed bugs since gcc-4.2.4. Hopefully it should work without much hassle. If anybody has a reason to stick with 4.2.4 (except 4.3.1 being too new), please speak now

I just tested the livecd I'm building and keymap selection at boot is still somehow screwed up: "Please select a keymap from the list or hit enter for default" shows up correctly but entering a choice has no results -.-" Luckily this is not a showstopper.
At least I reached a good kernel config for i686 architecture and programs that have been integrated work as expected (truecrypt and ndiswrapper as major ones) ^^
madwifi-ng shows some module issues, but I think they were caused by the fact that I opted for incremental builds (aided with caches to reduce compile time) instead of a fresh one (I'll do it once everything's ready for release).

Stay tuned

ps. in this period I'm not on holidays, thus my hope is to plan and start builds before leaving home and finding them finished when I come back from university. If releases are delayed, this is the major cause.

[EDIT]Latest news: on amd64 platform gcc-4.3.1 enforces >=glibc-2.7-r2 to workaround multilib issues. Thus, I need to recompile glibc also -.-"
I'll do it on both my systems and test. Please note that glibc's home page marks 2.7 as a stable version (2.8 is in development). So, if it works fine, I'll consider it "stable" (at least for my purposes), prune glibc-2.6 and gcc-4.2.4, and then proceed with liveCD builds. This hopefully won't take long, but CDs devel will hang for a while, at least until I'll have everything working correctly... Sorry about that.[/EDIT]

Neo2 · Post by **Neo2** » Mon Jun 16, 2008 8:10 pm

Hi all guys, I'm back! This time I wanted to make you a surprise. Having done "emerge -e1 world" on both my systems (x86+amd64) with gcc-4.3.1 everything's running perfectly with glibc-2.7-r2 (compiled against linux-headers-2.6.25-r4).
Some notes for this release:
-> I decided to modularize whole PATA/SATA stuff, meaning that at boot the kernel will spot automatically which modules do work for your hardware. New drivers take precedence against the old ones, so you may find your CDs renamed as /dev/sr0 /dev/sr1 etc and your HDDs named /dev/sda /dev/sdb etc regardless of actually being PATA or SATA. This is because I tried to compile in-kernel only ide-generic (to make CD work at boot) but: a) this is not needed b) this leads to impossibility to enable DMA on all IDE ports that are managed by ide-generic (and this would have been rather annoying)
-> kernel is 2.6.25-reiser4-r5 (based upon 2.6.25.6, custom ebuild)
-> glibc is 2.7-r2 for both architectures. This was to avoid troubleshooting two systems in parallel (x86/amd64). I just keep configs in sync
-> linux-headers is 2.6.25-r4. I've been using them for two months now and can't really find something to complain about. Since that config is the same I use on my systems and I've tested it to death with many days of uptime (and every kind of everyday use), I consider it to be a good config.
-> last three lines mean that your package.keywords will need at least those entries:

sys-devel/gcc ~arch
=sys-libs/glibc-2.7.-r2 ~arch
=sys-kernel/linux-headers-2.6.25-r4 ~arch

Note that I've keyworded specific atoms for glibc and linux-headers. This is because generally updating those packages involves recompiling of some others. Gcc doesn't worry about that instead.
If you want to use my stage3 to build up a new system and want to emerge multimedia stuff etc, there are some packages that need to be keyworded to compile against glibc-2.7. I'll post you a list of what I've found during my emerge -e world as soon as I can figure them out (I accidentally resorted package.keywords with a script I'm writing... -.-").
Note also that when I say they're "stable" I'm compiling with these flags:

CFLAGS="-march=native -O2 -pipe"
CXXFLAGS="${CFLAGS}"
CPPFLAGS="${CFLAGS}"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,--enable-new-dtags,--hash-style=both"

and I've never had problems of any sort with them. Don't blame me if a config like this one:
#CFLAGS="-march=k8 -O2 -fgcse-after-reload -pipe -fgcse-sm -fgcse-las -ftree-loop-im -ftree-loop-ivcanon -fmodulo-sched -freschedule-modulo-scheduled-loops -fivopts -fpeel-loops -fweb -fsched-spec-load -msse3 -falign-functions=4 -falign-loops=4"
crashes 1 package out of 3 during compile or at runtime. It's normal. (BTW, I tried that config, I know what I'm saying

)
See the ChangeLog at thread start for full overview. Still thanks to all, let me know comments, critics and whatelse!
/me goes to study for exams

Cheers

kernelOfTruth · Post by **kernelOfTruth** » Fri Jun 27, 2008 11:50 pm

me goes to study for exams

Cheers

good luck !

thanks for this release ! amazing !
you did both amd64 + x86

rock'n'roll, I'll test & use these CDs in the next months and let you know if anything goes wrong (hopefully not

)

it's great to have up-to-date liveCDs again after a long time ...

sPHERE911 · Post by **sPHERE911** » Thu Jul 03, 2008 6:48 pm

I was trying to install gentoo with your CD for AMD64, but Im unsure about how I should compile the kernel for use with reiser4? I know genkernel doesn't support reiser4, so I tried the manual method, but there are SO many settings and I dont have a clue of which to select, also I couldn't find the setting for reiser4 support in the "filesystems" menu in menuconfig. Can anyone please explain to me? Step by step?:D

If it helps, here's my hardware:

AMD Athlon 64 3200+
XFX GeForce 6600GT PCI-E
2x512mb OCZ PC3200 RAM
SoundBlaster Live! 24-bit PCI soundcard
USB Mouse, PS2 keyboard
Asus A8N-SLI Deluxe MB
250GB S-ATA hdd

Much appreciated :)

Need4Speed · Post by **Need4Speed** » Thu Jul 03, 2008 7:18 pm

sPHERE911 wrote:I was trying to install gentoo with your CD for AMD64, but Im unsure about how I should compile the kernel for use with reiser4? I know genkernel doesn't support reiser4, so I tried the manual method, but there are SO many settings and I dont have a clue of which to select, also I couldn't find the setting for reiser4 support in the "filesystems" menu in menuconfig. Can anyone please explain to me? Step by step?:D

Reiser4 is not currently in the kernel. You must use a patched version. See http://gentoo-wiki.com/HOWTO_Reiser4_Wi ... oo-Sources

Neo2 · Post by **Neo2** » Thu Jul 03, 2008 7:20 pm

My liveCD enables support for reiser4 for the CD only. It's not that by compiling a kernel while chrooted you will gain automatic reiser4 support =)
For that you need to use a custom kernel ebuild patched with reiser4. The ebuilds I use are available at http://neo2.servizi-web.net/overlay/sys ... 4-sources/. Just download it, place it your overlay, digest it and emerge it.
ex:

Code: Select all

cd /my/overlay
mkdir -p sys-kernel/reiser4-sources
cd sys-kernel/reiser4-sources
wget http://neo2.servizi-web.net/overlay/sys-kernel/reiser4-sources/reiser4-sources-2.6.25-r6.ebuild
ebuild reiser4-sources-2.6.25-r6.ebuild digest
emerge reiser4-sources

Obviously, "/my/overlay" is the same as PORTDIR_OVERLAY="/my/overlay" in make.conf
Anyway, once configured the kernel, genkernel works just fine for building the kernel

sPHERE911 · Post by **sPHERE911** » Thu Jul 03, 2008 7:28 pm

Yes I know this, I digested & emerged your reiser4 kernel after creating the overlay-dir, but I mean, what options to select in the menuconfig after doing the "make menuconfig"? I can't select "reiser4" in the filesystems category!

From the Reiser4+Gentoo-sources wiki:

"reiser4 has a few requirements:

* You must enable (say Y to build it in) deflate under Cryptographic Options!
* You must enable "Prompt for development and/or incomplete code/drivers" in the "Code maturity level" options

If you do not do that, you will not even have the option of enabling the reiser4 filesystem in the kernel config."

I dont understand this, where do I say "Y" to build in "deflate" in the cryptographic options, and "Promp for development and/or incomplete code/drivers" ?

Neo2 · Post by **Neo2** » Thu Jul 03, 2008 7:36 pm

Okay, I've had a look at reiser4 DEPENDS. You need to select at least

Code: Select all

General setup ---> [*] Prompt for development and/or incomplete code/drivers
[*] Enable the block layer  --->

After that it should become visibile right over the "Reiserfs support" option.

[edit] Actually deflate gets automatically selected when you select "reiser4 support" [/edit]

kernelOfTruth · Post by **kernelOfTruth** » Thu Jul 03, 2008 7:36 pm

search for reiser4, deflate, whatever via:

Code: Select all

shift + /

sPHERE911 · Post by **sPHERE911** » Thu Jul 03, 2008 7:50 pm

Thanks guys, appreciate it. :) Will try it later (using Arch64 atm, dont like it at all compared to gentoo)