Official thread: "zen-sources" - Part 8

Message

Etrnls · Post by **Etrnls** » Thu Oct 01, 2009 7:04 pm

Could you please add a live ebuild for zen-sources in zen-overlay ?

kernelOfTruth · Post by **kernelOfTruth** » Thu Oct 01, 2009 7:29 pm

wrc1944 wrote:I installed zen2 last night on another box, but today keep getting this:
Code: Select all
amd64 src # git clone git://zen-sources.org/zen/zen-stable.git zen-sources
Initialized empty Git repository in /usr/src/zen-sources/.git/
fatal: Unable to look up zen-sources.org (port 9418) (Temporary failure in name resolution)
amd64 src # git clone git://zen-sources.org/zen/zen-stable.git zen-sources
Initialized empty Git repository in /usr/src/zen-sources/.git/
fatal: read error: Connection reset by peer
amd64 src #
Is it just something on your end- like updating to a new version, or a server problem? Nothing seems wrong on this system.

the git-servers seemingly are undergoing maintenance

http://git.zen-sources.org wrote: 404 - Not Found

rahulthewall · Post by **rahulthewall** » Thu Oct 01, 2009 8:51 pm

tranquilcool wrote:
broch wrote:I can confirm that with 2.6.31-zen2
suspend to RAM fails
in contrast patched with bfs vanilla 2.6.31.1 does not have problems with suspend-to-ram, so maybe this is not a bfs fault?

additionally, when shutting down kernel generates error regarding "mounting/read only"
system shuts down but error is a little bit worrying.

again, this is specific to zen, bfs patched vanilla *.31.1 works
shutdown errors here too.

Yeah, I am getting these again. I thought I had solved them - but no, they are still there. Suspend to RAM, Shutdown - nothing works.

cheater1034 · Post by **cheater1034** » Fri Oct 02, 2009 12:38 am

you guys are killing me

i know git repo is broken atm, but when it's back up you will have bfs 300 and such

anyway:
1. vbox problems have been fixed for a few days
2. shutdown errors are fixed in bfs 300
3. problems should be fixed in bfs 300 (s2ram, etc) - so dont compare the 240 in the zen you're using to the 300 on con's website

4. live-ebuild is a little more complex for zen-sources - but i see it as somewhat pointless, considering you can install a dummy ebuild and clone it yourself (the ebuild can't maintain/update the git tree for you)

DigitalCorpus · Post by **DigitalCorpus** » Fri Oct 02, 2009 4:08 am

And I'm back. I know no one missed me. Anyhow a corrupted .bash_history file under my normal login was causing the crashes.

I feel I have some extraneous options in my kernel config. Once I clean those up I'll jump back on board with all of you and BFS testing. Since I had the opposite reaction with that scheduler that all of you have, I want to see what is wrong in my config. If you care to help out, please have a look here.

@cheater1034
Thank you for that heads up about lattency testing. Do I need to enable anything kernel side for that? I run a minimal one if I can get away with it.

cheater1034 wrote:If you're running a server/desktop i would disable dynticks. If you're running X on the server/desktop, set HZ=1000
voluntary preemption is junk, don't use it (you can't use it with bfs since it's disabled in Kconfig when bfs is selected. Full preemption is probably your best bet, no preemption is acceptable too - but if you are running X i'd suggest it.

I have a quad core. I thought it stood to reason that the selected frequency x # of cores/procs would be the real measure of ticks on a system. I'm not looking to run at the very lowest latencies on my system. I'm not looking to run at maximum bandwidth either. Since I can select a range from 100 to 1000 Hz in the Zen config with it seems that the author(s) share this same paradigm (of which you are one, correct?).

What would be a good middle ground? 250Hz, which is effectively 1000Hz? 216Hz (suggested for a quad core), effectively the "magic" 864Hz?

I understand dynamic ticks means that timer interrupts are only triggered when needed, thus can be power saving. I never had a performance issue with them on, nor noticed anything with them off. I have a WattsUp Pro, but never dawned on me to check consumption at the plug iirc. I just find it odd that my system had better latency with dynamic ticks, no preemption, 108Hz, CFS under 2.6.30-zen1 than with what is suggested here. I'll test more soon as I'm about to have my work weekend soon.

pappy_mcfae · Post by **pappy_mcfae** » Fri Oct 02, 2009 7:47 am

Whereas core-too liked bfs240, it loves bfs300! I've never seen gcc compile so quickly. It has even survived openGL screensavers going on and off while doing an emerge -aev system and serving A Farewell To Kings by Rush to pappy-lap. All that, three and a half hours of operation, and still running. No lockups. Yay! This might be licked.

Now, onto a bug. It seems to stop certain temperature sensor devices from operating; in specific, W83627EHF on core-too, and W8362HF on bam-bam. Neither module shows up under lsmod. The CPU core sensor is unaffected.

Now to find out if BFS is all it's supposed to be.

Blessed be!
Pappy

DigitalCorpus · Post by **DigitalCorpus** » Fri Oct 02, 2009 9:22 am

Okay, so for me, I have slightly different usage patterns where I require performance. I do a lot of HD TV recording and encoding. I bumped my RAM up to 8GB for this reason to let the kernel keep things cached and to use it as a scratch disk. Anyhow, I have a fairly large amount of disk I/O and though filesystems and I/O schedulers play a big part in performance here, I'm seeing that CPU task scheduler is playing a roll in disk I/O as well.

This is one test I run to help determine if BFS is better, the same, or worse than CFS.

Code: Select all

cp /disk1/$(largefile) /disk2/ $ sleep 3
time echo 1 > /proc/sys/vm/drop_caches
time sleep .2
time echo 1 > /proc/sys/vm/drop_caches
time sleep .2

I tested this with 4 kernel configs so far. CFS w/o preemption and w/ full preemption & BFS w/o preemption and w/ full preemption. I also went about trying to do simple tasks such as tab completion. I won't post numbers because they were not that easy to produce, but I did learn that 2.6.31-zen1 is baaaaaaad for business under these testing circumstances on my configuration. Whether it's XFS's fault, BFQ's fault, or just the kernel (that is where my money is at), but Trying to run a simple command like editing a file in nano, executing df, tab completion a path, no matter which of the four configurations, I experienced a delay from ~7 seconds to as long as it took to finish copying the file, ~2.25 minutes, before my input was processed.

Overall, CFS usually sprang to life first (initial response times under 1 second but did take as long as about 5 at times) , but both had such high initial latencies though. I blame the kernel revision and am waiting until .32 since the evens always seem to work for me. I'm going to test CFQ over BFQ as my I/O scheduler and SLUB instead of SLQB to see if either of those have an effect. In the past, the type of system load has made a noticeable difference for me with those changes. This warrants investigation since I should've seen 90 to 105 MB/sec consistently through the transfer, but it jumped around from 5 to 105 MB/sec averaging at about 60MB/sec. That is about a 40% performance deficit for me.

Now I did test straight CPU utilization with what everyone else has done here, kernel compilation. All four cores were 99 to 100% utilized for me under BFS which resulted in compilation times of 1 min 29 sec on average whereas CFS resulted in 1 min 35 sec on average. Changing /proc/sys/kernel/rr_interval from 18 to 4 or to 72 caused no deviation from 1:29 compilation time under BFS. Then again, there was only one thread per core.

For me, boot times are 1 to 3 seconds slower with CFS performing consistently at 18 seconds and BFS bouncing between 19 and 21. I'm booting from a gzip cryptcompress Reiser4 root partition.

I also ran an 3 simultaneous low-res encodings of 1 hour of 720 MPEG-2 to H.264 content (last night's episode of Bones actually) Usually I get ~85 to 95% CPU utilization with 5 threads allotted to each mencoder instance. This usually takes about 22 to 26 minutes for each one to finish. With BFS, I saw nothing less than 100% utilization. Now finishing times are from 16 minutes to 22 minutes at the same priorities.

Thus far, the lesson I've learned is that as long as I don't have a disk bound task, BFS has a solid 7% to 20% performance increase due to complete CPU utilization.

Mr. Tao · Post by **Mr. Tao** » Sat Oct 03, 2009 8:10 am

DigitalCorpus wrote:I'm going to test CFQ over BFQ as my I/O scheduler and SLUB instead of SLQB to see if either of those have an effect.

I have zen with BFQ on different dektops and notebooks installed. All but one desktop behaved great. It felt laggy and CPU kernel usage (red bar in htop) was unusually high. I compared kernel configs of my machines and it actually was particular SLAB allocator responsible for lack of responsiveness.

DigitalCorpus · Post by **DigitalCorpus** » Sat Oct 03, 2009 8:53 am

Mr. Tao wrote:
DigitalCorpus wrote:I'm going to test CFQ over BFQ as my I/O scheduler and SLUB instead of SLQB to see if either of those have an effect.
I have zen with BFQ on different dektops and notebooks installed. All but one desktop behaved great. It felt laggy and CPU kernel usage (red bar in htop) was unusually high. I compared kernel configs of my machines and it actually was particular SLAB allocator responsible for lack of responsiveness.

Which ones worked best for you?

kernelOfTruth · Post by **kernelOfTruth** » Sat Oct 03, 2009 10:08 am

the git repo-server is live again !

edit:

git pull
fatal: The remote end hung up unexpectedly

rahulthewall · Post by **rahulthewall** » Sat Oct 03, 2009 10:19 am

kernelOfTruth wrote:the git repo-server is live again !

edit:

git pull
fatal: The remote end hung up unexpectedly

Yup, we should be more patient. (I have been issuing that command every half hour

)

cheater1034 · Post by **cheater1034** » Sat Oct 03, 2009 2:10 pm

Only the http is working for a reason that I and about everybody else strongly disagrees with.

It should be back up today, if not we'll have some major beef

Or you could use gitweb to diff a patch if you want it (you can find that tutorial on the FAQ where it talks about the download links)

wrc1944 · Post by **wrc1944** » Sat Oct 03, 2009 3:15 pm

I had been getting all the other problems with git, but today I get this one, if it helps:

Code: Select all

amd64 src # git clone git://zen-sources.org/zen/zen-stable.git zen-sources
Initialized empty Git repository in /usr/src/zen-sources/.git/
zen-sources.org[0: 134.155.50.250]: errno=Connection refused
fatal: unable to connect a socket (Connection refused)

I also tried the diff patch method, but always get "No differences found."

Just tried again, and I'm back to:

Code: Select all

initialized empty Git repository in /usr/src/zen-sources/.git/
fatal: The remote end hung up unexpectedly

cheater1034 · Post by **cheater1034** » Sat Oct 03, 2009 5:44 pm

GIT is back up but the URLS HAVE CHANGED - so check out git.zen-sources.org for the new ones (i think the only change was removing /zen/

kernelOfTruth · Post by **kernelOfTruth** » Sat Oct 03, 2009 6:45 pm

cheater1034 wrote:GIT is back up but the URLS HAVE CHANGED - so check out git.zen-sources.org for the new ones (i think the only change was removing /zen/

thanks !

could you guys please replace reiser4 from mmotm with reiser4 from kernel.org or vanilla-patches in order to get a working reiser4 again ?

many many thanks in advance !

kernelOfTruth · Post by **kernelOfTruth** » Sat Oct 03, 2009 6:48 pm

ok guys,

when using stable change

[remote "origin"]
url = git://zen-sources.org/zen/zen-stable.git
fetch = +refs/heads/*:refs/remotes/origin/*

to

[remote "origin"]
url = git://zen-sources.org/zen-stable.git
fetch = +refs/heads/*:refs/remotes/origin/*

when using (un)stable change

[remote "origin"]
url = git://zen-sources.org/zen/zen.git
fetch = +refs/heads/*:refs/remotes/origin/*

to

[remote "origin"]
url = git://zen-sources.org/zen.git
fetch = +refs/heads/*:refs/remotes/origin/*

in the file

.git/config

ponciarello · Post by **ponciarello** » Sat Oct 03, 2009 7:06 pm

all ok (vbox too) with new git repo and bfs300

Ant P. · Post by **Ant P.** » Sat Oct 03, 2009 7:15 pm

The keyboard/mouse no longer locks up when I try to run nexuiz. Just need to wait for the radeon driver to catch up now so I can play UT2k4 too...

wrc1944 · Post by **wrc1944** » Sat Oct 03, 2009 7:30 pm

Code: Select all

git clone git://zen-sources.org/zen-stable.git zen-sources

is apparently working. Is this the 2.6.31 with bfs300, bfq, or what? I'm having difficulty figuring out what gitweb actually is. AFAIK, zen-stable.git is newer that zen.git, which I thought from the sept.24 notice was the other way around.

I still don't understand why this git method is better for the end users of zen-sources, unless it is somewhat analogous to how kde-svn works, which I am very familiar with, and have used for 2 years.

EDIT: A popup just appeared saying I requested to uncompress zen (I hadn't), although only 123mb had been downloaded. I looked in my terminal, and it said done, but obviously wasn't, and when I typed in the first character it repeated many weird characters, and I had to end the terminal session.

I then removed /usr/src/zen-sources, and tried again, and get this:

Code: Select all

amd64 src #git clone git://zen-sources.org/zen-stable.git zen-sources
bash: clone: command not found
amd64 src #

EDIT2: My mistake- had a typo in the command.

Download is now started.
This is really getting frustrating and consuming way too much time, all simply to patch a specific kernel with the great zen patches, which use to be a very simple matter. I have to echo another zen users comment (I think it was Jupiter1TX) that this has become a huge "mind fuck".

I can follow directions, but I must be missing something obvious, because not much on gitweb has ever worked for me. I can't understand that when trying to download patches it tries to download in the browser first, and even when you use "raw" as instructed, it still downloads in a browser window (then you apparently have two downloads of the same file going on), then you have to convert it to a simple text file. If this is suppose to be a better way of getting the zen patch to the end user, I'm missing the point. If the devs already have the patches ready to download, I can't see how much trouble it would be to simply put them on the download page, such as 2.6.xx.x-zen1, zen2, etc., as before.

All the instructions, explanations, and documentations apparently leave out vital info, or are worded in such a way that english only users can miss important points. I get the impression that english must not be the first language of the person(s) producing the zen doc/info. (Not that I could ever translate into another language correctly)

Anyway, please don't get me wrong- I'm not complaining, and really much appreciate those making the zen patches available for us, and really wish to use them. I'm just getting frustrated at the problems I'm having getting a grasp of this git stuff.

ponciarello · Post by **ponciarello** » Sat Oct 03, 2009 8:12 pm

wrc1944 wrote:This is really getting frustrating and consuming way too much time, all simply to patch a specific kernel with the great zen patches, which use to be a very simple matter. I have to echo another zen users comment (I think it was Jupiter1TX) that this has become a huge "mind fuck".

you can wait for stable patches on http://zen-sources.org or you can also try a snapshot of the full source (the snapshot link on the right of the last commit, the first counting from beginning of page -

- here).

wrc1944 · Post by **wrc1944** » Sat Oct 03, 2009 8:44 pm

ponciarello,
Thanks for the info, but I've already tried it many times- nothing happens. Just tried it again, from your link. With snapshot, it tries to download a snapshot.htm file, and says "starting," but nothing happens, no matter how long you wait. I've never heard of a .htm patch file before.

On the "stable patches page," it hasn't changed for days from zen1, and I read here that they are discontinuing this, and no longer will post regular kernel patches in the near future.

cheater1034 · Post by **cheater1034** » Sat Oct 03, 2009 9:00 pm

Guys, guys, guys

settle down:
- zen.git isn't simply newer than zen-stable.git ........ it is the UNSTABLE zen, meaning -rc kernels, etc. so it's in the PROCESS of being rebased to 2.6.32, so naturally zen-stable.git is newer than zen.git if zen.git hasnt been updated since zen0
- you don't need patches on the downloads page - you can diff patches on gitweb, or use git itself (and diff patches actually using git?) - i wrote a tutorial on the FAQ page at zen-sources.org
- Cloning GIT is not like getting a patch, you can update git of course - you should only have to clone once as long as the tree is on your system, update with git pull

Jeez you guys make this harder than it has to be, of course zen is updated too guys - don't need to ask about bfs 300, when many of us are in #ck all the time when con makes the releases, if you are unsure pull and check the log (git log), and look for the commit message (search is / in less)

Always, if you simply want a patch, go to git.zen-sources.org, click on the git tree you want (zen.git/zen-stable.git) and read the tutorial on the FAQ on zen-sources.org

Mr. Tao · Post by **Mr. Tao** » Sat Oct 03, 2009 10:17 pm

DigitalCorpus wrote:
Mr. Tao wrote:I have zen with BFQ on different dektops and notebooks installed. All but one desktop behaved great. It felt laggy and CPU kernel usage (red bar in htop) was unusually high. I compared kernel configs of my machines and it actually was particular SLAB allocator responsible for lack of responsiveness.
Which ones worked best for you?

I have SLQB selected now. The problematic one was, if I'm not mistaken, SLOB.

wrc1944 · Post by **wrc1944** » Sat Oct 03, 2009 10:35 pm

cheater1034,
Thanks for the info. I really appreciate the time you spend trying to help.
I've read all the FAQ's and the patch tutorial several times, but getting patches still seems impossible. Things don't seem to happen as described, but mostly nothing happens.

I'll carefully review all the stuff once again, as I obviously must be misunderstanding something. I think you're right- I must be making it more complicated than it really is.

As it is, I just succeeded in cloning zen-sources, and compiled/installed zen2 OK. Haven't rebooted yet (currently booted to a vanilla 2.6.31-bfs300 I compiled yesterday).

EDIT: Reboot to zen2 is OK- very responsive and snappy. Have bfs, bfq, CUSTOM_CFLAGS="-march=native", SLQB, and HZ_1000=y.

Hmmm. On shutdown ,I got the hanging problem at Remounting / Read only. No big deal, I guess.

I also saw an option to enable something like SCHED_ISO X for X sever, but can find nothing much explaining it anywhere. It is mentioned on the Arch forums. Is this a recommended option with zen?

So- if I understand correctly, when/if zen3, zen4, etc. based on 2.6.31 come out, all I'll need to do is cd to /usr/src/zen-sources, do a git pull, and recompile from the same directory? (sort of like doing a complete kde-svn update from the kde-testing overlay?)

mantoo · Post by **mantoo** » Sat Oct 03, 2009 11:12 pm

cheater1034 wrote:Guys, guys, guys

settle down:
- zen.git isn't simply newer than zen-stable.git ........ it is the UNSTABLE zen, meaning -rc kernels, etc. so it's in the PROCESS of being rebased to 2.6.32, so naturally zen-stable.git is newer than zen.git if zen.git hasnt been updated since zen0
- you don't need patches on the downloads page - you can diff patches on gitweb, or use git itself (and diff patches actually using git?) - i wrote a tutorial on the FAQ page at zen-sources.org
- Cloning GIT is not like getting a patch, you can update git of course - you should only have to clone once as long as the tree is on your system, update with git pull

Jeez you guys make this harder than it has to be, of course zen is updated too guys - don't need to ask about bfs 300, when many of us are in #ck all the time when con makes the releases, if you are unsure pull and check the log (git log), and look for the commit message (search is / in less)

Always, if you simply want a patch, go to git.zen-sources.org, click on the git tree you want (zen.git/zen-stable.git) and read the tutorial on the FAQ on zen-sources.org

^^

sry for that post

but u´re really doing good job

thanks!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[edit]

bfs 300 running 1st time kinda stable 'til now ("emerge -avuDN world": ~60 pkgs.) after never really being able to use it due to issues with reiser (which i banned now from my box) , instability, using bfs is like a felt overall upgrade of all my hardware. incredible

cant say anythingabput shutdown-errors/locks as my box is rather supposed to run than to halt/reboot.

[edit2]

yaaaay, my box definitely likes BFS 300, smoothly halting and rebooting