Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
[RFC/RFT PATCH v3] sched: automated per tty task groups
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3  Next  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
Lori
Apprentice
Apprentice


Joined: 30 Mar 2004
Posts: 265
Location: Barcelona, Spain

PostPosted: Tue Nov 16, 2010 9:24 am    Post subject: [RFC/RFT PATCH v3] sched: automated per tty task groups Reply with quote

Any kernel in portage with this patch included?

[RFC/RFT PATCH v3] sched: automated per tty task groups

Any of them plans to include it in the near future?
_________________
"The hunt is sweeter then the kill."
Registered Linux User #176911
Back to top
View user's profile Send private message
Shining Arcanine
Veteran
Veteran


Joined: 24 Sep 2009
Posts: 1110

PostPosted: Tue Nov 16, 2010 1:13 pm    Post subject: Reply with quote

It would probably need to be applied manually. I assume that it will be included when it is accepted into the Linux kernel. I just tried applying it to the 2.6.35.8 kernel and it does not apply cleanly.
Back to top
View user's profile Send private message
Lori
Apprentice
Apprentice


Joined: 30 Mar 2004
Posts: 265
Location: Barcelona, Spain

PostPosted: Tue Nov 16, 2010 1:20 pm    Post subject: Reply with quote

Shining Arcanine wrote:
I just tried applying it to the 2.6.35.8 kernel and it does not apply cleanly.


It was written against 2.6.37-rc2. Maybe the zen-kernel guys will backport it before it hits 2.6.38.
_________________
"The hunt is sweeter then the kill."
Registered Linux User #176911
Back to top
View user's profile Send private message
Shining Arcanine
Veteran
Veteran


Joined: 24 Sep 2009
Posts: 1110

PostPosted: Tue Nov 16, 2010 1:56 pm    Post subject: Reply with quote

Lori wrote:
Shining Arcanine wrote:
I just tried applying it to the 2.6.35.8 kernel and it does not apply cleanly.


It was written against 2.6.37-rc2. Maybe the zen-kernel guys will backport it before it hits 2.6.38.


I think I read on Phoronix that they backported it to the 2.6.36 tree.

Anyway, I just installed 2.6.37-rc2 and installed this patch just to try it. Linus Torvalds was serious. My web browser actually seems faster. In addition, KDE's desktop effects appear to be more fluid than they were before.
Back to top
View user's profile Send private message
wazoo42
Apprentice
Apprentice


Joined: 13 Apr 2004
Posts: 165

PostPosted: Tue Nov 16, 2010 7:57 pm    Post subject: Reply with quote

It also applied cleanly to git-sources-2.6.37-rc1-r12. As noted by everybody, the computer is now noticeably more responsive.
Back to top
View user's profile Send private message
cruzki123
Apprentice
Apprentice


Joined: 16 May 2008
Posts: 233

PostPosted: Tue Nov 16, 2010 8:31 pm    Post subject: Reply with quote

I have tried to patch against vanilla 2.6.37_rc2 without lock. Can anyone provide my the file patch (copy paste the one email didn't work... but I'm not sure if I have doing it well)
Back to top
View user's profile Send private message
wazoo42
Apprentice
Apprentice


Joined: 13 Apr 2004
Posts: 165

PostPosted: Tue Nov 16, 2010 8:42 pm    Post subject: Reply with quote

Here's the one I used. I saved it to sched.patch and then ran the following in /usr/src/linux.

Code:

# cat sched.patch | patch -p1


Code:

Index: linux-2.6/include/linux/sched.h
===================================================================
--- linux-2.6.orig/include/linux/sched.h
+++ linux-2.6/include/linux/sched.h
@@ -509,6 +509,8 @@ struct thread_group_cputimer {
    spinlock_t lock;
 };
 
+struct autogroup;
+
 /*
  * NOTE! "signal_struct" does not have it's own
  * locking, because a shared signal_struct always
@@ -576,6 +578,9 @@ struct signal_struct {
 
    struct tty_struct *tty; /* NULL if no tty */
 
+#ifdef CONFIG_SCHED_AUTOGROUP
+   struct autogroup *autogroup;
+#endif
    /*
     * Cumulative resource counters for dead threads in the group,
     * and for reaped dead child processes forked by this group.
@@ -1931,6 +1936,20 @@ int sched_rt_handler(struct ctl_table *t
 
 extern unsigned int sysctl_sched_compat_yield;
 
+#ifdef CONFIG_SCHED_AUTOGROUP
+extern unsigned int sysctl_sched_autogroup_enabled;
+
+extern void sched_autogroup_create_attach(struct task_struct *p);
+extern void sched_autogroup_detach(struct task_struct *p);
+extern void sched_autogroup_fork(struct signal_struct *sig);
+extern void sched_autogroup_exit(struct signal_struct *sig);
+#else
+static inline void sched_autogroup_create_attach(struct task_struct *p) { }
+static inline void sched_autogroup_detach(struct task_struct *p) { }
+static inline void sched_autogroup_fork(struct signal_struct *sig) { }
+static inline void sched_autogroup_exit(struct signal_struct *sig) { }
+#endif
+
 #ifdef CONFIG_RT_MUTEXES
 extern int rt_mutex_getprio(struct task_struct *p);
 extern void rt_mutex_setprio(struct task_struct *p, int prio);
Index: linux-2.6/kernel/sched.c
===================================================================
--- linux-2.6.orig/kernel/sched.c
+++ linux-2.6/kernel/sched.c
@@ -78,6 +78,7 @@
 
 #include "sched_cpupri.h"
 #include "workqueue_sched.h"
+#include "sched_autogroup.h"
 
 #define CREATE_TRACE_POINTS
 #include <trace/events/sched.h>
@@ -605,11 +606,14 @@ static inline int cpu_of(struct rq *rq)
  */
 static inline struct task_group *task_group(struct task_struct *p)
 {
+   struct task_group *tg;
    struct cgroup_subsys_state *css;
 
    css = task_subsys_state_check(p, cpu_cgroup_subsys_id,
          lockdep_is_held(&task_rq(p)->lock));
-   return container_of(css, struct task_group, css);
+   tg = container_of(css, struct task_group, css);
+
+   return autogroup_task_group(p, tg);
 }
 
 /* Change a task's cfs_rq and parent entity if it moves across CPUs/groups */
@@ -2006,6 +2010,7 @@ static void sched_irq_time_avg_update(st
 #include "sched_idletask.c"
 #include "sched_fair.c"
 #include "sched_rt.c"
+#include "sched_autogroup.c"
 #include "sched_stoptask.c"
 #ifdef CONFIG_SCHED_DEBUG
 # include "sched_debug.c"
@@ -7979,7 +7984,7 @@ void __init sched_init(void)
 #ifdef CONFIG_CGROUP_SCHED
    list_add(&init_task_group.list, &task_groups);
    INIT_LIST_HEAD(&init_task_group.children);
-
+   autogroup_init(&init_task);
 #endif /* CONFIG_CGROUP_SCHED */
 
 #if defined CONFIG_FAIR_GROUP_SCHED && defined CONFIG_SMP
@@ -8509,15 +8514,11 @@ void sched_destroy_group(struct task_gro
 /* change task's runqueue when it moves between groups.
  *   The caller of this function should have put the task in its new group
  *   by now. This function just updates tsk->se.cfs_rq and tsk->se.parent to
- *   reflect its new group.
+ *   reflect its new group.  Called with the runqueue lock held.
  */
-void sched_move_task(struct task_struct *tsk)
+void __sched_move_task(struct task_struct *tsk, struct rq *rq)
 {
    int on_rq, running;
-   unsigned long flags;
-   struct rq *rq;
-
-   rq = task_rq_lock(tsk, &flags);
 
    running = task_current(rq, tsk);
    on_rq = tsk->se.on_rq;
@@ -8538,7 +8539,15 @@ void sched_move_task(struct task_struct
       tsk->sched_class->set_curr_task(rq);
    if (on_rq)
       enqueue_task(rq, tsk, 0);
+}
 
+void sched_move_task(struct task_struct *tsk)
+{
+   struct rq *rq;
+   unsigned long flags;
+
+   rq = task_rq_lock(tsk, &flags);
+   __sched_move_task(tsk, rq);
    task_rq_unlock(rq, &flags);
 }
 #endif /* CONFIG_CGROUP_SCHED */
Index: linux-2.6/kernel/fork.c
===================================================================
--- linux-2.6.orig/kernel/fork.c
+++ linux-2.6/kernel/fork.c
@@ -174,8 +174,10 @@ static inline void free_signal_struct(st
 
 static inline void put_signal_struct(struct signal_struct *sig)
 {
-   if (atomic_dec_and_test(&sig->sigcnt))
+   if (atomic_dec_and_test(&sig->sigcnt)) {
+      sched_autogroup_exit(sig);
       free_signal_struct(sig);
+   }
 }
 
 void __put_task_struct(struct task_struct *tsk)
@@ -904,6 +906,7 @@ static int copy_signal(unsigned long clo
    posix_cpu_timers_init_group(sig);
 
    tty_audit_fork(sig);
+   sched_autogroup_fork(sig);
 
    sig->oom_adj = current->signal->oom_adj;
    sig->oom_score_adj = current->signal->oom_score_adj;
Index: linux-2.6/drivers/tty/tty_io.c
===================================================================
--- linux-2.6.orig/drivers/tty/tty_io.c
+++ linux-2.6/drivers/tty/tty_io.c
@@ -3160,6 +3160,7 @@ static void __proc_set_tty(struct task_s
    put_pid(tsk->signal->tty_old_pgrp);
    tsk->signal->tty = tty_kref_get(tty);
    tsk->signal->tty_old_pgrp = NULL;
+   sched_autogroup_create_attach(tsk);
 }
 
 static void proc_set_tty(struct task_struct *tsk, struct tty_struct *tty)
Index: linux-2.6/kernel/sched_autogroup.h
===================================================================
--- /dev/null
+++ linux-2.6/kernel/sched_autogroup.h
@@ -0,0 +1,18 @@
+#ifdef CONFIG_SCHED_AUTOGROUP
+
+static void __sched_move_task(struct task_struct *tsk, struct rq *rq);
+
+static inline struct task_group *
+autogroup_task_group(struct task_struct *p, struct task_group *tg);
+
+#else /* !CONFIG_SCHED_AUTOGROUP */
+
+static inline void autogroup_init(struct task_struct *init_task) {  }
+
+static inline struct task_group *
+autogroup_task_group(struct task_struct *p, struct task_group *tg)
+{
+   return tg;
+}
+
+#endif /* CONFIG_SCHED_AUTOGROUP */
Index: linux-2.6/kernel/sched_autogroup.c
===================================================================
--- /dev/null
+++ linux-2.6/kernel/sched_autogroup.c
@@ -0,0 +1,140 @@
+#ifdef CONFIG_SCHED_AUTOGROUP
+
+unsigned int __read_mostly sysctl_sched_autogroup_enabled = 1;
+
+struct autogroup {
+   struct kref      kref;
+   struct task_group   *tg;
+};
+
+static struct autogroup autogroup_default;
+
+static void autogroup_init(struct task_struct *init_task)
+{
+   autogroup_default.tg = &init_task_group;
+   kref_init(&autogroup_default.kref);
+   init_task->signal->autogroup = &autogroup_default;
+}
+
+static inline void autogroup_destroy(struct kref *kref)
+{
+   struct autogroup *ag = container_of(kref, struct autogroup, kref);
+   struct task_group *tg = ag->tg;
+
+   kfree(ag);
+   sched_destroy_group(tg);
+}
+
+static inline void autogroup_kref_put(struct autogroup *ag)
+{
+   kref_put(&ag->kref, autogroup_destroy);
+}
+
+static inline struct autogroup *autogroup_kref_get(struct autogroup *ag)
+{
+   kref_get(&ag->kref);
+   return ag;
+}
+
+static inline struct autogroup *autogroup_create(void)
+{
+   struct autogroup *ag = kmalloc(sizeof(*ag), GFP_KERNEL);
+
+   if (!ag)
+      goto out_fail;
+
+   ag->tg = sched_create_group(&init_task_group);
+   kref_init(&ag->kref);
+
+   if (!(IS_ERR(ag->tg)))
+      return ag;
+
+out_fail:
+   if (ag) {
+      kfree(ag);
+      WARN_ON(1);
+   } else
+      WARN_ON(1);
+
+   return autogroup_kref_get(&autogroup_default);
+}
+
+static inline struct task_group *
+autogroup_task_group(struct task_struct *p, struct task_group *tg)
+{
+   int enabled = ACCESS_ONCE(sysctl_sched_autogroup_enabled);
+
+   enabled &= (tg == &root_task_group);
+   enabled &= (p->sched_class == &fair_sched_class);
+   enabled &= (!(p->flags & PF_EXITING));
+
+   if (enabled)
+      return p->signal->autogroup->tg;
+
+   return tg;
+}
+
+static void
+autogroup_move_group(struct task_struct *p, struct autogroup *ag)
+{
+   struct autogroup *prev;
+   struct task_struct *t;
+   struct rq *rq;
+   unsigned long flags;
+
+   rq = task_rq_lock(p, &flags);
+   prev = p->signal->autogroup;
+   if (prev == ag) {
+      task_rq_unlock(rq, &flags);
+      return;
+   }
+
+   p->signal->autogroup = autogroup_kref_get(ag);
+   __sched_move_task(p, rq);
+   task_rq_unlock(rq, &flags);
+
+   rcu_read_lock();
+   list_for_each_entry_rcu(t, &p->thread_group, thread_group) {
+      sched_move_task(t);
+   }
+   rcu_read_unlock();
+
+   autogroup_kref_put(prev);
+}
+
+void sched_autogroup_create_attach(struct task_struct *p)
+{
+   struct autogroup *ag = autogroup_create();
+
+   autogroup_move_group(p, ag);
+   /* drop extra refrence added by autogroup_create() */
+   autogroup_kref_put(ag);
+}
+EXPORT_SYMBOL(sched_autogroup_create_attach);
+
+/* currently has no users */
+void sched_autogroup_detach(struct task_struct *p)
+{
+   autogroup_move_group(p, &autogroup_default);
+}
+EXPORT_SYMBOL(sched_autogroup_detach);
+
+void sched_autogroup_fork(struct signal_struct *sig)
+{
+   sig->autogroup = autogroup_kref_get(current->signal->autogroup);
+}
+
+void sched_autogroup_exit(struct signal_struct *sig)
+{
+   autogroup_kref_put(sig->autogroup);
+}
+
+static int __init setup_autogroup(char *str)
+{
+   sysctl_sched_autogroup_enabled = 0;
+
+   return 1;
+}
+
+__setup("noautogroup", setup_autogroup);
+#endif
Index: linux-2.6/kernel/sysctl.c
===================================================================
--- linux-2.6.orig/kernel/sysctl.c
+++ linux-2.6/kernel/sysctl.c
@@ -382,6 +382,17 @@ static struct ctl_table kern_table[] = {
       .mode      = 0644,
       .proc_handler   = proc_dointvec,
    },
+#ifdef CONFIG_SCHED_AUTOGROUP
+   {
+      .procname   = "sched_autogroup_enabled",
+      .data      = &sysctl_sched_autogroup_enabled,
+      .maxlen      = sizeof(unsigned int),
+      .mode      = 0644,
+      .proc_handler   = proc_dointvec,
+      .extra1      = &zero,
+      .extra2      = &one,
+   },
+#endif
 #ifdef CONFIG_PROVE_LOCKING
    {
       .procname   = "prove_locking",
Index: linux-2.6/init/Kconfig
===================================================================
--- linux-2.6.orig/init/Kconfig
+++ linux-2.6/init/Kconfig
@@ -728,6 +728,18 @@ config NET_NS
 
 endif # NAMESPACES
 
+config SCHED_AUTOGROUP
+   bool "Automatic process group scheduling"
+   select CGROUPS
+   select CGROUP_SCHED
+   select FAIR_GROUP_SCHED
+   help
+     This option optimizes the scheduler for common desktop workloads by
+     automatically creating and populating task groups.  This separation
+     of workloads isolates aggressive CPU burners (like build jobs) from
+     desktop applications.  Task group autogeneration is currently based
+     upon task tty association.
+
 config MM_OWNER
    bool
 
Index: linux-2.6/Documentation/kernel-parameters.txt
===================================================================
--- linux-2.6.orig/Documentation/kernel-parameters.txt
+++ linux-2.6/Documentation/kernel-parameters.txt
@@ -1622,6 +1622,8 @@ and is between 256 and 4096 characters.
    noapic      [SMP,APIC] Tells the kernel to not make use of any
          IOAPICs that may be present in the system.
 
+   noautogroup   Disable scheduler automatic task group creation.
+
    nobats      [PPC] Do not use BATs for mapping kernel lowmem
          on "Classic" PPC cores.
 
Back to top
View user's profile Send private message
cruzki123
Apprentice
Apprentice


Joined: 16 May 2008
Posts: 233

PostPosted: Tue Nov 16, 2010 9:58 pm    Post subject: Reply with quote

This patch didn't apply cleany to 2.6.37-rc2 (as the other one I use). To witch version do you have been successfully?

Code:

artemisa linux # cat sched.patch | patch -p1
patching file include/linux/sched.h
Hunk #1 succeeded at 509 with fuzz 1.
Hunk #2 FAILED at 578.
Hunk #3 succeeded at 1932 (offset -1 lines).
1 out of 3 hunks FAILED -- saving rejects to file include/linux/sched.h.rej
patching file kernel/sched.c
Hunk #2 FAILED at 606.
Hunk #3 succeeded at 2017 (offset 10 lines).
Hunk #4 FAILED at 7981.
Hunk #5 FAILED at 8511.
Hunk #6 FAILED at 8540.
4 out of 6 hunks FAILED -- saving rejects to file kernel/sched.c.rej
patching file kernel/fork.c
Hunk #1 FAILED at 174.
Hunk #2 FAILED at 904.
2 out of 2 hunks FAILED -- saving rejects to file kernel/fork.c.rej
patching file drivers/tty/tty_io.c
Hunk #1 FAILED at 3160.
1 out of 1 hunk FAILED -- saving rejects to file drivers/tty/tty_io.c.rej
patching file kernel/sched_autogroup.h
patching file kernel/sched_autogroup.c
patching file kernel/sysctl.c
Hunk #1 FAILED at 382.
1 out of 1 hunk FAILED -- saving rejects to file kernel/sysctl.c.rej
patching file init/Kconfig
Hunk #1 succeeded at 728 with fuzz 2.
patching file Documentation/kernel-parameters.txt
Hunk #1 FAILED at 1622.
1 out of 1 hunk FAILED -- saving rejects to file Documentation/kernel-parameters.txt.rej
Back to top
View user's profile Send private message
hvengel
Guru
Guru


Joined: 19 Sep 2004
Posts: 515

PostPosted: Tue Nov 16, 2010 10:34 pm    Post subject: Reply with quote

Shining Arcanine wrote:
It would probably need to be applied manually. I assume that it will be included when it is accepted into the Linux kernel. I just tried applying it to the 2.6.35.8 kernel and it does not apply cleanly.


There have been a number of other changes related to scheduling in 2.6.36 and 2.6.37 that this patch needs. The merge window for 2.6.37 is now closed so the soonest this will show up in a vanilla kernel is 2.6.38.
Back to top
View user's profile Send private message
cruzki123
Apprentice
Apprentice


Joined: 16 May 2008
Posts: 233

PostPosted: Tue Nov 16, 2010 11:24 pm    Post subject: Reply with quote

This is really weird. I have tried with git-sources-2.6.37-rc1-git12 and fails to patch as well.

Code:

artemisa linux # cat sched.patch | patch -p1 --dry-run
patching file include/linux/sched.h
Hunk #1 succeeded at 509 with fuzz 1.
Hunk #2 FAILED at 578.
Hunk #3 succeeded at 1932 (offset -1 lines).
1 out of 3 hunks FAILED -- saving rejects to file include/linux/sched.h.rej
patching file kernel/sched.c
Hunk #2 FAILED at 606.
Hunk #3 succeeded at 2017 (offset 10 lines).
Hunk #4 FAILED at 7981.
Hunk #5 FAILED at 8511.
Hunk #6 FAILED at 8540.
4 out of 6 hunks FAILED -- saving rejects to file kernel/sched.c.rej
patching file kernel/fork.c
Hunk #1 FAILED at 174.
Hunk #2 FAILED at 904.
2 out of 2 hunks FAILED -- saving rejects to file kernel/fork.c.rej
patching file drivers/tty/tty_io.c
Hunk #1 FAILED at 3160.
1 out of 1 hunk FAILED -- saving rejects to file drivers/tty/tty_io.c.rej
patching file kernel/sched_autogroup.h
patching file kernel/sched_autogroup.c
patching file kernel/sysctl.c
Hunk #1 FAILED at 382.
1 out of 1 hunk FAILED -- saving rejects to file kernel/sysctl.c.rej
patching file init/Kconfig
Hunk #1 succeeded at 728 with fuzz 2.
patching file Documentation/kernel-parameters.txt
Hunk #1 FAILED at 1622.
1 out of 1 hunk FAILED -- saving rejects to file Documentation/kernel-parameters.txt.rej
artemisa linux #
Back to top
View user's profile Send private message
SlashBeast
Moderator
Moderator


Joined: 23 May 2006
Posts: 2837

PostPosted: Tue Nov 16, 2010 11:29 pm    Post subject: Reply with quote

Works for me on 2.6.37-rc2 from kernel.org.

For people who have problems with this patch, check mine (exported from my local git tree) http://paste.pocoo.org/raw/f9YBb2BVIPpfUKPbMDrj/
_________________
BitBucket -- better-initramfs
Back to top
View user's profile Send private message
wazoo42
Apprentice
Apprentice


Joined: 13 Apr 2004
Posts: 165

PostPosted: Tue Nov 16, 2010 11:31 pm    Post subject: Reply with quote

Here's a dumb question, but did you remember to change the symlink to /usr/src/linux each time and cd into /usr/src/linux before applying the patch?
Back to top
View user's profile Send private message
hvengel
Guru
Guru


Joined: 19 Sep 2004
Posts: 515

PostPosted: Tue Nov 16, 2010 11:36 pm    Post subject: Reply with quote

cruzki123 wrote:
This patch didn't apply cleany to 2.6.37-rc2 (as the other one I use). To witch version do you have been successfully?

Code:

artemisa linux # cat sched.patch | patch -p1
patching file include/linux/sched.h
Hunk #1 succeeded at 509 with fuzz 1.
Hunk #2 FAILED at 578.
Hunk #3 succeeded at 1932 (offset -1 lines).
1 out of 3 hunks FAILED -- saving rejects to file include/linux/sched.h.rej
patching file kernel/sched.c
Hunk #2 FAILED at 606.
Hunk #3 succeeded at 2017 (offset 10 lines).
Hunk #4 FAILED at 7981.
Hunk #5 FAILED at 8511.
Hunk #6 FAILED at 8540.
4 out of 6 hunks FAILED -- saving rejects to file kernel/sched.c.rej
patching file kernel/fork.c
Hunk #1 FAILED at 174.
Hunk #2 FAILED at 904.
2 out of 2 hunks FAILED -- saving rejects to file kernel/fork.c.rej
patching file drivers/tty/tty_io.c
Hunk #1 FAILED at 3160.
1 out of 1 hunk FAILED -- saving rejects to file drivers/tty/tty_io.c.rej
patching file kernel/sched_autogroup.h
patching file kernel/sched_autogroup.c
patching file kernel/sysctl.c
Hunk #1 FAILED at 382.
1 out of 1 hunk FAILED -- saving rejects to file kernel/sysctl.c.rej
patching file init/Kconfig
Hunk #1 succeeded at 728 with fuzz 2.
patching file Documentation/kernel-parameters.txt
Hunk #1 FAILED at 1622.
1 out of 1 hunk FAILED -- saving rejects to file Documentation/kernel-parameters.txt.rej


I got the same error when I tried the patch in the above post. But I got it to work by going to the link in the first post and using the patch located there.
Back to top
View user's profile Send private message
Shining Arcanine
Veteran
Veteran


Joined: 24 Sep 2009
Posts: 1110

PostPosted: Wed Nov 17, 2010 1:04 am    Post subject: Reply with quote

hvengel wrote:
Shining Arcanine wrote:
It would probably need to be applied manually. I assume that it will be included when it is accepted into the Linux kernel. I just tried applying it to the 2.6.35.8 kernel and it does not apply cleanly.


There have been a number of other changes related to scheduling in 2.6.36 and 2.6.37 that this patch needs. The merge window for 2.6.37 is now closed so the soonest this will show up in a vanilla kernel is 2.6.38.


Is it possible to have the patch included as a bug fix?
Back to top
View user's profile Send private message
Naib
Advocate
Advocate


Joined: 21 May 2004
Posts: 4197
Location: Removed by Neddy

PostPosted: Wed Nov 17, 2010 9:37 am    Post subject: Reply with quote

Shining Arcanine wrote:
hvengel wrote:
Shining Arcanine wrote:
It would probably need to be applied manually. I assume that it will be included when it is accepted into the Linux kernel. I just tried applying it to the 2.6.35.8 kernel and it does not apply cleanly.


There have been a number of other changes related to scheduling in 2.6.36 and 2.6.37 that this patch needs. The merge window for 2.6.37 is now closed so the soonest this will show up in a vanilla kernel is 2.6.38.


Is it possible to have the patch included as a bug fix?

That would result in alot of backporting (once it was figured out what patches are important).
Prob best just to goto 2.6.37
_________________
A free press is the unsleeping guardian of every other right that free men prize; it is the most dangerous foe of tyranny. Where men have the habit of liberty, the Press will continue to be the vigilant guardian of the rights of the ordinary citizen.
Back to top
View user's profile Send private message
depontius
Advocate
Advocate


Joined: 05 May 2004
Posts: 2689

PostPosted: Wed Nov 17, 2010 1:19 pm    Post subject: Reply with quote

There was also some discussion that most of the benefits of this patch can be accomplished with a little work in userspace. Here are 2 posts to LKML, one by the author of systemd presenting a few simple scripts, and another that is a tweak of the first.
http://lkml.org/lkml/2010/11/16/330
http://lkml.org/lkml/2010/11/16/413

This looks to me like an awfully easy thing to try, even on my current kernel source tree. Only one problem...

When the CGROUP stuff first appeared, it had a horrible effect on system responsiveness. The advice soon came to disable it, and that advice helped me. Since then I've followed that advice, and given it myself. My understanding was that CGROUPs could be really neat, but they required userspace support in order to do so, and without that userspace support it appeared that they were a hindrance. I've seen one start at a userspace CGROUP daemon, and systemd is another, though I'm not sure how it treats CGROUPs for non-daemon stuff. The scripts in the referenced posts are another, and the new patch does something by default in kernel space that looks pretty neat.

But I need OpenAFS, and it doesn't even work with 2.6.36 yet, let alone beyond. So for the time being I'm stuck at 2.6.35, which makes me more interested in the simple userspace scripts. Which leaves me with a simple question.

What CGROUP stuff do I have to enable in my kernel build to make the simple userspace scripts work? It looks to me as if CGROUP has had a lot of creeping featurism happen since it first appeared. Does anyone actually use this stuff now, and can comment intelligently about it? It looks to me as if most of the CGROUP work has been oriented toward virtualization, containers, etc.
_________________
.sigs waste space and bandwidth
Back to top
View user's profile Send private message
M
Guru
Guru


Joined: 12 Dec 2006
Posts: 427

PostPosted: Wed Nov 17, 2010 9:07 pm    Post subject: Reply with quote

I just patched vanilla-sources-2.6.36 with some little troubles, there was a one line reject in sched.c and tty_io.c was not in tty but char directory. I have now
sched_autogroup_enabled = 1 in proc.
Now to see how system behave when compiling chromium :lol: I removed it because system was not usable when compiling it and every day some new version pop up.
Back to top
View user's profile Send private message
j-kidd
Apprentice
Apprentice


Joined: 20 Feb 2003
Posts: 213

PostPosted: Thu Nov 18, 2010 3:57 am    Post subject: Reply with quote

M wrote:
Now to see how system behave when compiling chromium :lol: I removed it because system was not usable when compiling it and every day some new version pop up.

Those who use ck-sources-2.6.36 have no problem compiling chromium daily :P

depontius wrote:
It looks to me as if CGROUP has had a lot of creeping featurism happen since it first appeared. Does anyone actually use this stuff now, and can comment intelligently about it? It looks to me as if most of the CGROUP work has been oriented toward virtualization, containers, etc.

I can't comment intelligently about it, but I can say that CGROUP is the perfect proof that "CFS is for desktop" is a damn lie and "there should only be 1 scheduler" is a stupid decision.
Back to top
View user's profile Send private message
Shining Arcanine
Veteran
Veteran


Joined: 24 Sep 2009
Posts: 1110

PostPosted: Thu Nov 18, 2010 4:07 am    Post subject: Reply with quote

M wrote:
I just patched vanilla-sources-2.6.36 with some little troubles, there was a one line reject in sched.c and tty_io.c was not in tty but char directory. I have now
sched_autogroup_enabled = 1 in proc.
Now to see how system behave when compiling chromium :lol: I removed it because system was not usable when compiling it and every day some new version pop up.


I thought that tty_io.c was something new in the 2.6.37 kernel. I had no idea that it had been moved in the tree. That information made it easy to backport this patch to 2.6.35.8.

http://paste.pocoo.org/show/292920/

By the way, I fixed the slowness that occurs when doing system updates months ago by setting PORTAGE_NICENESS=20 in /etc/make.conf. It works fairly well.
Back to top
View user's profile Send private message
wrc1944
Advocate
Advocate


Joined: 15 Aug 2002
Posts: 2776
Location: Gainesville, Florida

PostPosted: Thu Nov 18, 2010 3:26 pm    Post subject: Reply with quote

All this sounds great, but notwithstanding the "automated" in the title of this thread, I'm having trouble understanding how this stuff works or is useful for a "normal" 2 or 4 core cpu desktop system.
I read this kernel.org documentation here http://www.kernel.org/doc/Documentation/cgroups/cgroups.txt
and gained some knowledge, but it seemed to attempt using cgroups you need an in depth understanding of how and why you would need to implement all or some of the different features. I guess this is where the "automated" part of the patch comes in, correct?

This is in make xconfig for 2.6.36, without the cgroup patch:
Quote:
Control Group support (CGROUPS)

CONFIG_CGROUPS:

This option adds support for grouping sets of processes together, for
use with process control subsystems such as Cpusets, CFS, memory
controls or device isolation.
See
- Documentation/scheduler/sched-design-CFS.txt (CFS)
- Documentation/cgroups/ (features for grouping, isolation
and resource control)

Say N if unsure.

Symbol: CGROUPS [=y]
Type : boolean
Prompt: Control Group support
Defined at init/Kconfig:489
Depends on: EVENTFD [=y]
Location:
-> General setup

The options to enable/disable are:
Example debug cgroup subsystems (NEW)
Namespace cgroup subsystem (NEW)
Freezer cgroup subsystem
Device controller for cgroups
Cpuset support
Resource counters (NEW)
Block IO controller (NEW)


I seem to recall when compiling previous kernels looking at cgroups in make xconfig, and it saying that it was only helpful with systems using many or even hundreds of cpus, and not recommended for average desktops. Can someone comment on this, or if it's really going to be useful for the average desktop user? Would the idea be to manually put stuff like say portage, web browsing, multimedia, graphics, xorg, office apps, etc. in permanent discrete cgroups, or is all this going to be auto-created for each session, and then destroyed when you log out? Am I fundamentally misunderstanding the entire concept? :roll:
_________________
Main box- Gigabyte GIGABYTE GA-990FXA-UD3 AM3+ rev.-4.0
Amd FX 8320, 3.5 GHz, 16GB GSkill DDR3 1866mhz
Samsung SATA 1000GB, Radeon HD 6570 2GB DDR3
Gentoo ~x86, ~amd64, glibc-2.19-r1, gcc-4.9.1 kernel 3.17.1-gentoo-r1 (USE=experimental "native")


Last edited by wrc1944 on Sat Nov 20, 2010 2:10 pm; edited 1 time in total
Back to top
View user's profile Send private message
M
Guru
Guru


Joined: 12 Dec 2006
Posts: 427

PostPosted: Thu Nov 18, 2010 5:30 pm    Post subject: Reply with quote

j-kidd wrote:
M wrote:
Now to see how system behave when compiling chromium :lol: I removed it because system was not usable when compiling it and every day some new version pop up.

Those who use ck-sources-2.6.36 have no problem compiling chromium daily :P


Actually, I had ck-sources 2.6.35, I compiled again vanilla to try the patch, btw. nice with default 10 adjustment didn't help before, but I compile this moment chromium, and I do not notice it at all, firefox is really responsive :) I have 2GB ram and amd dual core 4000+ cpu, so I think this is useful for normal machines, really good stuff!
Back to top
View user's profile Send private message
j-kidd
Apprentice
Apprentice


Joined: 20 Feb 2003
Posts: 213

PostPosted: Fri Nov 19, 2010 2:58 am    Post subject: Reply with quote

wrc1944 wrote:
Would the idea be to manually put stuff like say portage, web browsing, multimedia, graphics, xorg, office apps, etc. in permanent discrete cgroups, or is all this going to be auto-created for each session, and then destroyed when you log out?

You can refer to this post (and the associated thread) about how cgroups are setup in maemo 5. What I can say is that n900 is one slow unresponsive phone compared to android phones.

M wrote:
Actually, I had ck-sources 2.6.35

Cool, you should give ck-sources 2.6.36 a try. For me, the difference is pretty huge. I think some IO related fix went into 2.6.36.
Back to top
View user's profile Send private message
Picander
n00b
n00b


Joined: 15 Jul 2005
Posts: 34
Location: Italy

PostPosted: Fri Nov 19, 2010 11:13 am    Post subject: Reply with quote

Interesting patch, anyway how should I behave if I want a background daemon have the maximum priority (even higer than mouse movement)?
Back to top
View user's profile Send private message
kernelOfTruth
Watchman
Watchman


Joined: 20 Dec 2005
Posts: 5743
Location: Vienna, Austria; Germany; hello world :)

PostPosted: Fri Nov 19, 2010 3:46 pm    Post subject: Reply with quote

Picander wrote:
Interesting patch, anyway how should I behave if I want a background daemon have the maximum priority (even higer than mouse movement)?


probably via cgroups:

http://broadcast.oreilly.com/2009/06/manage-your-performance-with-cgroups-and-projects.html

and nice/renice :wink:
_________________
https://github.com/kernelOfTruth/ZFS-for-SystemRescueCD/tree/ZFS-for-SysRescCD-4.3.0-r2
2.6.37.2_plus_v1: BFS, CFS,THP,compaction, zcache or TOI
Hardcore Linux user since 2004 :D
Back to top
View user's profile Send private message
Fitzcarraldo
l33t
l33t


Joined: 30 Aug 2008
Posts: 609
Location: United Kingdom

PostPosted: Sat Nov 20, 2010 5:49 am    Post subject: Solution in userspace instead of applying the kernel patch Reply with quote

@depontius:

I'm using the 2.6.36 kernel and I've got CONFIG_CGROUP_SCHED=y, CONFIG_FAIR_GROUP_SCHED=y and CONFIG_SCHED_DEBUG=y (not sure if the last of those is necessary, but it doesn't do any harm).

Code:
$ cat /usr/src/linux/.config | grep CGROUP
CONFIG_CGROUPS=y
CONFIG_CGROUP_DEBUG=y
CONFIG_CGROUP_NS=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y
# CONFIG_CGROUP_CPUACCT is not set
CONFIG_CGROUP_MEM_RES_CTLR=y
# CONFIG_CGROUP_MEM_RES_CTLR_SWAP is not set
CONFIG_CGROUP_SCHED=y
CONFIG_BLK_CGROUP=y
# CONFIG_DEBUG_BLK_CGROUP is not set
# CONFIG_NET_CLS_CGROUP is not set


The Ubuntu blog post Alternative To The "200 Lines Kernel Patch That Does Wonders" Which You Can Use Right Away basically uses both Lennart Poettering's and Vivek Goyal's approaches (see their posts http://lkml.org/lkml/2010/11/16/330 and http://lkml.org/lkml/2010/11/16/413). I followed the blog post:

~/.bashrc
Code:
# /etc/skel/.bashrc
#
# This file is sourced by all *interactive* bash shells on startup,
# including some apparently interactive shells such as scp and rcp
# that can't tolerate any output.  So make sure this doesn't display
# anything or bad things will happen !


# Test for an interactive shell.  There is no need to set anything
# past this point for scp and rcp, and it's important to refrain from
# outputting anything in those cases.
if [[ $- != *i* ]] ; then
        # Shell is non-interactive.  Be done now!
        return
fi


# Put your fun stuff here.

###########################################################################
# http://www.webupd8.org/2010/11/alternative-to-200-lines-kernel-patch.html
if [ "$PS1" ] ; then
   mkdir -p -m 0700 /dev/cgroup/cpu/user/$$ > /dev/null 2>&1
   echo $$ > /dev/cgroup/cpu/user/$$/tasks
   echo "1" > /dev/cgroup/cpu/user/$$/notify_on_release
fi
###########################################################################


/etc/conf.d/local
Code:
# Here is where you can put anything you need to start
# that there is not an init script for.

local_start() {
        # This is a good place to load any misc programs
        # on startup (use &>/dev/null to hide output)

###########################################################################
# http://www.webupd8.org/2010/11/alternative-to-200-lines-kernel-patch.html
mkdir -p /dev/cgroup/cpu
mount -t cgroup cgroup /dev/cgroup/cpu -o cpu
mkdir -m 0777 /dev/cgroup/cpu/user
echo "/usr/local/sbin/cgroup_clean" > /dev/cgroup/cpu/release_agent
###########################################################################

        # We should always return 0
        return 0
}

local_stop() {
        # This is a good place to unload any misc.
        # programs you started above.

        # We should always return 0
        return 0
}


/usr/local/sbin/cgroup_clean
Code:
#!/bin/sh
rmdir /dev/cgroup/cpu/$*


Make the script executable:
Code:
# chmod +x /usr/local/sbin/cgroup_clean


Simple. But, whether it's the kernel patch or the userspace equivalent, it's over-hyped unless you're doing a lot of Konsole/Terminal stuff concurrently:

Lennart Poettering wrote:
Jeez. Don't mentione the desktop. On the desktop this is compleltely irrelevant. There are not TTYs on the desktop. There's no "make -j" of the kernel tree on the desktop.

The kernel patch discussed here *has* *no* *relevance* for normal users.

The kernel patch discussed here is only relevant for people which start mplayer from one terminal, and "make -j" from another.

Well, maybe it has some use for Gentoo folk. :)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum