The following experimental patch implements token _base_d thrashing protection, using the algorithm described in:         http://www.cs.wm.edu/~sjiang/token.htm When there are pageins going on, a task can grab a token, that protects the task from pageout (except by itself) until it is no longer doing heavy pageins, or until the maximum hold time of the token is over. If the maximum hold time is exceeded, the task isn't eligable to hold the token for a while more, since it wasn't doing it much good anyway. I have run a very unscientific benchmark on my system to test the effectiveness of the patch, timing how a 230MB two-process qsbench run takes, with and without the token thrashing protection present. normal 2.6.8-rc6:       6m45s 2.6.8-rc6 + token:      4m24s This is a quick hack, implemented without having talked to the inventor of the algorithm.  He's copied on the mail and I suspect we'll be able to do better than my quick implementation ... Please test this patch.  include/linux/sched.h |    4 ++  include/linux/swap.h  |   21 ++++++++++  kernel/fork.c         |    2 +  mm/Makefile           |    2 -  mm/filemap.c          |    1  mm/memory.c           |    1  mm/rmap.c             |    3 +  mm/thrash.c           |  100 ++++++++++++++++++++++++++++++++++++++++++++++++++  8 files changed, 133 insertions(+), 1 deletion(-)

I have run a very unscientific benchmark on my system to test the effectiveness of the patch, timing how a 230MB two-process qsbench run takes, with and without the token thrashing protection present. normal 2.6.8-rc2:  6m45s 2.6.8-rc2 + token: 4m24s OK, I've now also ran day-long kernel compilate tests, 3 times each with make -j 10, 20, 30, 40, 50 and 60 on my dual pIII w/ 384 MB and a 180 MB named in the background. For make -j 10 through make -j 50 the differences are in the noise, basically giving the same result for each kernel. However, for make -j 60 there's a dramatic difference between a kernel with the token _base_d swapout and a kernel without. normal 2.6.8-rc2:       1h20m runtime / ~26% CPU use average 2.6.8-rc2 + token:        42m runtime / ~52% CPU use average Time to dig out a dedicated test machine at the office and do some testing with (RE-)AIM7, I wonder if the max number of users supported will grow...

  I have run a very unscientific benchmark on my system to test   the effectiveness of the patch, timing how a 230MB two-process   qsbench run takes, with and without the token thrashing   protection present.   normal 2.6.8-rc2:    6m45s   2.6.8-rc2 + token:   4m24s  OK, I've now also ran day-long kernel compilate tests,  3 times each with make -j 10, 20, 30, 40, 50 and 60 on  my dual pIII w/ 384 MB and a 180 MB named in the background.  For make -j 10 through make -j 50 the differences are in  the noise, basically giving the same result for each kernel.  However, for make -j 60 there's a dramatic difference between  a kernel with the token _base_d swapout and a kernel without.  normal 2.6.8-rc2: 1h20m runtime / ~26% CPU use average  2.6.8-rc2 + token:          42m runtime / ~52% CPU use average OK.  My test is usually around 50-60% CPU occupancy so we're not gaining in the moderate swapping range. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to This e-mail address is being protected from spam bots, you need JavaScript enabled to view it More majordomo info at  http://vger.kernel.org/majordomo-info.html Please read the FAQ at  http://www.tux.org/lkml/

 However, for make -j 60 there's a dramatic difference between  a kernel with the token _base_d swapout and a kernel without.  normal 2.6.8-rc2:    1h20m runtime / ~26% CPU use average  2.6.8-rc2 + token:     42m runtime / ~52% CPU use average OK.  My test is usually around 50-60% CPU occupancy so we're not gaining in the moderate swapping range. I wonder if measuring minor faults too would help here ... Btw, here's a slightly updated patch.  It's got the definition for put_swap_token fixed for !CONFIG_SWAP and calls put_swap_token before mmput. I also cut the 4G/4G split line out of the mm/Makefile patch chunk, so that should now apply better. It doesn't have any functional changes I'm aware of.

I have run a very unscientific benchmark on my system to test the effectiveness of the patch, timing how a 230MB two-process qsbench run takes, with and without the token thrashing protection present. normal 2.6.8-rc2: 6m45s 2.6.8-rc2 + token:        4m24s OK, I've now also ran day-long kernel compilate tests, 3 times each with make -j 10, 20, 30, 40, 50 and 60 on my dual pIII w/ 384 MB and a 180 MB named in the background. For make -j 10 through make -j 50 the differences are in the noise, basically giving the same result for each kernel. However, for make -j 60 there's a dramatic difference between a kernel with the token _base_d swapout and a kernel without. normal 2.6.8-rc2:      1h20m runtime / ~26% CPU use average 2.6.8-rc2 + token:       42m runtime / ~52% CPU use average OK.  My test is usually around 50-60% CPU occupancy so we're not gaining in the moderate swapping range. We have some results that need interpreting with contest. mem_load: Kernel    [runs]        Time    CPU%    Loads   LCPU%   Ratio 2.6.8-rc2      4        78      146.2   94.5    4.7     1.30 2.6.8-rc2t     4        318     40.9    95.2    1.3     5.13 The load with mem_load is basically trying to allocate 110% of free ram, so the number of loads although similar is not a true indication of how much ram was handed out to mem_load. What is interesting is that since mem_load runs continuously and constantly asks for too much ram it seems to be receiving the token most frequently in preference to the cc processes which are short lived. I'd say it is quite hard to say convincingly that this is bad because the point of this patch is to prevent swap thrash. It would get far more complicated to create a list of tasks trying to get the token and refuse to hand it back to the same task until it cycled through all the other tasks to prevent this... and I'm not even sure that would help since these are all short lived tasks... Any other thoughts? To be honest I dont think this contest result is truly a bad thing... Con






 

signature.asc
1K
Download