reflections.concept-delivery.com» performance

Sorting: bufsize and sortsize

Reflections — Thu, 19 Mar 2009 22:18:59 +0000

Initial results for sorting data suggest that a small sortsize improves sortation, whilst the time taken is largely independent of the buffer size (bufsize) used to create the file.

These are only initial findings – based on a single set of observations for one size of dataset – and these taken on a day when the server complained of memory problems.

Curious!

I will update this page when I have more confidence in the results

Technorati Tags: performance

Getting data into memory

Reflections — Tue, 27 Jan 2009 16:09:38 +0000

Buffers seem to be the first step, specifically the bufsize and bufno system options.

These tell SAS® how much data is read into memory: bufsize sets the size of a page in the file cache and bufno sets the number of pages to be read in one go.

The product gives the memory requirement: bufsize x bufno.

After extensive testing, see Buffer size and CPU time for details, it appears that using the full 2GB produces the best results, but is a very greedy way of accomplishing a good result. It seems that for our set up a memory allocation of 256MB produces results that are nearly as good, and the best settings for this are achieved with bufsize = 256M and bufno = 1024.

Incidentally, 256MB seems to be a useful number to know as it works well as the sortsize setting for sorting.

The idea behind the testing was the SAS technical note Achieving Better I/O Throughput Using SGIO.

Technorati Tags: buffers, bufno, bufsize, data, memory, performance, SAS

Performance considerations for SAS

Reflections — Tue, 27 Jan 2009 15:07:41 +0000

Having recently taken over as SAS® admin, I’m having fun and games trying to establish quite what is where, …, and why.

We’re using v8, planning to upgrade to v9 shortly, and the box seems very slow. This may be because the server is doing more work – more users and larger datasets than I was used to – but nonetheless it seems a little sluggish and I’m suspicious of Win2k3, having been happy with Unix.

I’ve been reading up on performance and, to put it gently, there seems to be an opportunity to tune the system.

Firstly memory usage: our system has 3.2GB of memory but performance counters on the server suggest that only a fraction of this (say a tenth to a quarter) is used at any one time.

Paging: performance monitor logs show cache file usage and paging is very high – typically in the thousands. This is more to do with the way SAS reads data, a page at a time via the file cache, than a memory issue. Nonetheless I would be keen to bring this figure down as I’m concerned that the high usage stats may be creating a bottle-neck.

Looking into this further revealed that a typical SAS session, running on the server, uses about 12MB of memory. At this rate we could comfortably accomodate two hundred users – we currently have closer to thirty and it unusual to have more than twenty sessions running concurrently.

My plan: to increase memory usage so we can get data into memory quickly.

Technorati Tags: config, performance, SAS, v8