Getting data into memory

Buffers seem to be the first step, specifically the bufsize and bufno system options.

These tell SAS® how much data is read into memory: bufsize sets the size of a page in the file cache and bufno sets the number of pages to be read in one go.

The product gives the memory requirement: bufsize x bufno.

After extensive testing, see Buffer size and CPU time for details, it appears that using the full 2GB produces the best results, but is a very greedy way of accomplishing a good result. It seems that for our set up a memory allocation of 256MB produces results that are nearly as good, and the best settings for this are achieved with bufsize = 256M and bufno = 1024.

Incidentally, 256MB seems to be a useful number to know as it works well as the sortsize setting for sorting.

The idea behind the testing was the SAS technical note Achieving Better I/O Throughput Using SGIO.

Technorati Tags: , , , , , ,

This entry was posted in SAS and tagged , , , , , , . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

One Trackback

  1. [...] plan: to increase memory usage so we can get data into memory quickly. Leave a comment | Trackback No comments [...]

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>