When I own a Ferrari I will expect to have to pay for regular servicing.

Having said all of which, I did notice the costs of the project mounting. I wonder if one could correlate the time spent with SAS to the project budget, and compare this to a benchmark project in which they were not involved in some way?

When I own a Ferrari I don’t plan to call on the services of a chauffeur too often.

It just goes to show, I suppose, that best practice can be expensive and this is, perhaps, why our SAS environment is in the mess it is. One thing I’m very wary of is the amount of resource assigned to seemingly mundane tasks where I can’t see the value they will add. This may be myopia on my part, and I am optimistic that experience (to be gained) will teach me the error of my ways, or that experience (already earnt by colleagues) will find the budget for the SAS resource.

Technorati Tags: criticism, SAS

There was a great article on hplus magazine by Rudy Rucker, whose life seems to have been entwined with Wolfram for a long time. Rudy seems to have got an early dose of the excitment I’ve picked up around Wolfram’s plans.

He quotes Stephen Wolfram as saying:

Wolfram|Alpha isn’t really a search engine, because we compute the answers, and we discover new truths. If anything, you might call it a platonic search engine, unearthing eternal truths that may never have been written down before.

he goes on to explain that:

Wolfram|Alpha can pop out an answer to pretty much any kind of factual question that you might pose to a scientist, economist, banker, or other kind of expert. The exciting part is that you’re not just looking up pages on the web, you’re getting new information that’s generated by computations working from the known data.

Stephen Wolfram seems confident that he will be able to cope with capacity issues:

Most of our computational needs center on converting user inputs into Mathematica-like queries, and then computing the answers to these queries. Answering one query might use a hundred of our computers running flat out for a fraction of a second. So we’re still trying to gauge how many computers we’ll need.

I’m itching to see this in action, but if it is as good as it sounds, will Wolfram sell out to the highest bidder once this is off the ground?

I’m not a selling things kind of guy, …
We’d rather look for things like partnerships or licensing deals or APIs. I see a new field of knowledge-based computing. Imagine a spread sheet that can pull in knowledge about the entries.

A spreadsheet, or a business intelligence platform? Imagine Wolfram|Alpha’s computational natural language interface licensed to interface with all of the data on the web, the logical structures in Wolfram|Alpha, …, and all of your corporate data! That could be an explosive combination: the Power to Dream?

SAS bloggers, such as Tammi Kay George and Waynette Tubbs, have been talking about Data Driven Management Culture and how to Make the Most of a Bad Situation: Analytics.

Which prompt me to ask two questions:

• How soon before this scientific development is suborned to commercial ends?
• Would this be a bad thing?

Technorati Tags: alpha, SAS, wolfram

I’ve skimmed the titles and there are a few that I’ve earmarked for further study, see below; sadly some of the interesting ones came up with the disappointing “No paper was submitted for publication in the Proceedings.” message.

Which ones are you reading?

I’ve divided the papers up into the following groups:

Paul Dorfman
Worth reading at least twice…

010:From Obscurity to Utility: ADDR, PEEK, POKE as DATA Step Programming Tools
038:The DOW-Loop Unrolled

SAS Admin
Work

094:At the Controls: An Approach to Security via SAS® Management Console
124:Building a Data Repository Using Base SAS®
177:The Many Hats of the SAS® Administrator
179:Supporting a SAS® Server Architecture
271:Configuration Management for SAS® Software Projects

Performance
Obsession

333:Troubleshoot Your Performance Issues: SAS® Technical Support Shows You How
008:Formats, MP/Connect, and Other Efficiency Techniques to Save Time and Space
012:Maximum SAS®: Analyzing and Increasing Performance
024:A Faster Index for Sorted SAS® Datasets
037:Scalability of Table Lookup Techniques
065:Effectiveness and Cost of SAS® Compression
071:Choosing the Right Technique to Merge Large Data Sets Efficiently

Miscellaneous
Just for fun

086:Could You Remind Me? Creating Calendar Events from SAS®
201:SAS® Magic Squares
220:Show Them What’s Important with Communication-Effective SAS® Graphs: Solutions for a Finite Work Day in an Era of Information Overload

Now all I need are an extra ten hours in each day to read them!

Technorati Tags: forum, global, papers, proceedings, SAS, sugi

A flurry of press releases from SAS®.

Two caught my eye: SAS to offer R integration to support analytical innovation, and SAS and Teradata Announce Availability of Analytic Advantage and Optimization Services Packages.

SAS to offer R integration to support analytical innovation

SAS, the leader in business analytics, is expanding analytical options for its customers with a new interface to R open-source statistical software. SAS’ initial integration with R will be included in SAS/IML ® Studio 3.2 (formerly SAS Stat Studio) scheduled for summer 2009.

…

“It is no secret that SAS has been working on interfacing with R,” said Anne Milley, SAS’ Senior Director of Technology Product Marketing. “SAS and R are here to stay, and as organizations work to harness the full potential of their data, an expanded set of analytics options can only help.”

So if you want the R interface you need to license software (SAS/IML) that under-performs the software (R) that you currently use. I think that this is a brave step by SAS, but it looks a little misguided: the power of R is that it interfaces with SAS data and provides functionality not available in SAS and all of this for little cost (cost as effort, rather than just license dollars).

If R was being used as a marketing tool to try to sell SAS, I could imagine an antipathy or backlash developing in the R community.

SAS and Teradata Announce Availability of Analytic Advantage and Optimization Services Packages

Why is it that when we’re paying for SAS and Teradata we feel that we can expect to pay for their integration, but when one component (eg R, see above) is free, we expect the package should be free?

“Our partnership with SAS creates a highly innovative dream team focused on helping companies activate and optimize analytic model development and execution inside the database, where we see processing speeds increase as much as 45 times,” said Rob Berman, SAS Partnership Vice President for Teradata. “These two packages are just the beginning of an in-database planned portfolio of offers we’ll deliver to a market that is ready and waiting for it. In view of the economic pressures companies now face, these packages have come at exactly the right time.”

…

The Teradata and SAS partnership has continued to earn high marks from the analyst community. “Both SAS and Teradata are large, financially stable companies with loyal customer bases,” said Helena Schwenk, Senior IT Analyst at Ovum, in a new analysis. “More importantly, both companies’ software, applications and infrastructure are essential tools for helping companies beat the downturn – whether they are used to highlight cost reduction opportunities, improve supply chains, optimize inventory levels, help businesses understand risk or rapidly changing customer preferences and spending.”

I look forward to hearing more!

Technorati Tags: analytics, integration, r, SAS, teradata

These are only initial findings – based on a single set of observations for one size of dataset – and these taken on a day when the server complained of memory problems.

Curious!

I will update this page when I have more confidence in the results

Technorati Tags: performance

It looks as if we might have quite a basket of products to compare when we move into the data mining arena.

I’ve yet to look into weka in detail, but I’m amazed by how blinkered I have been in accepting the default option: stick with a single software provider. If you always do what you’ve always done, you’ll always get what you’ve always got!

Actually, since SAS® has been our main software provider, what we’ve always had is great software and support.

What the credit crunch and economic downturn are forcing us to do is re-evaluate the price we pay and the return on investment, and this is the selling point of open source software.

Technorati Tags: data mining, open source, price, weka

Any ideas for more imaginative, or non-intuitive uses?

How about

proc alpha;
   q='';
run;

Technorati Tags: alpha, SAS, wolfram

Well, I couldn’t quite believe how simple the syntax is, it feels quite strange using <- all over the place, and not a proc in sight (R exposes functionality through functions).

Moving very slightly forwards from the intro code I wrote, with a bit of trial and error, the following:


library("foreign")
testsas <- read.ssd("c:/temp", "test", sascmd="c:/program files/sas institute/sas/v8/sas.exe")
attach(testsas)
summary(testsas)
fm<-lm(y~x,testsas)
summary(fm)
plot(x,y)
abline(coef(fm))


And it worked!

The "exciting" thing about this rather mundane code is that the read.ssd() function is reading a SAS® dataset into R! Reading a SAS dataset is no more difficult than loading the "foreign" library, and then passing the path, the dataset name (without the .sas7bdat extension), and the path to sas.exe.

The dataset is fairly dull - it has two variables x and y, where y is x plus x times a random amount between 0 and 1.

The attach() function makes the dataset available as a data frame, the summary() function gives basic summary stats, the lm() function is equivalent to proc reg, and stores the regression coefficients to a new variable structure fm.

Another call to fm() gives me linear regression coefficients (the m and c, from y = mx + c) and goodness of fit statistics.

Finally the calls to plot() and abline() plot a chart of observations, overlayed with a line of best fit.

Not very interesting but: (a) reading SAS data into R is quick and easy; (b) writing R code is quick and easy; (c) the results from the R code were verified by those produced in SAS.

Point (c) isn't much of a surprise - but the painlessness of both (a) and (b) is!

I look forward with great interest to hear what comes out of the SAS Global Forum in a few days time!

Technorati Tags: data, r, SAS

My belated New Year’s resolution is to join up the dots and bring all of the SAS users together into a user group. But what do all of these people have in common, other than an analytical turn of mind? How can we tailor content to stimulate their little grey cells?

Thinking along these lines, I stumbled across a slew of (to me) new SAS websites: Jared, SAS from Out in Left Field and Using SAS in the real world.

It looks as if an aggregated, if not integrated, SAS blogosphere is emerging, but I’m amazed by the diversity of these sites. The interests of these individual bloggers mean that high quality content is being produced on topics such as office integration (Left Field), improving effectiveness (Real world) as well as more opinionated postings from Jared.

The winner from all of this effort will, hopefully, be the end user and their down-stream customers.

Technorati Tags: community, group, licenses, SAS, users

It was interesting to read about SAS’s use of open source software:

Open source software provides many options and benefits. We see many (SAS included) embracing open source for different things…we plan to do more with open source in the future.

But nothing more definite yet, we’ve got to wait for SAS® Global Forum:

The first step of SAS integrating with R will be shown at SAS Global Forum coming up in DC later this month. Other announcements for new offerings are also planned at this event.

We will wait with baited breath, and continue on the path towards R, happy to have Anne Milley’s full backing:

I have great respect for people who invest in learning (or even getting exposure to) more than one language and who appreciate the strengths of different languages for certain tasks and applications.

I only wish I could go!

I wonder if there will be any meaningful talk there of the longer term plans for SAS Institute, a topic avoided in the interview.

Technorati Tags: r, SAS