There was a great article on hplus magazine by Rudy Rucker, whose life seems to have been entwined with Wolfram for a long time. Rudy seems to have got an early dose of the excitment I’ve picked up around Wolfram’s plans.

He quotes Stephen Wolfram as saying:

Wolfram|Alpha isn’t really a search engine, because we compute the answers, and we discover new truths. If anything, you might call it a platonic search engine, unearthing eternal truths that may never have been written down before.

he goes on to explain that:

Wolfram|Alpha can pop out an answer to pretty much any kind of factual question that you might pose to a scientist, economist, banker, or other kind of expert. The exciting part is that you’re not just looking up pages on the web, you’re getting new information that’s generated by computations working from the known data.

Stephen Wolfram seems confident that he will be able to cope with capacity issues:

Most of our computational needs center on converting user inputs into Mathematica-like queries, and then computing the answers to these queries. Answering one query might use a hundred of our computers running flat out for a fraction of a second. So we’re still trying to gauge how many computers we’ll need.

I’m itching to see this in action, but if it is as good as it sounds, will Wolfram sell out to the highest bidder once this is off the ground?

I’m not a selling things kind of guy, …
We’d rather look for things like partnerships or licensing deals or APIs. I see a new field of knowledge-based computing. Imagine a spread sheet that can pull in knowledge about the entries.

A spreadsheet, or a business intelligence platform? Imagine Wolfram|Alpha’s computational natural language interface licensed to interface with all of the data on the web, the logical structures in Wolfram|Alpha, …, and all of your corporate data! That could be an explosive combination: the Power to Dream?

SAS bloggers, such as Tammi Kay George and Waynette Tubbs, have been talking about Data Driven Management Culture and how to Make the Most of a Bad Situation: Analytics.

Which prompt me to ask two questions:

• How soon before this scientific development is suborned to commercial ends?
• Would this be a bad thing?

Technorati Tags: alpha, SAS, wolfram

A flurry of press releases from SAS®.

Two caught my eye: SAS to offer R integration to support analytical innovation, and SAS and Teradata Announce Availability of Analytic Advantage and Optimization Services Packages.

SAS to offer R integration to support analytical innovation

SAS, the leader in business analytics, is expanding analytical options for its customers with a new interface to R open-source statistical software. SAS’ initial integration with R will be included in SAS/IML ® Studio 3.2 (formerly SAS Stat Studio) scheduled for summer 2009.

…

“It is no secret that SAS has been working on interfacing with R,” said Anne Milley, SAS’ Senior Director of Technology Product Marketing. “SAS and R are here to stay, and as organizations work to harness the full potential of their data, an expanded set of analytics options can only help.”

So if you want the R interface you need to license software (SAS/IML) that under-performs the software (R) that you currently use. I think that this is a brave step by SAS, but it looks a little misguided: the power of R is that it interfaces with SAS data and provides functionality not available in SAS and all of this for little cost (cost as effort, rather than just license dollars).

If R was being used as a marketing tool to try to sell SAS, I could imagine an antipathy or backlash developing in the R community.

SAS and Teradata Announce Availability of Analytic Advantage and Optimization Services Packages

Why is it that when we’re paying for SAS and Teradata we feel that we can expect to pay for their integration, but when one component (eg R, see above) is free, we expect the package should be free?

“Our partnership with SAS creates a highly innovative dream team focused on helping companies activate and optimize analytic model development and execution inside the database, where we see processing speeds increase as much as 45 times,” said Rob Berman, SAS Partnership Vice President for Teradata. “These two packages are just the beginning of an in-database planned portfolio of offers we’ll deliver to a market that is ready and waiting for it. In view of the economic pressures companies now face, these packages have come at exactly the right time.”

…

The Teradata and SAS partnership has continued to earn high marks from the analyst community. “Both SAS and Teradata are large, financially stable companies with loyal customer bases,” said Helena Schwenk, Senior IT Analyst at Ovum, in a new analysis. “More importantly, both companies’ software, applications and infrastructure are essential tools for helping companies beat the downturn – whether they are used to highlight cost reduction opportunities, improve supply chains, optimize inventory levels, help businesses understand risk or rapidly changing customer preferences and spending.”

I look forward to hearing more!

Technorati Tags: analytics, integration, r, SAS, teradata

In his recent blog post Stephen Wolfram tries to answer some of these questions:

What is Wolfram|Alpha, then?

It’s going to be a website: www.wolframalpha.com. With one simple input field that gives access to a huge system, with trillions of pieces of curated data and millions of lines of algorithms.

What does that mean?

…one would be able to ask a computer any factual question, and have it compute the answer…
I’d always thought, though, that eventually it should be possible. And a few years ago, I realized that I was finally in a position to try to do it.

But that’s never going to work is it?

I wasn’t at all sure it was going to work. But I’m happy to say that with a mixture of many clever algorithms and heuristics, lots of linguistic discovery and linguistic curation, and what probably amount to some serious theoretical breakthroughs, we’re actually managing to make it work.

Wow! Google, your days are numbered!

Technorati Tags: alpha, google, wolfram

Well, whilst they may help, I think Edward Tufte has identified some of the key issues in his paper on the Cognitive Style of PowerPoint. I’m a great fan of Tufte and devoured his books on long commutes: the only time a fellow commuter has ever taken an interest in what I have been reading was after they had been admiring his rich pagination over my shoulder – I was too engrossed to notice.

Having bought the books through Amazon, their alogrithm suggested that, based on my spending patterns, I might enjoy certain other of their offerings. Two spring to mind as having been pushed relentlessly through “Amazon’s recommendations for you” pages: Information Dashboard Design and Presentation Zen. Neither was in the same league, nor did they aim to be. IDD was interesting, had some neat ideas and a cunning new graphical scheme for displaying a lot of information densely; coming from Tufte’s work, I found Presentation Zen attractive but very shallow. This disappointing performance has turned me off Amazon’s recommendations.

But is this fair?

I suppose I imagine reading as a journey, with each step taking me ever upwards. The problem is that if great books are distributed on a power law basis and you read one of the very best, then there is likely to be a lot of (relative) dross in the foothills, and I wonder whether this exposes a problem with Amazon’s analytics: whether they take into account the sequencing of purchases (a reading vector), or just correlations between purchases. (Of course, I’m assuming that my estimation of the relative merit of these books is absolute, which may be very far from the case!)

I took a route to Tufte via Jon Moon’s books; perhaps other people have travelled via Presentation Zen, and Amazon is simpy showing me the footsteps around Mount Tufte. I’d be interested to hear from anyone who knows more about how Amazon works: do they have someone interested in BI and Performance Improvement?

… talk of which, brings me neatly to Tammi Kay George: I was inspired to write this after reading her recent blog post in which she mentions Presentation Zen. I couldn’t help wondering if she had been bludgenoned into buying it by Amazon, too: I suspect there may be an overlap between her library shelves and mine, and if there is, my decision to buy Presentation Zen undoubtedly will have increased the likelihood of Amazon recommending it to her! It’s all my fault!

Technorati Tags: amazon, analytics, bi, powerpoint, presentation zen, recommendations, SAS, statistics, tufte

Hotfixes for both clients and server are now in, and the number of reported errors in the server’s event viewer have dropped dramatically.

On to planning the migration and upgrade to v9.1.3. IS have been very supportive and SAS® reps have been falling over themselves to be helpful, so I’m optimistic for the future… and we’ve got some prospective new customers! It’s just a little sad that the day after tomorrow v9.2 Phase 2 will be launched, but this upgrade should bring us considerable advantages and leave the way open to upgrade to 9.2 in due course once its teething problems have been solved, …

…, unless we wait for SAS X, or whatever v10 is called…

Technorati Tags: hotfix, new customers, potatoes, SAS, upgrade

The hotfixing and testing took about an hour, we were done by 7pm, and it appears that it worked – 11 sessions running happily and people trickling into work. We applied the Base hotfix bundle 82BX09 and then 82BC21, 82BC23, 82BC24, 82BC27, 82BC32, 82BC34, 82BC41 and 82BC42, and 82CT09 for SAS/Connect and 82TE08 for SAS/Access to Teradata, see the SAS Support site for details. Before installation we stopped and uninstalled the SAS job spawner service, reinstalling and restarting it at the end.

The installation notes for a number of the fixes mention upgrading the client version of sashost.dll to version 8.0.16713.46619. We haven’t done this yet, we still have 8.0.14958.52973 installed, but SAS is still working. We are rolling out a client version (actually the same files) packaged up using SSCM and these are scheduled to be dropped on Monday next week for desktops, and then Tuesday for laptops.

Technorati Tags: 82bx09, hotfix, SAS, server, v8.2

Photos – and this will make any parent’s blood run cold – the only copies, of our childrens’ first years!

Lost!

But not irrecoverably!

Shortly after ordering the new PC I ordered some more memory, a new graphics card and an external hard disk drive (HDD) enclosure. I’d been into PC World to look for the enclosure and they only had IDE and PATA enclosures. The hard disk I wanted to mount was a SATA disk, which meant ordering over the web. The memory was proprietary, and so the store had to order it in, and I had to wait, and wait!

I plugged in the memory, powered up the machine, and, …, it works – a bit quicker.

Shut down, insert graphics card, plug in funny monitor dongle, and monitor, and, …, it works, beautifully!

Now the bit I’d been dreading, …

But installing the old hard drive was simplicity itself, a matter of unscrewing and unplugging from the old machine, and then re-plugging and screwing into the enclosure, then connecting power and USB cables and turning on. Win XP already had the drivers.

I’ve got my photos back! And my monitor no longer says that a higher resolution would give better pictures!

By the way, the mount from Akasa has a barcode AK-ENP2SATA-BKUK, and is described as an integral/eSATA HDD external enclosure for 3.5 inch IDE PATA and SATA HDD.

Technorati Tags: akasa, external drive, graphics card, hard disk, lost, memory, new, sata, usb

It has been quite an effort getting IS to support our server hotfix. See It just crashed… for details. Part of the problem is that IS don’t support the server in any meaningful way at the moment. This is likely to change in the future: watch ths space!

When I discussed hotfixing with SAS® Technical Support, we came to the conclusion that applying the bundled hotfixes for v8 seemed the best bet. These need to be applied to both the client and the server and the next question was: in which order, client first or server first?

For version 8, it doesn’t matter. For version 9, check with SAS Tech Support!

Technorati Tags: bundle, hotfix, order, SAS, server

We’re using v8, planning to upgrade to v9 shortly, and the box seems very slow. This may be because the server is doing more work – more users and larger datasets than I was used to – but nonetheless it seems a little sluggish and I’m suspicious of Win2k3, having been happy with Unix.

I’ve been reading up on performance and, to put it gently, there seems to be an opportunity to tune the system.

Firstly memory usage: our system has 3.2GB of memory but performance counters on the server suggest that only a fraction of this (say a tenth to a quarter) is used at any one time.

Paging: performance monitor logs show cache file usage and paging is very high – typically in the thousands. This is more to do with the way SAS reads data, a page at a time via the file cache, than a memory issue. Nonetheless I would be keen to bring this figure down as I’m concerned that the high usage stats may be creating a bottle-neck.

Looking into this further revealed that a typical SAS session, running on the server, uses about 12MB of memory. At this rate we could comfortably accomodate two hundred users – we currently have closer to thirty and it unusual to have more than twenty sessions running concurrently.

My plan: to increase memory usage so we can get data into memory quickly.

Technorati Tags: config, performance, SAS, v8

The price of computers has dropped to such an extent that, for the amount I paid for the original, I was able to buy a laptop and a desktop. Both are HP and were delivered quickly by dabs.com.

Now, with two machines to connect to the internet, I have invested in a Belkin N1 wireless router. This works fine, when it is working, but seems to drop the connection in the middle of the day just as my wife tries to connect.

Thankfully, I hope, a new firmware pre-release for the F5D8632 (1.00.07, an upgrade to the installed 1.00.06) is available from the Belkin site here.

Installing firmware on a new product is a scary job, but the routersetup system did the job and the router is working again.

Update (29 January 2009)

The firmware upgrade has been a success and has solved the problem – the router has been “always on” for the last ten days or so without dropping the connection even once.

Technorati Tags: belkin, hardware, HP, new, router, wireless