What is data science?

By Nick Hortovanyi

"The future belongs to the companies and people that turn data into products" says Mike Loukides on O'Reilly Radar. His "What is data science?" article, is an interesting read, that can be found here http://oreil.ly/dknxJV.

I've used some of the tools mentioned like the Python programming language and the Beautiful Soup library to clean up HTML. It has allowed me to deliver some analytics that have combined together data from multiple sources over the internet to project some predictions about the future. In one customer assignment, I combined together Australian Federal Government data, with local State based Population Projections to effectively create a wealth of data about future market shares. This was all done with Python and Beautiful Soup on my Mac Book Pro. I didn't need my own database or data warehouse as I was working with thousands of bits of summary data that was readily available over the internet. 

In other activities, I've used Apache Hadoop and its Map Reduce framework to process Australian financial market statistics on the full trading day history of all 2000+ listed companies on the ASX. I've also recently investigated Apache Mahout with its machine learning capabilities and am in the process of learning Apache Pig & Apache Hive to store and process data on top of Apache Hadoop.

All this software is free open source and scales to process large volumes of data on commodity infrastructure.

However, some strong analysis and programming skills are required. I'm working on advancing my knowledge also of statistics that are pertinent to these endeavours. In the past I've found the O'Reilly's book Programming Collective Intelligence to be excellent.

I agree with the Hal Varian quote also mentioned in Mike's post  "The ability to take data -- to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it -- that's going to be a hugely important skill in the next decades."

Tags trends data visualization datascience analytics statistics   |  Comments 0

What Innovation in Enterprise IT?

By Nick Hortovanyi

Big Tech is Broken - badly  writes Vinnie Michandani. It makes for an interesting read.

There is lots of talk about innovation in Enterprise IT (and vendors) but not much in regards to R&D budgets to back up the claims being made it would seem.

The most poignant point being made is that big technology companies are inefficient organisms - much goes in, little comes out in innovation. No wonder they spend billions acquiring companies to seek the magic outside their walls!

Tags innovation enterprise it   |  Comments 0

Has Gartner been pulling my leg for the last decade on Portal technology?

By Nick Hortovanyi

I keep reading how IBM's WebSphere Portal technology has been the leader in Gartner's Magic Quandrant for Portal technology for years. In fact, its the been the case ever since I can remember going back to WebSphere Portal 4.1.

Really can't see how that can be the case. Its just not reflected in any of the markets I've engaged in.

In the early naughties I worked on telstra.com applications to build and provision online communities. All the development work, I was involved with was Java based using the early J2EE 1/1.1/1.2 speced application servers like ATG Dynamo, iPlanet, BEA WebLogic and IBM WebSphere. One of them, which I won't name, would only start up with out crashing during its internal boot process every now and then. These were crazy times but enjoyable.

After these endeavours, I realised that there were clear patterns for frameworks to provide the plumbing and reduce delivery risk. In fact, as the Internet bubble had burst, most customers, 1) weren't interested in funding framework building and 2) didn't understand/weren't willing to learn what was required in a framework (most of the effort was behind the scenes). Thus products should emerge to provide these frameworks. In essence, when a developer was going to be working on an application I wanted them to focus on the Use Cases (that add value to the paying customer) and not the plumbing.

So I went hunting for a product, that was Java enterprise based and found WebSphere Portal. I'd seen how Oracle had wrestled market share from its competitors and gained the number one spot and thought IBM would do the same in this genre. Gartner confirmed it that they were number one. iPlanet the Sun Netscape alliance didn't last and BEA & Sun Microsystems have now been acquired by Oracle. Not sure about ATG Dynamo, but I never really heard of it again. So my decision back then based on the Gartner magic quadrants was correct, wasn't it?

From my perspective, its been a resounding NO.

Outside of the larger organisations, both public and private, there are small pockets of WebSphere Portal. Everywhere you look there is Microsoft Sharepoint and if they use SAP, nearly without fail a SAP Portal.

In the larger organisations where integration between many disparate systems is an issue or they have had a large investment in other IBM middleware/technology there are significant deployments. The organisation demographics between the US and Australia is also interesting. I understand per capital we have more SMEs and less larger organisations here.

So from a Worldwide perspective the Gartner Magic Quandrant data may well be correct, if and only if its used in context of large multinational organisations. So have Gartner been pulling my leg or was I just naive to trust their magic quadrants?

Tags portal enterprise gartner technology websphere websphere+portal   |  Comments 0

Who said it was expensive to get a new web site up?

By Nick Hortovanyi

After spending zero dollars, I have created a new web site newco1.com.  Its integrated in with twitter @newco1 , has a Facebook Page and its own Linked In group.

It even works with the iPhone and iPod touch and the newco1.com theme was done without a Graphic Designer.

This has all been done in a day.

Why did I do it?

Well, I'm working on a new lean startup, which I'm hoping will be CollectiveCycling.com (still in stealth mode). What I'm endeavoring to do is to use NewCo1.com to show my journey. The goal being that if I share what I learn, that others may benefit, as well as assist me to shorten the time it takes to iterate through the idea.

This is important as my time is the highest cost that CollectiveCycling.com has at this time. This is also invariably the case with most entrepreneurs. So you are welcome to participate through NewCo1.com or just to observe. Its up to you!

Tags linkedin facebook wordpress newco1 observe journey twitter startup participation lean   |  Comments 0

Bridging the IT generation gap

By Nick Hortovanyi

An excellent post, "Bridging the IT generation gap" over at infoworld.com about the IT generation gap.

"Older generations learned tech. The younger generation lives it. Organizations that want to succeed need the skills of both."

I see this every day I'm engaged in the IT workforce here in the real world city of Adelaide.

Key observations reproduced here from the above post.

..... 

IT generation gap: The death of 9-to-5
Memo to the old farts: It's OK to update your Facebook status if the job's getting done
News flash for young punks: Try not 2 text your BFFs during company meetings, K?

....

News flash for young punks: Don't move fast if you don't know where you're going

....

News flash for young punks: There's more to software than a pretty interface

....

News flash for young punks: Think twice before you post something we'll all regret

....

News flash for young punks: Listen to their war stories. You might actually learn something

....

The questions I now face, am I an old fart or a young punk or a combination of the two? What are you? 

Tags facebook gap it twitter generation friction socialmedia   |  Comments 0