In Search Of Open-Source Experts

Applied Industrial Technologies Inc., a distributor of fluid-power and engineered products, runs most of its infrastructure on Microsoft Windows. But two years ago Applied Industrial began running some Web, file, application, and directory servers on the open-source operating system Red Hat Linux. "Our technicians [were] saying, 'we need to have Linux servers in place--we could pull down costs,'" says IT director Bob Falkowski. "We changed out hardware, introduced new software components. But what we failed to realize is, when you do this type of process there's some added burden. You have to fall back on yourself as being the ultimate solution provider when things don't work."

Applied Industrial Technologies isn't the only company to be caught short by the effort and expertise required to support an increasing number of open-source projects. Companies often look at the bright side of deploying Linux and other open-source systems--the cost savings, the standardization, the freedom from vendor lock-in--but aren't well-enough prepared for the challenges that come with implementing or expanding the use of technology that's still in the early stages of development.

It was no surprise when Microsoft CEO Steve Ballmer issued a memo last month asserting that Windows computing environments offer, among other advantages, a lower cost of ownership than Linux and open-source systems. What may be surprising is the correlation between Ballmer's claims and many companies' real-world experiences. Not only did Ballmer point to independent analysts' reports to bolster his case, but there's growing evidence that many companies are running into unexpected costs, especially in connection with implementation and support, when trying out or switching to open-source systems. Even the costs of signing on with an IT vendor to support open-source deployments may cut into their estimated cost savings.

"It was a learning experience for us," admits Applied Industrial's Falkowski. The company now is "going through the painful process of developing the know-how," he says, with a big focus on retraining staff. Applied Industrial won't disclose how much it has invested in efforts to ramp up open-source expertise, but it has negotiated Linux training from IBM as part of a contract to replace its Tomcat open-source Java servlet and JavaServer Pages with IBM WebSphere. "We looked at the positive aspect too heavily and didn't look at the negative aspects enough."

One of the chief complaints of CIOs and CEOs is that they can't find enough qualified open-source programmers, says Faber Fedor, an open-source consultant with services firm Linux NJ.com Inc. "I don't think there are enough experienced people out there for the demand," he says. Yankee Group analyst Laura DiDio agrees. "There's a dearth of skilled Linux administrators, by comparison to the more-mature Windows, Unix, NetWare, and Macintosh environments," she says. And what happens when too much demand meets too little supply? "They can command a premium," DiDio says. "They get a 20% to 30% salary premium in the large metropolitan markets."The shortage of open-source talent that Linux adoption is driving at the enterprise level will even out over time as colleges ramp up open-source programs and today's graduates become tomorrow's Linux experts, says Robert Jones, president of HotLinuxJobs, a division of Glacier Technology Services Inc. For now, there's a direct connection between how urgent a company's need for Linux talent is and how much it's willing to pay.

Some early adopters of open-source software in the data center recognized there would be no ready pool of talent to help with the effort but were lucky enough to have Unix pros on staff whose experience on that formerly open-system operating system translated to Linux with comparative ease.

"A Unix specialist and a Linux specialist are really cut from the same cloth," says Mike Prince, CIO at Burlington Coat Factory Warehouse Corp., which has been using Linux and the Apache open-source Web server since 1999. Two years ago, Burlington Coat rolled out point-of-sale systems running Red Hat and back-end data-center equipment running SuSE Linux. The company's large Oracle databases, now running on Linux, had to be switched over from Unix. The company had 15 seasoned Unix specialists on hand when it began the migration, which helped provide needed data-center experience.

It's a different story for companies that are primarily Microsoft shops. A total switch from Windows to Linux, or even a significant Linux deployment in the middle of a Windows environment, will be three to four times more expensive--and take three times as long to deploy--as an upgrade from one version of Windows to a newer release, according to a Yankee Group study Ballmer cited in his memo. He also cited a May Forrester Research study that said Linux training is on average 15% more expensive than Windows training. The author of that report, Forrester VP Julie Giera, says the caveat there is that her report's sample was very small, mainly because she couldn't find many companies who'd been running Linux more than a year and who closely tracked costs associated with the deployments.

KeyCorp spent $130,000 to get its Linux strategy off the ground during the fourth quarter of last year and the first of this year. Only about $10,000 to $15,000 was needed for initial training classes for staff Unix engineers. The rest went to Red Hat for helping the company do a total-cost-of-ownership study and develop a good open-source migration strategy. "Since then, this has very much been a bootstrap operation, with each successful five to 10 implementations driving demand for additional cross-training of our Unix staff and that additional cross-training enabling us to implement even more systems," says Bob Dutile, executive VP of Key Technology Services, KeyCorp's technology division. KeyCorp plans to migrate file, print, and document-processing apps this year to Red Hat Linux running on HP Intel-based ProLiant servers. Dutile expects that by the end of the year KeyCorp will have as many as 55 servers running Linux. "The specific training costs since the first quarter have been very low and have all been absorbed within the normal departmental operating budgets," he says.While Dutile is reluctant to hire programmers right out of school--"I don't know that I've ever found a developer out of college who was ready for the market"--he does have a trick for finding good talent. He sometimes searches Google and SourceForge.net, a repos- itory for open-source code that's maintained by VA Software Corp., to see which programmers are working on projects that demonstrate skills valuable to KeyCorp. This lets Dutile take a look at a programmer's work. "If I was hiring a programmer from another bank, it's not likely that bank would let me go and evaluate the code that person worked on for them," Dutile says.

SourceForge has more than 900,000 registered users and includes a repository of more than 89,500 open-source programs under development. Although VA Software won't divulge the specific demographics of its user base, the company confirms that more than 40,000 companies, government agencies, and other organizations have two or more users registered with SourceForge. A percentage of those other registered users may be high-school kids or hobbyists, not potentially employable talent.

Open-source expertise is building. Young programmers such as Columbia Public Schools' Norris find it hard to envision a world without open-source software. He's considering going to school to study computer science. One of the main reasons? "It would let me contribute more back to the open-source community," he says.

And companies can contribute to the growing pool of open-source talent. Applied Industrial positioned its open-source training as a career-enhancing step. "We offered people here a way to reinvent themselves," says IT director Falkowski. But considering that Windows servers outnumber Linux servers in his company by a ratio of 4-to-1, it's understandable that not everyone was eager to leave a well-established environment for the less-established world of open-source. Most of those who are retraining, Falkowski says, have backgrounds in Unix and C or C++ programming skills.

The open-source world, like any emerging environment, is fraught with peril. There's the threat of lawsuits--spurious or not--by vendors claiming open-source software infringes on their intellectual property. But potential users of open-source systems also have to consider that there might not be enough expertise to support their projects adequately and that getting that expertise can be more expensive--and more of an effort--than they anticipate. Is open source worth it? Steve Ballmer has his opinion, but most companies are going to have to answer that question for themselves.--With John Foley