Social Networks Present Storage Challenge

The last few years have seen a massive upswing in the use of social networking technologies, from blogs and wikis to complex interactive Websites and even virtual worlds such as Second Life.

As more and more businesses implement the types of interactive technology more traditionally associated with Facebook and YouTube, Web 2.0 is taking their IT pros into uncharted waters where many will be forced to rethink their approach to storage.

Storage vendors are starting to focus their efforts on developing technologies for social networking. EMC, for example, recently announced plans for its top-secret "Maui" software, which is rumored to be a clustered file system that will compete with products like Isilon's OneFS or NetApp's OnTap GX.

EMC, like IBM, is also talking about the opportunities presented by "cloud computing," where users access distributed storage systems via the Internet to develop Web 2.0 applications.

"Cloud storage is massive, very massive," wrote Chuck Hollis, EMC's vice president of alliances, on his blog today. "We're routinely encountering new requirements where terms like 'gigabyte' and 'terabyte' are not useful; the discussion starts at 'many petabytes' and goes up from there."With all this in mind, let's check out how social networking is affecting a few companies today.

eBay braces itself

Paul Strong, research scientist at online retail giant eBay, acknowledges that social networking requires a whole new storage paradigm.

"The big difference with respect to social networking is that it is not transactional in nature, rather it is more like fixed content as people include pictures, audio, video, and so forth," explains Paul Strong, research scientist at online retail giant eBay, in an email to Byte and Switch.

With the demand for fixed content expected to rise, firms will be confronted with a different set of challenges, he warns. "This type of storage is essentially cheaper, i.e. commodity, but you end up with a lot more of it to manage," writes the exec. "You care less about the performance and robustness of your individual storage components because you want capacity."

Essentially, firms will end up mirroring recent trends in servers, where lots of cheap, commodity components are connected together, Strong thinks. "This shifts the emphasis to how you manage very large volumes of files and the meta-data associated with them," explains Strong.Currently, most of eBay's 3.6 Pbytes of storage is transactional, but like many other organizations, the retailer is looking to jump on the social networking bandwagon. Last October, for example, eBay launched its "Neighborhoods" initiative, which aims to create hundreds of "mini-communities" for eBay members based around specific products or interests, from Audi automobiles to the Bon Jovi rock band.

SMBs shape up
Its not just household names that are looking to embrace the brave new world of Web 2.0. Smaller firms are also wondering how to deal with the complexities of building storage for social networks.

Lexington, Ken.-based Blood Horse Publications mainly publishes magazines and books for the equine industry, but is now starting to look seriously at Web-based social networking.

"As the older generation [in the industry] starts to age out, we're seeing more of [social networking]," says Mike Gallenstein, the firm's senior systems administrator, explaining that this prompted the firm to launch bloodhorsenow.com last year.

Offering information and analysis on thoroughbreds, as well as some news clips and interviews, Gallenstein has high hopes for the site. "If our social networking plans take off, we could become the MySpace of the horse industry," he says. "This could include blogs, wikis, messageboards, and even Webinars.""It's pretty much going to be an open forum where users can discuss everything from breeding statistics to how a certain horse performed in a particular race," adds Gallenstein.

The exec nonetheless admits he has little idea of the Website's impact on his storage infrastructure, which is built around a 10-Tbyte SAN from Xiotech. "It could be significant, but it's just one of those things where you don't know."

If Blood Horse does face a storage crunch, then virtualization could prove key. "If [storage networking] does take off, we would definitely try to do some more virtualization to try to distribute the workload," says Gallenstein. "The Magnitude gives us the ability to virtualize storage, and it works with VMware quite well."

Not all SMBs are looking to handle this problem themselves, and Blagica Stefanovski, founder of CondoPerks.com, is happy to let someone else take the strain. "At this point, being a startup entrepreneur in the Web 2.0 space, you use whatever you can so that you can do things as quickly as possible," she says, explaining her decision to rely on storage from hosting provider HostGator.

CondoPerks.com, which Stefanovski describes as a "mini-mini intranet" for people living in condos, aims to hook condominium associations up with special deals from retailers, as well as provide local news via RSS feeds."As the company starts to grow, we would like to custom-build a lot more of our technology," she says. "I am a firm believer in doing things yourself, but there are a lot of companies that could help me do what we needed to do."

To Page 2

Wikibon's warning

As social networking becomes a reality, at least one expert warns that companies who wait too long could live to regret it. "The writing is on the wall that this is going to happen," says Dave Vellante, co-founder of the Wikibon user group.

He acknowledges that for most users, social networking represents a journey into uncharted waters. "Most of the investment today in storage is going into traditional block-based architectures, but as Web 2.0 emerges, that is going to change," he says. "You have got substantially greater data growth with unstructured information."

The exec agrees with eBay's Strong that the type of storage system required for social networking will be radically different, particularly in terms of capacity. "The first difference is that you’re talking about scaling to Petabytes, rather than Terabytes, and instead of hundreds of users, you could be talking about thousands."Firms should follow the example of Google, which has built a gigantic storage infrastructure out of commodity components, according to Vellante. In particular, the exec highlights the importance of the search giant's Google File System (GFS). "GFS assumes that the servers will fail -- the file system says 'Not a problem,' and it shares everything out again."

Virtual virgins
One of the most unusual developments in social networking is the recent emergence of entire 'virtual worlds', such as Second Life and There.com. Not just a playground for nerdy gamers, more and more firms are now looking to these sites as an important business tool.

This is particularly true in the storage sector, where a number of big-name players are already exploiting Second Life as a way to network, recruit, and even market their products. In 2006, for example, IBM used Second Life to a host a "town hall" meeting in Beijing, and Big Blue makes extensive use of the platform for internal meetings.

EMC has also hosted virtual job fairs within Second Life and Sun opened an "in-world" pavilion for its technology a little over a year ago, underlining the current shift from physical to virtual networking.

Other organizations using this type of technology include the U.S. Army, BP, and Intel, although analyst firm Forrester warns that it is still early days for the virtual world. "You've practically got to be a gamer to use most these tools," wrote analyst Erica Driver in a recent report. "Setup can be arduous, navigating in a 3-D environment takes practice, and processing and bandwidth requirements remain high."Like other social networking technologies, there is a big technology question mark hanging over the virtual realm, something which is likely to make IT managers nervous when large numbers of end users take a spin around one of these sites. "The nature of these applications means that demand could increase at any moment as avatars relocate, stream video, or upload CAD designs," explains Driver. "This spiky, processing-intensive application is precisely the kind of thing that IT directors hate, as peak loads make it nearly impossible to plan capacity accurately."

Despite these challenges, the analyst says that virtual networking is here to stay. "Within five years, the 3-D Internet will be as important for work as the Web is today -- information and knowledge management professionals should begin to investigate and experiment with virtual worlds.

Have a comment on this story? Please click "Discuss" below. If you'd like to contact Byte and Switch's editors directly, send us a message.