IT Portfolio Management

Most ITPM solutions have implemented the core functions of project management as defined by the Project Management Institute, a 112,000-member nonprofit association. These functions include:

• time management, for tracking timelines, milestones and deliverables;

• resource management, for scheduling and leveling of resources through integration with external sources of data;

• skills management, for providing a searchable repository of employees' skills and experience to aid in assigning resources;

• cost management, for estimating and tracking all hardware, software, staff and training costs;• procurement management, for requesting resources and materials;

• risk management, for quantifying, monitoring and analyzing risk;

• reporting and forecasting, for reporting on project and IT department performance based on cost, benefits, risk and resource capacity, as well as project timeliness; and

• project management, for shepherding requests through the selection, prioritization and implementation phases.

Although ITPM products cover many different aspects of project management, they typically operate on the same basic premise: IT projects must be aligned with business units' objectives and stay within budget. Perhaps the most compelling reason IT needs such products is to automate planning budgeting--something IT departments are notoriously bad at. Without a formalized process, projects are selected and priorities are set only after months of paper shuffling, compromises and meetings.ITPM's Two Roles

ITPM helps bring IT and its clients together. Clients--meaning other departments--interact with IT through requests and status updates, while IT uses these products to determine which projects are feasible from a financial and resource perspective.

In a scenario we created for this article, using our fictitious midsize manufacturing organization NWC Inc., the company's customer-service department requested a way to improve its customer-facing order-entry system. IT's client, the customer-service manager, used the ITPM products in our review to enter a request for just such a project.

Once the client did her part, the IT manager put together scenarios for two projects that would meet the business's needs: upgrading the existing order-entry system or implementing a new one. The IT manager gathered data on new hardware and software costs and on all labor hours required to implement, deploy and manage the existing and new systems. Further, the manager used the ITPM products to weigh the risks, such as implementing new technology and even vendor viability, based on NWC Inc.'s specific concerns for each factor.

After the data was entered, both parties could compare each option's total cost and staff capacity. The IT manager could then determine which project would best meet the client's needs and fit the budget--just as a fund manager for a 401(k) would determine which investments best fit his clients' needs.In this manner, ITPM also facilitates the shift back to IT as a service organization: Internal business customers make the requests, and IT does its best to fulfill them. Use of the software fosters better relationships between IT and its customers because clients can stay up to date on relevant projects' status throughout the planning and implementation. Alerts and notifications let both IT management and interested business counterparts keep abreast of projects' status and take action when, for example, a project begins to fall behind schedule.

Our customer-service scenario was only part of a bigger picture for our fictitious company. NWC Inc.'s IT department needed to manage the planning process more effectively as the number of projects the business wanted IT to undertake grew exponentially. For this review, these projects also included automating shipping, as well as more automation of identity management.

We challenged ITPM vendors Artemis International Solutions, Business Engine, Changepoint, Niku and Pacific Edge Software to provide us with ITPM solutions that would help NWC Inc. meet its challenges. We judged these products on their architecture, implementation, customization and core functionality.

Artemis sent Artemis 7, Changepoint submitted Changepoint 8.01, and Pacific Edge offered up Portfolio Edge 2.0. Business Engine agreed to participate but backed out after learning that we wanted to touch its product in the lab. Niku declined to be tested as well.

Our examination of the participating vendors' three solutions left us fascinated with the systems' capabilities and promise, yet cringing at the intense scrutiny under which IT staff inevitably will be placed. From IT management's perspective, ITPM is the solution to the requirement that IT not only provide, but also prove, value to the business--by initiating requests, defining the business justification and offering insight into the strategic nature of top-priority projects. Clients can track progress and better understand the project life cycle without haranguing IT counterparts.

This level of visibility, for both the business and the CIO, promises a tighter-run IT ship. With a complete view of IT projects and life cycles, the CIO can analyze current and planned projects, as well as consider risk and benefits, strategic alignment, and budgets.What's all good for the CIO, however, can strike IT staff as too restrictive and without direct benefits. The issue of time accounting--how much time an employee spends on each project--has forever been a problem within IT.

IT staffers often chafe against the yoke of strict time accounting. Often, the data is invalid or based on guesswork, severely limiting an ITPM product's ability to aid strategic budget and planning decisions. IT managers who were promoted out of the technical, rather than business, ranks during the '90s will have to learn to use cost accounting and financial data. Without such business acumen, it will be difficult to adjust to ITPM solutions and supply the data necessary to perform cost and benefit analysis of projects.

Take All the Help You Can Get

These are not products you'll be implementing on your own. You'll need training and professional services help. Our grading of price includes at least two weeks of such assistance to emphasize this requirement. Artemis, Changepoint and Pacific Edge did not let us install their solutions on our own. Although each installation in itself was straightforward, configuring and customizing each product requires the vendor's assistance.

Before the products were useful in our lab, we also needed to understand the business relationships among projects, finance and accounting rules, and our own company's business jargon. All the products we tested use a hierarchical structure to organize projects; it was our job to build that hierarchy and determine how to account for assets, labor and projects. Should assets be tied to a project or accounted for separately? Should projects be grouped by client or by strategic objective?There is no right or wrong answer. In NWC Inc.'s case, we organized projects on the basis of what strategic objective a project served: customer service, supply-chain management support or internal infrastructure support. Determining this structure, however, requires business-IT cooperation.

Although regular IT and business staff don't need to worry about the deeper financial aspects of project management, IT managers and system implementers/administrators will need to configure such things as NPV (net present value) calculations, financial metrics, cost centers and (hey, you, wake up!) discount rates. The assistance of anyone who understands these figures will be necessary and, we anticipate, appreciated.

After taking each product through our scenarios, we gave our Editor's Choice Award to Changepoint's Portfolio Management 8.01. With its out-of-the-box collaboration and the ability to capture intangible benefits, such as customer satisfaction and peer-performance indicators through its survey features, Changepoint's product provided more automation and collaborative benefits than the competition. We not only could automate planning and prioritization, but we also had a mechanism for soliciting feedback from clients. Although all the products we tested will integrate with Microsoft Project for time reporting, Changepoint's IPM (Individual Performance Metrics) offered a method of encouraging compliance with time-reporting policies so as to lead to higher adoption rates and better data quality.

In this product, the eye candy is not only flashy, it's also highly useful. This highly customizable, secure solution offers an easy-to-navigate, well-organized, Web-based user interface for both IT and business staff. We found its fully searchable knowledge-management system extremely useful, too.

Changepoint PM provides a wide variety of customization options, as well as a portal from which clients can interact with the system and the IT department. PM was a clear winner, with its ability to provide alerts and notifications for such details as new requests, approvals, completed tasks and project assignments throughout a project life cycle.

After logging in to the system, the user sees a personalizable "home page" that offers at-a-glance status of projects, assigned tasks--such as approval of requests and project-level items--that need attention, and a menu of options for managing time sheets, expenses and interaction with other IT staff and business counterparts.Changepoint's client portal site allows customers limited interaction with the system. Logged-in clients can submit requests for new projects or changes to existing projects and defect reports, and track those submissions as they move through the planning process. Clients can view and modify those requests, but cannot see the same level of detail that an IT manager or staff member sees. Although clients can view their requests' status--percent completed and estimated timelines, for example--they can't drill down to see specific project-related data, such as resource rates and individual IT staff time spent on their project, or technical knowledge-base items shared between IT project team members. Such information is not considered essential to the client's role within Changepoint.

Changepoint's security model is extensive--overwhelming, really. Although pleased that we could have such granular control over field-level access, we found the process' tediousness disheartening. For instance, the list of access rights includes privileges to which you're not licensed. If you're trying to choose an entire category, such as "manage clients," but are unlicensed for even one item in that group, you're denied access. We'd like to see this process smoothed out a bit.

All aspects of the application, including much of the interface's look and feel, can be customized for your organization, but this process takes many hours. Still, being able to adapt the application to your own business will speed up the adoption process, so it's worth the time. Both Changepoint PM and Portfolio Edge are highly customizable, likely owing to their implementation architecture. Artemis is customizable in many ways, but interface manipulation isn't something that's easily done within that product's J2EE architecture.

Approval of project- and nonproject-related time and expenses is easily configurable in Changepoint PM, as are other accounting tasks. The client can be designated as an approver on charge-backs, which smooths the collaboration between IT and its clients. One- and two-level approvals can be configured. Only PM includes expenses and charge-backs--one way this product excels over the competition. During testing, after entering the time I spent on different projects during the week, I logged on to PM as the approving manager and saw the approval waiting on my dashboard.

All the products we tested are diligent about the breadth and depth of support for managing requests. It is this core functionality that helps executives align IT with the business, so we were not surprised to find that all three products performed these tasks well. In PM's case, the software manages all types of requests--planning, work, change, defects--and gives teams access to specific queues. Planning requests can be routed through your approval process and may include any number of IT and non-IT users from whom you can gather data, documents or input. Furthermore, using PM to organize the requests by time period (yearly, quarterly and so on) helps the CIO determine which projects to include based on cost, risk and a host of customizable variables.Reporting and charts were, of course, good-looking. But they were also quite functional--we especially like the bubble chart, a handy tool for plotting three or more data points in one graph. All charts are configurable, and let you drill down all the way to the individual task level. This is true of all the products we tested; the differences between the solutions lie in how they generate charts and reports rather than their capabilities.

Changepoint PM's deployment model, like that of Artemis 7, requires a SQL Server or Oracle database as its repository. Unlike Pacific Edge, Changepoint relies only partially on Microsoft's .Net framework. Its administrative console is completely .Net-based, while its user interface is based on both ASP and .Net. Changepoint is in the process of migrating its architecture completely to .Net, but is taking its time in doing so.

Changepoint 8. Changepoint Corp., (800) 263-7189, (905) 886-7000. www.changepoint.comWith Artemis 7, we learned quickly that NWC Inc. had overallocated developers and figured out how delaying a project would affect the IT department's capacity. Outstanding visual representation of a project's cost and resources within a time period helped us draw the first conclusion; unique what-if scenario generation clued us in to the second.

Artemis 7's cost and resource-capacity capabilities beat the competition. After we entered all of NWC Inc.'s requested projects, Artemis 7 summed up the network engineer, developer and system admin labor hours necessary for every project scheduled, and we compared those figures to the number of labor hours available for those resources for 2004. Using a graphic visualization, the product showed that the number of hours scheduled for all resources was more than the available labor hours.

Armed with this knowledge, we set out to solve the problem, which Artemis handled deftly. The software showed the impact of each project on our resource capacity and how adding or removing each project would affect overall capacity. We could even drag a single project's resource allocation in time to see what effect a delay might have on capacity. Neither PM nor Portfolio Edge let us create this what-if scenario and use it so effectively.Both Artemis and Changepoint PM provide a personal dashboard for IT users that includes notifications of task assignments, pending approvals and request statuses. We were impressed with Artemis 7's inclusion of discussion feeds related to projects, as well as with what this feature added to the collaborative aspects of the product.

Unlike the competition, Artemis 7 comes with very few metrics. Artemis does have an advantage in its rule-building for metrics, as it uses a method similar to Microsoft's Query Builder to create metric calculations. Pacific Edge's use of MathML makes for a more flexible system, but may cause some users to panic at the thought of using an XML-based language to build metric rules.

Like Changepoint PM, Artemis 7 supplies simple work-flow features, such as approvals of time sheets and project requests. It was easy to add a number of approvers, who'd receive the requests in a specific order. After you log in as an approver, the investment shows up in your workspace. Once you approve the item, it's automatically routed to the next approver.

One Java app handles both customization and administration. That app, which is launched from the Web site, is accessible to all users. Menu options are limited to only what is allowed on a per-user basis. Security is accomplished via roles, and all user administration is manual.

Artemis indicated that the system can be integrated via XML and its Web services-based API; however, the vendor provides no means of integrating with any third-party application automatically except MS Project.The product uses a publish-and-subscribe model for its notification and alert capabilities, which appear on the user's home page upon logging in to the system. The vendor wisely includes methodology templates to describe projects' phases. As with Changepoint PM--but not Portfolio Edge--Artemis' time-management capabilities are integrated, so you won't need a third-party application.

Artemis 7 is a J2EE application supporting deployment in a few select application servers and platforms. We were confused by the limited support--because the application is J2EE and can run within WebSphere on Solaris, we would have thought it could run within WebSphere on Windows. So much for standards.

Artemis 7. Artemis International Solutions Corp., (949) 660-7100. www.aisc.comPortfolio Edge 2.0 was the most customizable of the three ITPM packages we tested. PE can be adapted easily to manage portfolios for any type of business--IT, manufacturing and insurance, to name a few--without a hassle. The downside to this approach is that implementation is slow--even with the "templates" included in the system, as you'll want to create your own templates to fit your business model. The upside, however, is that all lines of an organization's business can use a single portfolio-management product.

Out of the box, PE offers a host of metrics for reporting and analysis. The product's graphical-analysis capabilities are as appealing as Changepoint PM's and offer equally strong drill-down capabilities and flexibility in customizing reports. Calculations for metrics and reporting are extremely flexible; PE utilizes MathML, which can be used to create just about any type of calculation. We liked PE's ability to adjust project timelines automatically, without requiring project managers' manual intervention. Artemis required us to send an alert to project managers, instructing them to move projects up or back in time, whereas PE allowed anyone with the appropriate rights to do this automatically.

As with Artemis, we discovered we had overallocated staff during the second quarter of 2004. The IT manager could choose a project with a lower prioritization and click the "delay" button. By specifying a new start date, the project was modified automatically to start on the new date, changing all relevant staffing needs and moving all associated costs. This process lets the IT manager move projects in time and immediately gauge the effect of doing so on the budget as well as staff workload--though it's not as elegant as Artemis' what-if capability.Work flow and automation are more difficult to implement in PE than the competition, as they require custom coding to implement. A companion product, ProjectOffice (a competitor to Microsoft Project), provides basic routing and work flow. PO integrates handily into PE, though Microsoft Project can also easily be integrated to provide complete project-management data for PE.

Simple approvals can be used within PE, but out-of-the-box approval and automation capabilities of PE lag behind the competition. We'd like to see more automation without the requirement to code.

Notifications and alerts were also missing from the base PE product. Pacific Edge provides these capabilities as a module rather than integrating this base functionality within the product, as Artemis and Changepoint have done.

PE's method of scoring--that is, prioritizing projects based on how well they meet business objectives in light of the associated risks and benefits--was complex but well-suited to the enterprise environment. Weighted grading of risk, benefit and strategic alignment let the system automatically assist in the prioritization process. The ranking process within PE was more complete out of the box than that of the competition.

PE is completely .Net-based and, like Changepoint's solution, requires IE 5.5+ as its browser. With PE, you can import users from directories (LDAP, ADS); the vendor plans to offer authentication via ADS in Q4 2004. Database support was also limited to SQL Server; Pacific Edge indicated that Oracle support is forthcoming.Portfolio Edge 2.0; Project Office 4.1. Pacific Edge Software, (425) 897-8800. www.pacificedge.com

Lori MacVittie is a Network Computing technology editor working in our Green Bay, Wis., labs. She has been a software developer, a network administrator and a member of the technical architecture team for a global transportation and logistics organization. Write to her at [email protected].

Post a comment or question on this story.

As upper management demands business justification from the IT department, choosing and managing projects wisely has become a crucial task. IT portfolio-management products provide a sophisticated, though cumbersome, solution. These software systems let IT managers make well-justified decisions about which projects to start, drop or back-burner, while they give the rest of the company access to relevant data about each IT investment.

ITPM products include tools for tracking projects' time, resource and skill demands, as well as managing costs, mitigating risks, and shepherding each request through selection, prioritization and implementation. Analyzing the solutions' data requires a level of business acumen that rarely comes naturally to technical workers. Furthermore, IT staff may balk at the painstaking time records they must keep to make the products effective.

We tested three such packages: Artemis International Solutions' Artemis 7, Changepoint's Changepoint 8.01 and Pacific Edge Software's Portfolio Edge 2.0. All did their jobs well once we got the training we needed. Changepoint, however, wowed us with its broad list of features and high level of automation.We tested the products by implementing them within our NWC Inc. infrastructure and attempting to integrate them with various existing systems--Active Directory for user authentication, as well as Oracle and SQL Server.We entered the same data into each system: the hardware, software, applications and resources that represented NWC Inc.'s assets. Next, we walked through two scenarios to evaluate how each product handled these common jobs.

In the first, NWC Inc.'s customer-service manager requested a new customer-service application. This scenario would flex each product's work-flow and automation capabilities, assisting us in evaluating each solution's method of scoring and prioritizing projects according to strategic objectives and quantification of risk factors.

The second scenario involved a downsizing: NWC Inc. would have to cut its budget by 20 percent. We used each product to determine which projects needed to be cut or delayed to achieve such a reduction. We evaluated each product's method of helping us make this decision, using the software's graphs, reports and collaborative tools.

In both scenarios, we scrutinized the inner workings of each solution and considered how customizable the product was, how formulas were specified and what would be necessary to implement the product fully in a production environment.

R E V I E W

IT Portfolio Managers

Sorry,
your browser
is not Java
enabled

Welcome toNETWORK COMPUTING's Interactive Report Card, v2. To launch it, click on the Interactive Report Card ® icon

above. The program components take a few moments to load.

Once launched, enter your own product feature weights and click the Recalc button. The Interactive Report Card ® will re-sort (and re-grade!) the products based on the new category weights you entered.

Click here for more information about our Interactive Report Card ®.