Application System Development Guidelines

I. Purpose and Goals

This document contains guidelines for developing a major application system on Campus. "Major" means either a system that has users in more than one department, or a single-department system that is expected to cost more than $100,000, in direct costs, to develop and implement. Direct costs include hardware, software, and contract personnel.

The intent of these guidelines is to:

Increase the likelihood that developed systems will be both effective (meet the needs of present and future users) and efficient (have a reasonable initial cost; and reasonable operational, support, and enhancement costs).
Assist in a clear understanding of, and agreement on, the roles that different departments should play in the development of a system.
Help users and developers understand and agree on appropriate steps in the development process.

These guidelines apply to all departments (administrative and academic) on Campus, and cover both purchased vendor packages and systems created by in-house developers (programmers). They pertain only to applications in the administrative and student services areas. Following these guidelines is particularly important if the system being developed is used for financial or management purposes.

If a department is developing a major application system, and does not intend to follow some or all of these guidelines, the department head should notify the Associate Vice Chancellor—Information Systems and Technology (IST) of that intent. IST will not ensure support for any application which does not conform to all applicable campus information technology standards, particularly with respect to interoperability, accessibility, and communications compatibility.

For systems not covered by these guidelines, it is recommended that departments review these guidelines and select those parts appropriate for their systems.

II. Responsibilities

The project ownership of a major application system resides with the department(s) that will use the system—even for campuswide systems that Information Systems and Technology (IST) develops or operates.

Each major application system should have a "primary department" (the department that will use the new system the most, or the one that "owns" the data). For successful system development, it is important that the primary department have one or more individuals who will be able to spend sufficient time, over the life of the development process, on project-related work.

Where a unit of IST plans to develop a system that will support multiple departments independently, the Associate Vice Chancellor—IST or his/her designee will request interested departments to volunteer to represent the user community, and will formally create a system committee consisting of employees from user departments. The committee will determine one owner of the system for accountability purposes.

III. Budgets and Time Estimates

Each new system should have a detailed budget showing both projected initial (one-time) costs and projected on-going costs of the system (both programming and operational). Any budgetary (hard dollar) costs that are expected to be reduced, such as the cost of operating a system to be replaced, or additional revenue that is expected to be generated, should be stated and explained at the very beginning of the system development process.

Projected costs and time estimates made in the early phases of the development of a system should be stated as a range of possibilities. It is particularly important, early in the life of a system, not to commit to or publicize a very specific implementation date, because of unavoidable uncertainties. The focus of both cost and time management should be the cost and time estimated to complete each phase, or the parts of each phase.

At the completion of each phase, projected costs and time estimates should be revised and the range of time estimates narrowed. The revised budget and schedule should be published after approval by the head of the primary department and (where IST is participating in a partnership to develop the system) the Associate Vice Chancellor—IST.

The primary department is responsible for informing departments that will be affected by the implementation of the system, (1) what the existing time estimates are and (2) when there is a significant change in the scope of the initial implementation of the system or in the projected implementation date or range of dates. The primary department is also responsible for notifying higher-level administrators of expected delays in completing a phase, major changes in the scope of the initial implementation, or major cost changes.

Key Phases of the System Development Process

1. Feasibility Analysis

1.1 Goals of this phase:

Key features of the new system, both functional (on-line updates, context-sensitive help, etc.) and technical (cross- platform, relational database, etc.) should be explicitly stated.

Choices made at this point can be changed as new information (for example, that no vendor package provides adequate functionality at a reasonable price) is obtained in a later phase. The alternative—to leave major choices open until a later phase—means that it will be difficult to determine the feasibility of the proposed new system (absent hard cost figures), or the resources needed to develop it, or what a reasonable timetable will be.

To identify and document the improvements and changes that are expected from implementing a new system, and a very rough timeline (range of possibilities) for the completion of each remaining phase of the development process. Computer systems to be replaced should be identified; the boundaries (scope) of the new system should be specified; and the probable interfaces to other computer systems should be described.
To identify and document the significant development choices ("make" versus "buy" versus "share or borrow"; hardware and software architectures) that seem feasible, their relative costs and timeliness, and the proposed approach for the next phase (process reengineering/requirements definition)—for example, "buy a mainframe package that uses client/server technology".

1.2. This phase concludes with:

The presentation of a formal feasibility study to Administrative and Student Services Computing Subcommittee (ASSCS) of the Campus Computing and Communications Policy Board (CCCPB), if requested by that body. An initial, comprehensive budget for the system should be included.
Where the proposed approach is to purchase a vendor package, the issuance of a Request for Information (RFI) by Campus Purchasing to potential vendors, asking for vendor-specific information that will help the Campus evaluate whether the proposed approach is practical and appropriate.

1.3. In developing the feasibility study document and any derived RFI, the primary department should:

Work as a partner with IST in the feasibility analysis, whether the primary department or IST takes the lead in doing the analysis. The extent of IST involvement, and which IST units are involved, will depend on the extent to which IST will develop or operate the proposed system. The expected minimum involvement would be providing specific technical advice and assistance.
Ensure that affected departments participate in a meaningful and substantive way in the analysis. Participation through individual or small group interviews, while important, does not by itself constitute adequate departmental participation. Critical means of participation include periodic and announced meetings, circulation of draft documents, and membership on formal committees. The final feasibility study document should include the names of individuals (particularly those not in the primary department) who participated to a significant extent, and the extent or means of the participation of individuals not in the primary department.
Before completing the final version of the feasibility study, provide an opportunity to interested Campus departments to comment on a draft version.

1.4. The contents of the final version of the feasibility study, and other documents specified in these guidelines, are the (ultimate) responsibility of the head of the primary department, and of the Associate Vice Chancellor—IST where IST is participating in a partnership to develop the system.

2. Process Reengineering/Requirements

2.1. Goals of this phase:

To ensure that opportunities for improved methods and processes, supported by the new system, are fully pursued, rather than simply replacing, with enhancements, an older system, or simply automating manual processes. In particular, the value added by each step in the existing process, and the opportunity for radical restructuring of the process, should be closely examined.
To identify and clearly document what the new system should do (requirements). Requirements, in this phase, should clearly be related to the goals of departments (timely, accurate, efficient, and effective actions).

2.2. This phase concludes with the creation of either a Request for Proposal (RFP) to be sent from Campus Purchasing to prospective vendors, or one or more internal documents, at the same level of detail but without procurement-related information, to be used to design the new system in-house.

2.2.1. Documentation created at this stage should include, at a minimum, information about the following components (requirements) of the new system, and should be agreed to by users, developers, and those who will operate the system once it is implemented:

Functional (process, other than generating reports) requirements, particularly interfaces to other systems.
The design (user interface) approach.
The initial draft of data element requirements (names of data elements, descriptions, formats, and acceptable ranges or values).
How reporting needs of users will be met.
Technical/platform requirements, including performance. requirements (maximum connections, response time, etc.).
Security/backup requirements.

2.2.2. Documents created in this phase should not include the following, which belong to the next phase:

An extensive breakout of the system by modules.
Menus or menu samples.
Sample screens.
Output (report) samples.
Data conversion plans.

2.3. In developing the documents, those planning the system (the primary department, usually in partnership with IST) should:

Identify and observe automated systems, in use at other campuses and in other organizations, that perform the same functions as those of the proposed system.
Aggressively question, working with other affected departments, whether current processes can be significantly modified (reengineered) to reduce costs or improve services, rather than simply automating manual processes.
Take advantage of the technical expertise of IST to ensure that requirements are clear and complete. Where IST will develop or operate the new system, IST should formally sign off on documents.
Provide affected or interested departments with the opportunity to participate in the definition of requirements. Besides interviews and other information gathering, this should include, as a minimum, the establishment of a Steering Committee and an opportunity for interested departments to comment on a final or near-final draft of documents. The Steering Committee should include one or more members of IST. "Interested departments" will always include Internal Audit, and, for systems involving accounting or financial information, will include Accounting Services.

2.4. Documents should clearly state what functionality the system is intended to provide (where the boundaries are, and what functions lie outside those boundaries), and should explain, clearly:

How the system will integrate into the overall campus information technology environment.
Why certain features are not included (e.g., because of storage costs or excessive processing time).
Which features are planned as enhancements, after implementation, what the priority of the enhancements is, and approximately when (in terms of time after the initial implementation date) the enhancements are projected to be operational.

2.5. Once past the Process Reengineering/Requirements Definition Phase, additions to, or changes in requirements will be significantly more expensive to develop and probably will delay implementation of the new system, often significantly.

While such changes are often necessary, if not inevitable, they should be made only with full evaluation of their impact.
Often—if not in most cases—it will be best to postpone changes until after the system has been implemented, particularly for major enhancements.
If Rapid Application Development (RAD) is used, either in this phase (to define requirements) or in the next phase (to verify and refine requirements), then changes in requirements can often be done with minimal impact on total costs and schedules.

2.6. For systems of interest to ASSCS, the primary department, and IST if involved in developing or operating the system, will:

Report to that subcommittee on a periodic basis.
Bring to the attention of the subcommittee any significant or potentially significant budget deviations, in a timely manner.
Bring major contract or policy issues to the subcommittee for review and approval.

3. Detail Design/Vendor Selection

3.1. Goals of this phase:

To select or design the specific software, and the related computer hardware and networking enhancements, to meet the system requirements.
For in-house systems (or where significant modifications to a vendor package will be done), to design or redesign:

Data structures.
The aspects of the system that the user will use and interact with (such as menus and screens).
Business logic.

3.2. The conclusion of this phase depends on whether the system will be purchased (vendor package) or created in-house:

For purchased vendor packages needing minimal modifications, this phase concludes with the signing of a contract with a vendor. (Minimal modifications, for the purpose of these guidelines, means modifications costing less than ten percent of the purchase price.)
For in-house systems and purchased vendor packages requiring significant modifications, this phase concludes with the creation of menus, data display and entry screens, and standard reports (if used); detailed written descriptions of the business logic to be used for screens, reports, and any batch processing (including batch interfaces); draft documentation of business procedures; drafts of the user manual and the help screens; a detailed data element dictionary and descriptions of data edits and triggers; and detailed data conversion plans.

3.3. The benefits of any Campus modifications, other than parameters, to a vendor package should be carefully compared to the continuing costs of supporting that modification and dealing with questions and problems associated with having a mixed vendor/in-house system. Where the Campus wants to add functionality to a vendor package, it is usually best to try to get the vendor to add that functionality to the base (supported) version of the package, or to wait until the functionality is added as part of a periodical vendor update of the package.

3.4. It is critical that user involvement be maximized during this phase. Once a vendor package has been purchased, or detailed programming begins (in the next phase), the cost of user-requested system changes, in both time and dollars, increases sharply.

3.5. For systems developed in-house, the following actions should be taken to maximize user involvement and minimize surprises when the system is in production:

If IST is developing the system, the primary department should clearly agree (for example, by formally signing off) with the proposed aspects of the system, as listed above.
System developers should create prototypes, do mock-ups, and otherwise involve users with hands-on exposure to proposed aspects of the new system. These actions should be done in place of the expensive, limited-user-involvement approach of writing extremely detailed requirements documents that are "handed off" to programmers who then write code.
The drafts of the user manual and help screens should either be written by appointed or designated users, or extensively reviewed by users in an iterative process. Extensive user involvement with these two functions, early in the development process, will result in (1) clarifying, to programmers, what the system should do, and why, and (2) minimizing support needs once the system is implemented. On-line help screens should be designed for easy modification.

3.6. When deciding which software package to purchase, together with any related hardware and network enhancements, it is critical to evaluate the extent to which each package both meets users needs (functional, design, data element, and reporting requirements) and has a sound and easily supportable underlying architecture.

To evaluate whether a vendor package will meet user needs, users and technical personnel (from both IST and using departments) should see the vendor package in use and have the opportunity to ask questions of the vendor.
To evaluate the soundness and supportability of the underlying architecture, a technical subcommittee of the Evaluation Committee should meet with vendor technical personnel. It is critical that the technical subcommittee have a wide range of expertise, both of current technology and of the current and likely future hardware, software, operational, and network practices and policies of IST and the Campus.
The Evaluation Committee should recommend two or three vendor packages, preferably ranked, to the official who will make the final decision. The recommendations should identify strengths, weaknesses, and risks (for example, vendor financial difficulties) of each top vendor package.
Where IST will be expected to develop, modify, maintain, or operate the purchased vendor package, concurrence of the Associate Vice Chancellor—IST for the selection should be obtained.

3.7. When deciding which vendor package to purchase, it is normally efficient to narrow candidates in a set of steps:

At the end of this step, is should be possible to assign a tentative ranking to the vendors.

An initial evaluation can identify those vendors who merit further interviews—normally only those vendors who have an equivalent system in operation elsewhere. (An "equivalent" system is one that would require minimal or no modifications to be implemented on this Campus.) If there are very few or no vendors who meet this criterion, the feasibility or cost- effectiveness of either purchasing or creating such a system in-house should be seriously reassessed.
Face-to-face interviews, including vendor presentations, can identify vendor strengths and weaknesses, potential risks, and critical (unanswered) questions. Functional and technical requirements should be put into separate matrices that are used to evaluate each vendor package.
Visits to sites where the system is already in use are normally the final step. An evaluation checklist should be prepared before any visits, as a starting point for obtaining information. Those doing the visits should carefully note where the system will require modifications or enhancements to meet Campus needs, and where implicit processing procedures are not consistent with Campus practices.

3.8. Internal Audit should be provided the opportunity to participate actively in the design of data reliability and security controls, and to comment on written documents generated in this phase before the documents are finalized.

3.91. For financial systems where Accounting Services is not the primary department, Accounting Services should be provided the opportunity to participate actively in the design of accounting-related controls in the system, and to comment on written documents generated in this phase before the documents are finalized.

3.92. Normally, programming should not occur until the next phase (Implementation). However, if Rapid Application Development (RAD) or a similar approach is being used, parts of this and the next phase can be done simultaneously.

4. Implementation

4.1. The Implementation phase includes any programming for the system; unit testing, system testing, and user acceptance testing; any data conversion; training; and establishing operational procedures.

4.2. Where IST is participating in the implementation of the system, IST and the primary department should agree on the responsibilities and methods for:

Finalizing screen designs and any standard reports.
Modifying detailed written descriptions of the business logic to be used for screens, reports, and any batch processing (including batch interfaces).
Finalizing the documentation of business procedures.
Finalizing the user manual and on-line help screens.
Modifications to the data element dictionary.
Changes to data conversion plans.
Training (both procedural and on-line) for users.
Construction of the test data base(s) and the general testing approach.
Unit (module) testing, including testing scripts, and sign-off (if any) on tests by users and IST operational personnel.
Documentation of deficiencies and of enhancements not covered in the detail design phase, and a methodology for deciding if system changes will be made for these before or after the initial system implementation date.

4.3. The new system should comply with all applicable IST standards for such things as data element names and formats, user interfaces, user (logon) names, and security. To the extent it does not, the primary department should either document (justify, in writing) the reasons for non-compliance or should request IST to change the standards.

4.4. For a new system designed by IST, the system will not be implemented (go into production, replacing an old system[s] or manual process[es]) until the primary department has completed user acceptance testing and considers the system to be sufficiently ready. IST should provide the primary department with a list of all known deficiencies before recommending that the system go into production.

4.5. Premature implementation of a system can be identified by the variety and seriousness of problems that occur after the implementation date. To avoid these problems, it is recommended that:

The implementation date not be firmly established until the start of this phase, if a firm date is required to help affected departments plan for and implement the system. (The establishment of a range of expected development time can be used to measure, as necessary, the performance of those developing the system.)
If and when a firm date is set, an alternate date and implementation plan be formally established to provide a viable option to premature implementation.

5. Post-Implementation Review and Guidelines Updating

5.1. For the purposes of these guidelines, the post-implementation review is intended to identify areas of possible improvement in the way that application systems are developed, not to identify problems with the application that is now operating.

5.2. The post-implementation review should be conducted by the primary department and IST, jointly, and should start somewhere between three and six months after the new system is implemented.

5.3. After the post-implementation review, the primary department, IST, or both, may make recommendations

to the Associate Vice Chancellor—IST, for changes to IST policies and procedures.
through the Associate Vice Chancellor—IST to ASSCS, for changes to these guidelines.

5.4. IST will maintain and update these guidelines.