Organizations need records to carry out their business activities and to document actions and decisions. Today, most organizations increasingly manage work and make decisions on the basis of electronic information. With this shift from paper to digital information, many organizations find that their current electronic records are not sufficient to support their business, evidentiary, or historical needs.
Without question, organizations need electronic records that are reliable and authentic, usable for multiple purposes, and accessible over time for both business and secondary uses. Unfortunately, traditional system design methodologies do not give adequate attention to the creation, integration, management, and preservation of electronic records.
Agencies are seeking technological solutions and tools that will help them integrate records management requirements into their application development plans rather than addressing records management as an isolated additional activity.
In recent years, significant theoretical work has been done in the area of electronic records management; however, little has been translated into practical implementable solutions. This project was designed to bridge the gap between theory and practice by producing generalizable tools that link business objectives to sound records management practices. The project integrated and built upon several existing bodies of knowledge: electronic recordkeeping and archival theory and practice, business process improvement and reengineering (BPI/BPR) methodologies, and system development methodologies.
The project was an attempt to develop a practical way to incorporate essential electronic records requirements into the design of new information systems. Funded in large part by a research grant from the National Historical Publications and Records Commission (NHPRC), the project was conducted from 1996 to 1998 through a partnership between the New York State Archives and Records Administration and the Center for Technology in Government. The project team included staff of the NYS Adirondack Park Agency, eight corporate partners led by Intergraph Corporation, and University at Albany faculty and graduate students.
Traditional system design methodologies do not give adequate attention to the creation, integration, management, and preservation of electronic records. The following records management issues demonstrate this problem:
This project integrates practical and theoretical work in electronic records management with network-based system development methodologies and business process improvement practices. The goal of the project is to develop practical tools that include an expanded business process improvement component that includes steps to directly address electronic records management requirements in systems development efforts. The tools will guide the identification of technology, policy, and management factors to ensure that records are created, maintained, and stored in a manner that facilitates access for agency operations, secondary uses, evidentiary needs, and archival purposes.
The tool will be generalizable to public and private sector organizations seeking to improve records management in network-based information systems.
Many organizations are seeking to improve or streamline their business processes through business process reengineering or business process improvement programs. Rather than automating existing inefficient or unnecessary processes, organizations are identifying opportunities to recreate or reinvent their business processes. In some cases, organizations have found that substantial gains can be realized from business process improvement without the implementation of new technologies. Since many agency processes or procedures have been developed piecemeal over time, many activities or sequences of activities add no value to the overall transaction. Identifying these activities and eliminating or modifying them can lead to decreases in customer turnaround time, increased agency productivity, and reduced transaction costs as well as overall improvements in the quality of customer service. In order to improve business processes and simultaneously ensure that an organization's records management needs are addressed, the definition of 'processes' must be broadened to include those processes or activities that ensure records management and archival requirements will be met. Of course, not all of these activities can be supported by use of technology. Policies must be developed and management strategies must be devised and implemented to ensure that these requirements are appropriately and continuously addressed.
Based upon this investigation, the project will develop a system design model that incorporates the management of electronic records into the functionality of an automated system. The model will be tested, evaluated, and modified in the context of a sample public sector application: the permit issuance process of the NYS Adirondack Park Agency.
Our project approach therefore includes the following activities:
This two-year project is funded in part by a $140,000 grant (No. 96-023) from the National Historical Publications and Records Commission (NHPRC), the funding arm of the National Archives.
A prototype system to support APA's minor project review process was based on information gathered from the interviews, business process improvement activities, and the use of the RRAIT. It was designed to help APA staff determine which features and functionality of a fully implemented system would best support the Agency's productivity and customer service goals. The prototype also served as a mechanism for identifying management and policy strategies that would need to be developed to complement the system implementation. The following technology components were used in the development of the prototype system:
The prototype is a network-based integrated document management and workflow system capable of supporting a fully electronic record of the minor project review process. It is also capable of accessing, analyzing, and capturing information from the Agency's geographic information system (GIS) and archiving the project record. The prototype functionality includes:
A high-level diagram of the improved project review process that resulted from the BPI work and the use of the RREC is shown in Figure 7. It represents a number of changes from the current process, some enabled by technology and others representing changes to the process itself or changes in the manner or order in which the sub-tasks of the process are completed. An example of a technology-enabled change is the parallel processing made possible by simultaneous access to digital project records. As can be seen from the figure, projects often require consultations from natural resource staff, such as soil scientists or wetlands biologists, as well as legal consultation. Under the current paper process, the review and analysis must be conducted sequentially as there is only one copy of the project file. This technology-enabled change would allow these reviews to take place concurrently. Technology could also improve project-related correspondence. For example, standard language for permits and additional information requests could be automatically inserted in these documents as they are prepared for specific project applications, thus saving staff time and assuring consistency across projects. Other types of correspondence, currently generated on paper or in electronic form, are passed among staff for review in either hardcopy or by an exchange of disks. A networked system would improve performance by eliminating the use of 'sneaker net' in the sharing and review of Agency correspondence.
The use of the Business Process Level of the RREC led to the identification of the records management requirements within the project review process at the sub-task level. During the initial business process improvement activity, five sub-tasks were identified:
Expanded View of Flowchart
The Acceptance of Application sub-task includes initial receipt and cursory review of an application, assignment of staff including the Project Review Officer (PRO) and any required natural resource or legal staff, and forwarding the electronic application folder to the appropriate staff. A diversity of documentation is integrated into the project file during this sub-task such as the application, a site plan map, deeds, tax maps, and information about prior Agency actions on the project property. During the initial review, paper maps or digital spatial data is also accessed in-house and integrated into the record. An electronic system must therefore accommodate or reference the location of the project documents and other information sources used to support this sub-task. In addition, the receipt of an application starts a 15-day statutory clock. Information about the receipt date and the number of elapsed days since receipt is a critical activity required by state law. A number of authenticity requirements are important. For example, the original signature on the permit application is required by law, the date stamp on the application is required by regulation. Therefore, an electronic system must maintain proofs of authenticity. At this initial point in the process, only two individuals, the Director of Regulatory Programs (DRP), and the Secretary, are authorized by Agency practice to change the project record.
Under the revised technology-supported process, the Completeness Review sub-task begins when an assigned PRO receives the project file. The file is simultaneously available to legal and natural resource staff. The level of involvement of these staff may vary from simple notification that the project has been started to specific issues that need to be addressed in the project review. Information related to the 37 statutory development considerations must be accessed during this sub-task and items such as additional paper maps, GIS data, deeds, narratives of map analyses, property history notes, and engineering reports are integrated into the record. Site visits are conducted during this sub-task and therefore site visit notes, soil analysis results, visual analysis reports, and narrative about the potential impacts on other affected landowners is integrated into the record during the completeness review. If an Additional Information Request (AIR) is issued, this document is also integrated into the record. Under the current system the DRP, PRO, and Secretary are authorized to make changes during this phase. Under the modified process, legal and natural resource staff would also have authorization to add comments or documentation to the record or documents within it. Since this sub-task must be completed within 15 days of the receipt of the application, the timeclock must be updated. If the application is deemed incomplete, and an AIR is issued, the statutory timeclock is stopped until such time as the additional required information is received from the applicant. The AIR has authenticity requirements such as original signatures and an executed Notice of Complete Application once a project application has been deemed complete. It is also Agency practice to maintain a copy of the certified mail receipt from the AIR mailing.
During the Project Review sub-task, the components added to a record include memos reporting consultations with staff from APA and other agencies such as the NYS Department of Environmental Conservation (DEC), notes from meetings with the applicant, confidentiality requests, and determinations of trade secret status. The project review process must be completed within 45 days of the date that the application was deemed complete for minor projects. Therefore, timeclock information must be maintained. Under the current process only the PRO and the Secretary are authorized to change the record during this sub-task, but in the improved process legal and natural resource staff would also be able to modify certain elements of the record during their concurrent review. No authentication requirements were identified for this sub-task.
The Permit Approval sub-task results in either the issuance of a permit or a referral to the Agency Board for a public hearing. In the case where the permit is approved by APA staff, the record will include drafts of the permit, results of public comment, comments on permit drafts, a copy of the approved plan, the final issued permit, a reference to any oversized map (those that are too large to be included in a paper project file), and a transmittal letter to the applicant. In addition, the permit must be filed with the County Clerk's office. Once this is done, a stamped card is received by the Agency from the County Clerk and is also integrated into the project record. Proofs of authenticity include an original signature and notarization on the permit, the stamped card from the County Clerk's office, and a certified mail receipt. The issuance of a permit stops the regulatory review clock for the project and information about the end date must be included in the record. During this sub-task, the PRO, DRP, Executive Director, and Secretary are authorized to change the record.
If APA staff do not approve the project, a request for public hearing before the Agency Board is drafted and included in the project file. Additional memos from consultations may be added to the project file. The Board will issue either a denial order or a permit, and one or the other is added to the project record. If the project does go to public hearing, Agency Board minutes will also be included in the project record. Authentication requirements include an original signature on either the permit or the denial order. If a permit is issued, the other documentation noted above is also included in the record.
The modified process reflects a new sub-task in the process for Archiving a completed project record. This step involves the purging of documents in a project record that are not required for long-term access. Ideally, the project record would be reduced to the level of a minimum record in terms of legal and evidentiary requirements and secondary uses. Modifications to
the record during this sub-task are made only by the archivist or the DRP.
The following section discusses the components of the prototype system and ties these components to the records management requirements described above.
The prototype supports the scanning of all documents that are submitted to the Agency in application for a permit. The imaging component of the prototype supports the records capture requirements associated with the project review process, while the document management functionality supports many of the records access requirements. Additionally, document imaging was used to capture proofs of authentication. Scanned images of original signatures and notarization were deemed to be sufficient for legal admissibility of the record.
For each document, the prototype provides a data entry screen to capture information about the document itself. For example, if a map is submitted to the Agency with an application, the data entry screen for a map document allows for the entry of such information as the type of map (e.g. survey map, sketch, wetlands map), as well as the map scale, the map creator, and the date the map was created. As documents are scanned into the system, they are attached or related to the appropriate project record. Figure 8 shows the types of documents that can be accommodated by the prototype system. The bold-face type on some of the document types indicates that there is a document of that type in the project record. This functionality captures information about the components of a record as specified earlier by use of the Record Level of the RREC.
Expanded View of Data Entry Screen
The data entry screens were designed using Visual Basic. They are structured in a consistent format and operate in a manner similar to Windows applications. For those elements or attributes with pre-defined values such as town, county, project type, and map type, drop-down menus are provided to decrease data entry time and increase accuracy. Within each data entry screen, key information elements are defined as required, and the completion of the data entry for that screen requires that values for these fields be provided.
Expanded View of Workflow Diagram
The workflow component allows for the assignment of different types of staff to a project. For example, when creating a new project, the system allows for the assignment of a PRO and natural resource and legal staff. The system also allows for the assignment of individuals to receive notification about a given project. This feature allows individuals who have no pre-defined responsibility or assignments within a project to be kept posted on project progress. The workflow component also routes a project record through the process after an individual has completed a step or sub-task within the workflow. As people sign off on a task, they are allowed to provide comments or notes that can be read by the next individual in the process. The project record moves to the next step in the process based upon the value selected upon sign-off. The diagrammatic representation of the workflow for the minor project permit process is shown in Figure 9. The workflow diagram can be used to identify where in the process, a given project is at any point in time. The workflow software also provides a narrative list of the steps that have been conducted including the start and finish time for the step and the individual who conducted it. The workflow functionality can also be used to address records management requirements associated with authorizations for modifications to a record. While not fully implemented in the prototype, the software has the capability to allow deletions, additions, or modifications to a project record or to individual record components based on where the project is in the overall review process or other conditions.
The forms generation feature serves a number of purposes. First, it decreases staff time in repeatedly typing the same information across documents related to a specific project. Second, it minimizes the potential for error by drawing upon attribute values already in the database. Third, the use of standard clauses increases the consistency of permits and AIRs (assuming the Agency staff reach consensus about standard language for use in the system).
The prototype is designed so that relevant values entered into the system are automatically placed in the template for the document type under development. Additionally, both permits and AIRs contain some set of standard language. The system provides a pick list or menu of these standard clauses that can automatically be added to a document under development. Once this language is added in the data entry screen, it can be modified if necessary, either within the data entry screen or within the document itself. Figure 10 shows an example of a document creation screen for an AIR. This screen shows the menu choices for items requested in an AIR and also provides data entry points for other information that will be placed into the document.
As indicated above, any information needed for the document that is already in the database, will automatically be called up and placed in the appropriate place in the document. Following the creation of the document, the user has two choices. One is to print the document to a file which will create an actual electronic file of the document that can then be attached to the project record. The second choice is the print option which sends the document to a printer so that it can be mailed to an applicant. In those cases where the document has to be signed or notarized by APA staff per legal or regulatory requirements, the signed and notarized original can be scanned into the system and attached to the project record.
Expanded View of Data Entry Screen
Perhaps one of the most useful features of the system in terms of improving access to records at APA, is the search and retrieval capability. This feature allows the user to search at either the document level or the project level based on any attributes contained in the database. The interface supports simple or complex queries that can be developed easily using pull down menus. In much the same way that the data entry screens were developed, the search screens allow for the selection of attributes on which one can search. For those attributes with pre-defined values, once the attribute is selected, the available values are presented for selection in the search. Once a search is developed and submitted to the system, the records or documents that meet the search criteria are listed in the bottom of the search screen. If the search was conducted at the record level, the search results will show all of the records that meet the search criteria. By double-clicking on a given record, all of the document types contained within that record will be displayed. By double-clicking on a document type, all of the actual documents or files of that document type will be displayed. Double-clicking on a specific document or file invokes the Intergraph Redline tool which allows for the viewing of a particular document. Figure 11 shows the document search and retrieval screen.
As indicated above, the prototype supports the viewing of a specific document or file within a project record by using a product that allows for the viewing of 70 different types of file formats. This is a particularly useful tool from a records management and archival perspective because it allows a user to view documents created in a multitude of file formats across a range of software packages without having to maintain the native software in which each was created. This has advantages for both current use and future migration. The Redline product also supports non-destructive mark-ups to documents or files. It allows users to create a layer that is associated with a given file without changing the file or document itself. For example, a user may create a layer with electronic 'post-it notes' or arrows or circles or other annotations that can be viewed by other users but that are maintained as a file or layer distinct from the original document. This feature enhances communication about documents or files among users while maintaining the document in its original form. These layers can be viewed, added, modified, or deleted separately from the original document or file. This feature is particularly useful at APA where the project review staff may want to draw attention to a specific element on a map while communicating with natural resource staff, for example.
Expanded View of Data Entry Screen
The prototype system also supports access to the Agency's geographic information system. Information contained in the Agency's GIS must be accessed in the project review process. Using Intergraph's GeoMedia product, users can access digital spatial data independent of the software with which it was created. This tool allows for the overlay of multiple map layers and automatically adjusts scales and projections. Following access to and analysis of map layers, the system will allow for the capture of the screen or analysis results into a project record and information about the resulting document or screen capture can be input using the data entry screen for map type documents.
As shown in Figure 7 the last step in the minor project review process is the archival function. This step is conducted at a pre-set interval after the project has been completed. At this point in the process, the individual responsible for archiving project records may remove any unnecessary documentation from a project record and move it into an archival vault. Once a project record is inside the vault, it cannot be changed. This feature ensures that records are not modified after a transaction is complete.
The prototype system was delivered to APA in March 1998, and was accompanied by presentations of the prototype functionality to the full staff and two levels of user training. One level focused on the functionality of the prototype within the minor project workflow and was provided to those staff who work directly on the project review process. The second type of training demonstrated the use of the prototype for accessing project records for secondary use in the Agency and focused on the prototype's search and retrieval capabilities. A second training review session was held in June to train staff who were not available during the prior training sessions.
Several of the Project Review Officers participated in the testing of the prototype along with one of the Directors of Regulatory Programs, support staff, and representatives from both legal and natural resources. All evaluation participants were asked to use the prototype and think about its features and functionality in terms of improvements to the way they do their own work. They were asked to envision how the system would help specifically within the project review process and more generally about how access to the records and information in the system would support other processes at the Agency.
During the first training session, several project applications that had just been received by the Agency were input into the system. All of the project documents such as the application, maps, and deeds were scanned and the information in and about the documents was input into the system. Staff were assigned to work on each of the projects so that a test could be conducted of routing a project through the system. The individuals participating in the training were asked to run several projects through the system while maintaining parallel paper processes.
At the time of this report the Agency is continuing to test the prototype. Preliminary feedback collected during the project indicated that the prototype successfully demonstrated the potential value of workflow and document management technologies to meet Agency goals. It also generated significant interest in the potential value of developing standardized permit conditions and AIRs, and in establishing a definition of a legal minimum record for the Agency. Testing of the prototype also served to identify to the Agency staff the necessary management and policy changes that would be required within the Agency to complement a full system implementation, especially the need for changes in the way individuals conduct their work within an automated workflow. They were also able to assess the relative merits of managing workflow electronically versus managing it through the adoption of standard policies and practices. The prototype also served to bring the issues related to effective records management to the attention of the Agency Board.
Business processes provide a common focus for records managers, archivists, technologists, and business managers. A business process perspective ties discussions of records management issues to work that is critical to an organization. By linking records management issues to business processes, the tools provide a common language for improved communication between records management professionals and other practitioners. Program managers indicated that this manner of presentation enabled them to understand the importance of records management requirements in terms of the issues that are critical to them in conducting their work. For technologists, the tools could be seamlessly integrated into the business process improvement phase of system design and generated requirements that led to well-defined system features and data requirements.
Comprehensive records management requirements directly support business objectives. The tools prompt participants to identify a comprehensive set of records management requirements associated with a business process. The Business Process Level of the RREC helps identify the specific record components that must be captured at each step during the course of a transaction. It also ties each component to specific legal or professional standards or organizational practices. The Record Level addresses the need for access to records over time. The RRIC can then be used to identify technology and other mechanisms to ensure that that records are appropriately captured and that they remain accessible for both current and future use. Moreover, the tools are capable of identifying all authenticity requirements tied to the business process and they help identify the diversity of forms and formats that a system must be able to accommodate in order to assemble a complete record. These requirements are not limited to 'recordkeeping' needs; they are integral to the business process itself.
Current and future access needs can be specified and accommodated in system design. The tools have the ability to deal with both internal and external primary and secondary access to records. They also call attention to long-term access issues such as migration strategies and meta data that are best addressed at the initial system design stage. The Business Process and Record Levels of the RREC support the identification of access needs from the perspective of internal users during a business transaction as well as internal and external access needs after the transaction has been completed. The questions are designed to identify the components of a record required by each of these user types as well as their preferred or required access methods.
In system design, focus first on business needs and records that support them; then focus on technology. In general, the use of the tools shifts the focus of system design and development away from technology and toward the capture, maintenance, and ongoing use of business records. The tools embed the importance of the record into the system development process from the perspective of both users and system developers. Records management requirements based on business process analysis are directly translated into user and system requirements. The responses to the questions in the Business Process and System Levels of the RREC are easily communicated to system developers in terms of technical specifications. In addition, the questions that focus on the documents that comprise a record and on internal and external access to records are readily translated into data model specifications.
Focus on system functionality before choosing specific technologies. The tools help organizations identify the functionality that is required in a system to support records management requirements, and emphasize technology solutions that maximize inter-operability and adherence to standards. They do not address the actual selection of hardware and software to provide the necessary functionality. This selection must be based on many factors such as existing infrastructure (both technical and organizational), cost, and expected benefits. We strongly recommend that technology awareness activities be conducted in conjunction with the use of the tools. Product reviews, vendor presentations, and conferences focused on technology applications are all ways to increase awareness of technology capabilities and limitations among the staff who will work with the new system. These kinds of activities increase understanding of the strengths and weaknesses of various technology choices.
Supporting policies and management practices are essential, but challenging, components. The RRIC, with its focus on implementation, highlights the importance of policy and management strategies -- critical elements that often receive little or no attention in system development efforts. The tools facilitate the identification of related management and policy strategies, such as the range of user permissions and definition of a minimum legal record. Policies and practices ensure the entire organization is working in concert with the records management requirements that are built into the electronic system. In most organizations, these issues present the most difficulty because their content and execution depend on organizational consensus about the way work should be done.
All record users need to participate in the identification of requirements. One of the most critical factors for effective use of the tools is getting the right people to answer the questions. All primary and secondary users of the records that will be created and maintained by an information system should be represented in the elicitation of the records requirements. Other players who may not be direct records users, such as legal staff and executives, need to be involved in the development of management and policy strategies that will support users. Not every group needs to be involved in the entire process, but each needs to participate actively at the appropriate points so that all user needs are identified and incorporated into the system design.
The records requirements tools can be used in a variety of ways. The tools provide a sound framework for the identification of records management requirements that can be modified to suit the setting in which they are used. While we strongly recommend that the Business Process Level of the RREC be used in conjunction with business process analysis or improvement activities, the questions in the other sections can be posed using a variety of methods such as surveys and interviews. The manner in which the questions are asked and answered can be tailored for use across different organizational contexts. They should be selected for their compatibility with the organization's skills and time schedules, and their ability to minimize the total cost of the information collection process.
Awareness and willingness to change are preconditions for success. Perhaps the biggest weakness of the tools is the pre-condition for their use. That is, an organization must first recognize the importance of its business records and the costs and risks associated with ignoring them. Without this foundation, it is unlikely that an organization will invest the time and attention to detail that the tools demand. While the tools support the comprehensive identification of records management requirements and mechanisms for addressing them, the degree to which they are implemented depends on the organization's readiness and willingness to change. Change means more than new information systems; it requires supporting management and policy strategies as well as an understanding of the degree to which the requirements can be addressed by the chosen technologies. In sum, while the tools support the identification of requirements, the factors that surround their implementation determine the ultimate level of success.
This two-year project was funded in part by a $140,000 grant (No. 96-023) from the National Historical Publications and Records Commission (NHPRC), the funding arm of the National Archives.