Comsys-TIM System Metrics

Overview

Managing change within diverse and complex information systems, implemented under an array of styles and designs, continues to challenge IS management. Systems redevelopment has emerged to support software maintenance, migration and replacement initiatives. Planning and implementing redevelopment projects requires complete, objective data, or metrics, about systems. Success of any assessment effort is based upon the application metrics defined herein.

Comsys-TIM defines an extended set of application metrics to allow organizations to objectively quantify application quality and requirements conformity. Some of the metrics can be collected automatically. Others can not. Tools assisting with the collection process are referenced in the task tool guidelines. The processes used to gather these metrics are defined within the Comsys-TIM task guidelines.

Comsys-TIM metrics are grouped based on the assessment categories defined in Comsys-TIM. Two terms used throughout are important. A "count" is a raw number indicating how many occurrences of a particular object exist within a system or within an application area. A "defect count" is a special high risk type of count.

A "score" is a derived metric that involves a calculation using one or more counts. A score can also result from combining two or more scores or from a subjective analysis of a current situation. Scores defined in one category may utilize counts or scores collected in a prior category.

Comsys-TIM provides a consistent, universal scoring scheme. Analysis is measured on a one to five rating scale. The Comsys-TIM rating process, built on many years of project experience and research, allows organizations to compare application against industry averages. Summary metric scores correspond to the following qualitative measurements:

5 - Organization scores well above the industry average

4 - An above average score

3 - An average score

2 - A below average score

1 - A score well below the industry average

Universally, "5" is always a good score, and "1" is always an undesirable score. This scoring scheme is used for all tasks, activities and stages within Comsys-TIM. For example, if a program is rated as a "5" for Complexity, this is an indication that the program logic is not complex, and would be much easier to modify and maintain than a program that was rated as a "1" in Complexity.

Each metric is assigned a unique identifier so that spreadsheets, process and project management tools, and other automated analyses can take advantage of these numbers. Gaps in the numbering scheme allow for the addition of user defined metrics. Finally, many of these metrics are used in the estimating guidelines defined for each Comsys-TIM step and are a critical factor in determining level of effort and resource requirements for redevelopment projects.

Global Assessment Metrics - GAM

Global assessment metrics represent high level information about the current system(s). The information contained in these metrics should be known even at the inception of a project, before any automated analysis has been done. This information is usually present in the form of subject matter expertise of senior analysts and others in the applications areas, or may be quickly calculated by the current system SMEs. Of course, as a project progresses, these metrics should be updated if more accurate information becomes available.

These metrics shall be used extensively to calculate work estimates for technical assessment tasks as well as other tasks throughout Comsys-TIM.

GAM 001 Enterprise scope multiplier

Assuming a default value of 1 for an enterprise of 10-12 major system areas, increase the value of this multiplier by 1 for each additional 10-12 major system areas which exist. This metric is needed when creating estimates for some enterprise tasks, which may vary considerably based on the size of the organization.

GAM 002 Total number of systems selected for analysis in this project

The number of systems selected for analysis will vary with the type of project. A code stabilization, redocumentation or language conversion project may involve only a single system, while redundant systems consolidation or application integration projects will typically involve two or more systems.

GAM 003 Total number of programs in the systems selected for analysis

A count of the total number of source modules potentially involved in this project (if multiple systems, add the total number from each system).

GAM 004 Number of batch systems to be analyzed

A count of the total number of batch systems or sub-systems involved in this project.

GAM 005 Number of on-line systems to be analyzed

A count of the total number of on-line systems or sub-systems involved in this project.

GAM 006 Estimated average number of programs per system

This metric will reflect an average number of programs per system. If the exact totals per system are unknown, they should be estimated. If the project involves only one system, this number will equal GAM 003.

GAM 007 Number of redevelopment hypotheses being assessed

This metric will reflect the number of potential scenarios being considered to solve the problem at hand. For instance, an organization with a system that is highly difficult and costly to maintain might consider three alternatives for solving the problem: buying a package to replace the code, stabilizing and improving the code, or rewriting the system. In this case, GAM 007 would be 3.

GAM 008 Number of sets of staging libraries needed for the project

This number will differ based on the redevelopment standards of each organization. Some organizations, for instance, will create an extra set of libraries in order to keep revised applications running parallel to the original systems for a time. Other organizations may need to segment projects based on timing considerations, while still others may combine redevelopment scenarios and thus require extra sets.

GAM 009 Percentage of programs for which there exists a process flow metric tool

This figure will often be 0% or 100%, depending on the availability of a tool which can process the language in the system at hand. If the system contains programs of more than one language, this metric may contain some other value if the tool(s) available do not support all the languages being used.

Enterprise Redevelopment Planning Metrics - ERP

Enterprise redevelopment planning (ERP) metrics represent information about systems at the enterprise level. ERP metrics are based on intelligent estimates from systems and application experts, and are expected to have "rolling accuracy." That is, as time progresses and more information is gathered, they can be continually revised to achieve more accuracy.

The information contained in these metrics should be gathered during an enterprise assessment project, and will help the project staff to estimate resources, time and cost for running an enterprise wide redevelopment project.

ERP 001 Total number of upgrade units

An upgrade unit is comprised of the programs, data stores, and related system objects required to complete a redevelopment project for a segment of an enterprise. For efficiency’s sake, organizations try to include related application elements in a single upgrade unit to minimize interface considerations between upgrade units.

ERP 002 Total number of systems

The total number of application systems in the enterprise.

ERP 003 Total number of hardware types

A count of the unique types of hardware used in the enterprise. Each hardware type should be counted just once.

ERP 004 Total number of operating system types

A count of the unique types of operating system used in the enterprise. Each operating system type should be counted just once.

ERP 005 Total number of language types

A count of the unique types of languages in used in the enterprise. Each language type should be counted just once.

ERP 006 Total number of DBMS/file types

A count of the unique types of databases and files used in the enterprise. Each database/file type should be counted just once.

ERP 007 Total number of TP monitor types

A count of the unique types of teleprocessing monitors in use in the enterprise. Each TP monitor type should be counted just once.

ERP 008 Total number of development environment types

A count of the unique types of development environments in use in the enterprise. Each development environment type should be counted just once.

ERP 009 Total number of executable modules

A count of the total number of executable modules in the enterprise.

ERP 010 Total number of source programs

A count of the total number of source programs in the enterprise.

ERP 011 Total number lines of code

A count of the total number of lines of code in all applications in the enterprise. Lines of code here means fully expanded lines of code, including copy books, includes, comments, and any other non-blank lines.

ERP 012 Total number of Year 2000 non-date compliant systems

A count of the total number of systems in the enterprise which are not date compliant for Year 2000. Any system which is procedurally compliant (has workaround code) or data compliant (has expanded data fields) should not be included in this number.

ERP 013 Total number of shared data stores in the enterprise

A count of the total number of data bases or on-line files shared between programs and applications across the enterprise. This metric is calculated by adding all figures for shared data stores per system (ERP 021). In demonstrating the amount of data sharing that is taking place in real time, this metric provides the user with the number of potential API candidates in the enterprise.

ERP 014 Total number of external data files in the enterprise

A count of the total number of data files input to and output from batch programs and systems across the enterprise. This metric is calculated by adding all figures for external data files per system (ERP 022). In demonstrating the number of input and output files from every batch program, this metric provides the user with the number of potential bridge candidates in the enterprise.

ERP 021 Total number of shared data stores per system

A count of the total number of data bases or on-line files shared between programs and applications in a system. In demonstrating the amount of data sharing that is taking place in real time, this metric provides the user with the number of potential API candidates in the system.

ERP 022 Total number of external data files per system

A count of the total number of data files input to and output from batch programs and systems in a system. In demonstrating the number of input and output files from every batch program, this metric provides the user with the number of potential bridge candidates in the system.

Technical Assessment Metrics

Technical assessment metrics include environmental metrics, data definition metrics and procedural metrics. They are required to determine the overall quality of an application system. The information required to create these metrics is, to a great degree, collected automatically.

Environmental Metrics - TEM

Data must be gathered to identify the size and complexity of an application at a macro level. This is based on the number of application objects and the interrelationships of those objects. These baseline counts are required to identify a system and its boundaries. Difficulty in collecting this environmental data signals configuration control problems.

The metrics below identify counts, defect counts and scores for an IBM MVS / ESA environment. Similar metrics may be developed for other environments. The numbering scheme allows for new object types to be defined as required for unique or non-standard application object types.

This category should include all live production data stores with each category representing a simple physical count.

TEM 001 Total number sequential files

Count of the sequential files used in this environment. Count each sequential file only once.

TEM 002 Total number ISAM files

Count of the ISAM files used in this environment. Count each ISAM physical file only once.

TEM 003 Total number VSAM files

Count of the VSAM files used in this environment. Count each VSAM physical file only once.

TEM 004 Total number IMS data bases

Count of the IMS data bases used in this environment. Count each IMS data base only once.

TEM 005 Total number IDMS data bases

Count of the IDMS data bases used in this environment. Count each IDMS data base only once.

TEM 006 Total number DB/2 data bases

Count of the DB/2 data bases used in this environment. Count each DB/2 data base only once.

TEM 007 Total number BDAM files

Count of the BDAM files used in this environment. Count each BDAM physical file only once.

TEM 008 Total number other data base types - specify

Count of the types of other data bases used in this environment. Count each data base type only once.

TEM 009 Total number other file types - specify

Count of the types of other files used in this environment. Count each file type only once.

TEM 010 Total number of data store types

Count of the types of data stores used in this environment. Count each data store type only once.

This category should include all production files with each category representing a simple physical count.

TEM 011 Total number of DBDs

Count of the data base definitions used in this environment. Count each physical data base definition only once.

TEM 012 Total number of PSBs

Count of the PSB’s used in this environment. Count each program specification block only once.

TEM 013 Total number of DB/2 tables

Count of the DB/2 tables used in this environment. Count each DB/2 table only once.

TEM 014 Total number of IDMS schema

Count of the IDMS schemas used in this environment. Count each IDMS schema only once.

TEM 015 Total number of other data base definitions - specify

Count of the other data base definitions used in this environment. Count each physical data base definition only once. Specify the data base definitions.

All production executable modules should be included with each category representing a simple physical count.

TEM 021 Total executable batch load modules

Count of the batch load modules in production.

TEM 022 Total executable CICS load modules

Count of the CICS load modules in production.

TEM 023 Total executable IMS DC load modules

Count of the IMS DC load modules in production.

TEM 024 Total executable IDMS DC load modules

Count of the IDMS DC load modules in production.

TEM 025 Total executable other types load modules - specify

Count of the other types of load modules in production. Specify load types.

TEM 026 Total executable on-line load modules

Count of all the on-line load modules in production. In certain situations, this may be an estimate rather than an actual count.

Requires analyzing all live production execution code with each category representing a simple physical count.

TEM 031 Total number of job streams

Count of the job control language items which run in production. A job stream contains one to several job steps, and specifies the load modules (or executables) and the data stores to be used. In an IBM environment, this would be a JCL or DCL job stream, in a Unisys environment, it would be an ECL job stream, etc. In certain situations, this may be an estimate rather than an actual count.

TEM 032 Total lines JCL

Count of the lines of control language instructions contained in all job streams in the systems to be analyzed.

TEM 033 Total executable job steps

A job step accomplishes a single operation within a job stream. This may include sorting files, executing load modules, printing reports, etc.

TEM 034 Total number of PROC definitions

In an IBM environment, a PROC is a formalized protocol of job control steps, which may be invoked by a job stream or by another PROC.

TEM 035 Total number of PROC references

Count of the number of times a PROC is used in the system.

Home grown TP monitors should be included in this category with each category representing a simple physical count.

TEM 041 Total number BMS objects

Count of the total number of screen layouts defined for CICS systems.

TEM 042 Total number BMS lines source

Count of the total number of lines of code used to create all BMS screen layouts.

TEM 043 Total number MFS objects

Count of the total number of screen layouts defined for IMS DC systems.

TEM 044 Total number MFS lines source

Count of the total number of lines of code used to create all IMS DC screen layouts.

TEM 045 Total number other on-line objects

Count of all other on-line screen layout definition objects.

TEM 046 Total number other on-line lines source

Count of the total number of lines of code used to create all other screen layouts. Specify other screen layout types.

Includes all production source modules with each category representing a simple physical count.

TEM 051 Total Lines COBOL

The total number of lines of executable COBOL instructions. (If automated analysis tool available, use the count provided. If counting manually, include comments but not blank lines.)

TEM 052 Total COBOL source programs

Count of the total number of COBOL source modules.

TEM 053 Total number COBOL copy members

Copy or include members may include data descriptions or procedural code, which is stored separately, but pulled into the program at compile and/or execution time. This metric counts the total number of all types of COBOL copy or include members.

TEM 054 Total number COBOL copy lines

The total number of executable lines of code. (If automated analysis tool available, use the count provided. If counting manually, include comments but not blank lines.)

TEM 055 Total number COBOL copy references

Count of the total number of times COBOL copy members are referred to by all relevant COBOL source modules.

TEM 056 Total lines Assembler

The total number of lines of executable Assembler instructions. (If automated analysis tool available, use the count provided. If counting manually, include comments but not blank lines.)

TEM 057 Total Assembler source programs

Count of the total number of Assembler source modules.

TEM 058 Total number Assembler copy members

Copy or include members may include data descriptions or procedural code, which is stored separately, but pulled into the program at compile and/or execution time. This metric counts the total number of all types of Assembler copy or include members.

TEM 059 Total number Assembler copy lines

The total number of executable lines of code. (If automated analysis tool available, use the count provided. If counting manually, include comments but not blank lines.)

TEM 061 Total number Assembler copy references

Count of the total number of times Assembler copy members are referred to by all relevant Assembler source modules.

TEM 062 Total lines PL/1

The total number of lines of executable PL/1 instructions. (If automated analysis tool available, use the count provided. If counting manually, include comments but not blank lines.)

TEM 063 Total PL/1 source programs

Count of the total number of PL/1 source modules.

TEM 064 Total number PL/1 copy members

Copy or include members may include data descriptions or procedural code, which is stored separately, but pulled into the program at compile and/or execution time. This metric counts the total number of all types of PL/1 copy or include members.

TEM 065 Total number PL/1 copy lines

The total number of executable lines of code. (If automated analysis tool available, use the count provided. If counting manually, include comments but not blank lines.)

TEM 066 Total number PL/1 copy references

Count of the total number of times PL/1 copy members are referred to by all relevant PL/1 source modules.

TEM 067 Total lines 4th generation language

The total number of lines of executable 4GL instructions. (If automated analysis tool available, use the count provided. If counting manually, include comments but not blank lines.)

TEM 068 Total 4th generation language source programs

Count of the total number of 4GL source modules.

TEM 069 Total number 4GL copy members

Copy or include members may include data descriptions or procedural code, which is stored separately, but pulled into the program at compile and/or execution time. This metric counts the total number of all types of 4GL copy or include members.

TEM 071 Total number 4GL copy lines

The total number of executable lines of code. (If automated analysis tool available, use the count provided. If counting manually, include comments but not blank lines.)

TEM 072 Total number 4GL copy references

Count of the total number of times 4GL copy members are referred to by all relevant 4GL source modules.

TEM 073 Other language total lines

The total number of lines of executable other language instructions. (If automated analysis tool available, use the count provided. If counting manually, include comments but not blank lines.)

TEM 074 Other language total source programs

Count of the total number of other language source modules.

TEM 075 Other language total number copy members

Copy or include members may include data descriptions or procedural code, which is stored separately, but pulled into the program at compile and/or execution time. This metric counts the total number of all types of other language copy or include members.

TEM 076 Other language total number copy lines

The total number of executable lines of code. (If automated analysis tool available, use the count provided. If counting manually, include comments but not blank lines.)

TEM 077 Other language total number copy references

Count of the total number of times other language copy members are referred to by all relevant other language source modules.

TEM 079 Total number of languages used

Count of the total number of languages used to create the current source code system(s). Count each language only once, even if more than one release of that language is being used.

TEM 080 Total number of programs to upgrade

Count of the total number of source programs which must be modified for a language level or run-time upgrade.

These metrics represent simple physical counts.

TEM 082 Total number Called entry points

Count of the total number of entry points in the system(s). This includes all locations at which subroutines may be initiated. In an IBM environment, these include Procedure Division Using clauses and CSECTS.

TEM 083 Total number Call verb references

Count of the total times subroutines are called in the system(s). In other words, the total number of times the Call (or equivalent verb, depending on the language) verb is used in all programs.

TEM 084 Total number unique Called subroutines

Count each Called subroutine in the system(s) just once.

Requires cross reference analysis to determine missing number of physical objects. Often these objects are not actually missing in production, but have just not been properly located for the analysis process.

TEM 101 Number of missing source programs

Count of the total number of source programs missing from the source libraries. These programs are identified as part of production system but source cannot be located. Missing source usually identified using cross reference tools.

TEM 102 Number of missing PROC statements

Count of the total number of PROC statements missing from the source libraries.

TEM 103 Number of missing on-line screen layouts

Count of the total number of on-line screens missing from source libraries. These screen layouts are identified as part of production system but source cannot be located. Missing source usually identified using cross reference tools.

TEM 104 Number of missing copy members

Count of the total number of copy members missing from the copy libraries. These copy members are identified as part of production system but source cannot be located. Missing source usually identified using cross reference tools.

TEM 105 Number of missing files

Count of the total number of files missing from the libraries.

TEM 106 Number of missing data base definitions

Count of the total number of data base definitions missing from the libraries.

TEM 107 Number of unreferenced source programs

Count of the total number of source programs present in the source libraries which are not executed by any of the job control commands for the system(s).

TEM 108 Number of unreferenced copy members

Count of the total number of copy members present in the copy libraries that are not accessed by any of the executable source modules for the system(s).

TEM 109 Number of unreferenced screen definitions

Count of the total number of screen definitions available which are not accessed by any of the executables in the system(s).

TEM 110 Number of unreferenced PROCs

Count of the total number of PROCs which are not executed by any of the job control commands in the system(s).

TEM 111 Number of non-standard or obsolete data stores

Total number of data stores which do not conform to data standards. May be comprised of non-standard data structures, or may contain data only used by obsolete and/or discontinued systems.

TEM 150 Total number physical files or data stores

The sum of data store metrics TEM 001 through TEM 009.

TEM 151 Total number data base objects

The sum of data base object metrics TEM 011 through TEM 015.

TEM 152 Total number executable programs

The sum of executable program metrics TEM 021 and TEM 026.

TEM 153 Total number screen definitions

The sum of screen definition metrics TEM 041, TEM 043 and TEM 045.

TEM 155 Total number copy members

The sum of copy member metrics TEM 053, TEM 058, TEM 064, TEM 069 and TEM 075.

TEM 156 Total number system objects

The sum of summary count metrics TEM 150, TEM 151, TEM 152, TEM 153 and TEM 155.

The following scores are based on a 1 to 5 scaling with 1 being very poor and 5 being excellent. The scaling assignments are calculated somewhat subjectively based on the Inventory / Analysis Technical Assessment determination of the particular situation and an organization's strategic direction.

TEM 201 Non-standard file usage score

Non-standard files are defined by an organization’s accepted technology standards. Assign the following scores where a 1 or 2 is low, a 3 is medium and a 5 is high:

Home grown structures assign 1 or 2

Acceptable but non-strategic structures assign 3 or 4

Ideal strategic structures assign 5

 

TEM 202 Non-standard data base usage score

Non-standard data bases are defined by an organization’s accepted technology standards. Assign the following scores where a 1 or 2 is low, a 3 is medium and a 5 is high:

Home grown structures assign 1 or 2

Acceptable but non-strategic structures assign 3 or 4

Ideal strategic structures assign 5

TEM 203 Non-standard language usage score

Non-standard languages are defined by an organization’s accepted technology standards. Assign the following scores where a 1 or 2 is low, a 3 is medium and a 5 is high:

2GL or unacceptable 4GL assign 1 or 2

Acceptable 3GL or 4GL assign 3 or 4

Ideal strategic language assign 5

TEM 204 Non-standard screen I/O definition score

Non-standard screen I/O definitions are defined by an organization’s accepted technology standards. Assign the following scores where a 1 or 2 is low, a 3 is medium and a 5 is high:

Home grown structures assign 1 or 2

Acceptable but non-strategic structures assign 3 or 4

Ideal strategic structures assign 5

TEM 205 Non-standard teleprocessing monitor usage score

Non-standard teleprocessing monitors are defined by an organization’s accepted technology standards. Assign the following scores where a 1 or 2 is low, a 3 is medium and a 5 is high:

Home grown structures assign 1 or 2

Acceptable but non-strategic structures assign 3 or 4

Ideal strategic structures assign 5

TEM 207 Missing object score

Assign the following scores for percentage of objects missing:

More than 25% assign 1

More than 15% assign 2

More than 10% assign 3

More than 5% assign 4

0 - 4% assign 5

TEM 208 Unreferenced object score

Assign the following scores for percentage of objects unreferenced:

More than 25% assign 1

More than 15% assign 2

More than 10% assign 3

More than 5% assign 4

0 - 4% assign 5

TEM 250 Environmental summary rating factor

Average score for non-standard, missing, and unreferenced objects. Score of under (4) indicates that individual scores should be reviewed more closely.

(TEM 201 + 202 + 203 + 204 + 205 + 207 + 208) / 7

Data Definition Metrics - TDM

Software metrics are generally recognized as important for companies wishing to judge the quality of their software systems. While much thought has been put into "procedural" metrics (e.g. McCabe & Halstead), very little has gone into "data" metrics. Information Engineering and Data Driven Design have spawned the realization within the IS community that the data side is the starting point for the new design or the re-design of application systems. Given this fact, meaningful metrics for identifying the attributes, quality and related characteristics of data are essential.

Data metrics differ from procedural metrics because of inherent differences in representation and use. Procedural metrics are gathered at the program level, the recognized logical unit of work. Systems are then rated by summarizing metrical data for all programs within a system. Data metrics differ because data transcends program boundaries. While it still important to view data complexity at a program level, system wide metrics are derived by viewing a system in its entirety.

The logical starting point in analyzing data is to identify the basic work unit called a file. Physical files are defined to a program in a context that a programming language understands. These definitions are called "records". Records are defined in a file definition section of a program, but are not limited to these sections. Other structures may define or redefine a file and are required to attain a complete view of data for a physical file.

A "logical record" contains a complete representation of a record's attributes. There may be many physical records to a single logical record. This is called redundancy. The attributes of these logical records are described by data elements. Data elements can be used globally in many programs or be local to one program. "Primary" data elements are passed between programs and exclude "local" elements defined to a single program. In COBOL, primary data elements are passed through linkages or file definitions and may be defined in Copy members for reusability.

TDM 001 Number of I/O(Input/Output) records defined in the system

· file definitions

· received / passed through Call

· moved to or from

· records read into

· written from

· received / passed through a data base access

 

TDM 002 Number of logical I/O records defined in the system

No redundant definitions.

TDM 003 Number of total records defined in the system including:

· all I/O records and

· all group structures defined for local use

 

TDM 004 Number of logical records

Number of records when all redundant data records are removed and consolidated.

TDM 005 Number of repeating records

Tables using occurs or similar clause.

TDM 006 Number of repeating logical records

No redundant tables allowed.

TDM 007 Number of copy members referenced

Total number of data definition copy members actually used within the system. Includes "Copy" and "Include" statements or similar constructs.

TDM 008 Number of logical Copy members (e.g. no redundant members)

Number of unique Copy, Include or similar constructs in the system(s).

TDM 009 Number of total data elements

This is a count of all defined element names in system. Copy / Include record and element names should not be counted more than once.

TDM 010 Number of logical data elements

No redundant data elements.

TDM 011 Number of total primary data elements

Primary elements are those elements that, used as a baseline, would be required to redesign / recreate this system or a replacement system. This includes all elements passed between programs or systems through linkage, file or data base structures. Does not include derived (i.e. calculated) values where possible.

TDM 012 Number of local data elements

Count of the number of data elements defined only within given programs, but not defined at the system level (in a copy member).

TDM 013 Number of aliases

Aliases are redundant data elements representing the same physical data with different root data names. Redundancy can be eliminated or reduced significantly. Aliases make programming more confusing and may even cause errors.

TDM 014 Number of synonyms

Synonyms are similar to aliases in that they represent the same physical data, but use different definitions. They typically redefine elements using a different Picture clause (COBOL). Synonyms are necessary in most languages, but should be named similarly and in a uniform manner for clear identification. Synonyms must also be identified for future transfer to a data dictionary.

TDM 015 Number of homonyms

Homonyms are basically the opposite of the alias and synonym problem. They are data elements with the same name, but represent different physical data. They often have different physical lengths. The homonym is dangerous because programmers may interpret them as related when they are totally unrelated.

TDM 017 Number of conditionals

These are equivalent to 88 levels in COBOL or similar allocations in other language types.

TDM 018 Total number of hard coded literals

In COBOL, these are created in the data division and are assigned static data values.

TDM 019 Number of logical literal definitions

Total number of unique hard coded literals.

TDM 020 Total number of field length mismatch counts

This is where two fields of differing physical lengths are related through a transfer statement (i.e. move) or conditional statement.

TDM 025 Number of logical date fields

Number of actual dates in system with actual duplicate or redundant references removed.

TDM 030 Total unreferenced elements in the system

Includes all elements that have no reference to a group level parent or to themselves directly.

TDM 040 Total physical records impacted by field expansion

Total physical number of group items determined to contain one or more references to field being expanded. This means all records, not just logical records. This is a critical metric for estimating field expansion efforts.

TDM 041 Total programs impacted by field expansion

Total number of source programs determined to contain one or more references to field being expanded. This is a critical metric for estimating field expansion efforts.

TDM 042 Total physical files impacted by field expansion

Total number of physical data files determined to contain one or more references to field being expanded. This is a critical metric for estimating field expansion efforts.

TDM 043 Total JCL streams impacted by field expansion

Total number of job control language members determined to contain one or more references to field being expanded. This is a critical metric for estimating field expansion efforts.

TDM 044 Total screen maps impacted by field expansion

Total number of physical screen definitions determined to contain one or more references to field being expanded. This is a critical metric for estimating field expansion efforts.

TDM 045 Total DBMS definitions impacted by field expansion

Total number of physical data base definitions (i.e. DBD, PSB, relational tables, etc.) determined to contain one or more references to field being expanded. This is a critical metric for estimating field expansion efforts.

TDM 047 Total number tables in record groups impacted by expansion

Total number of recurring records (tables) included within record groups impacted by the expansion.

TDM 048 Total number reports in record groups impacted by expansion

Total number of reports or other printed outputs included within record groups impacted by the expansion.

TDM 050 Total record groups to be rationalized (metric created in Interim Support Planning task)

Total number of record groups which have been selected to undergo data rationalization. (In a global rationalization project, this is equal to the number of logical records in a system.) This is a critical metric for estimating data name rationalization efforts.

TDM 051 Total programs impacted by data name rationalization (metric created in Interim Support Planning task)

Total number of programs containing records in the groups to be rationalized. This is a critical metric for estimating data name rationalization efforts.

TDM 052 Average records per group (metric created in Interim Support Planning task)

Average number of records per group to be rationalized. This is a critical metric for estimating data name rationalization efforts.

TDM 060 Total number date handling routines

Total number of called subroutines, copy/includes or existing in-line code involving date conversion logic.

TDM 061 Number logical date handling routines

The number of unique types of date handling routines found in the upgrade unit. To arrive at this number, identify all different types of date routines found, and count each type once regardless of how many times it is used.

TDM 062 Total number of date fields in procedural code

Count of the total number of occurrences of date fields in the procedural code of the upgrade unit.

TDM 063 Total number of date fields used in compares and calculations

Count of the total number of date fields involved in any comparison or calculation logic in the programs.

To judge data complexity at the system level we need to look at the amount of redundancy present in the data elements within the system. To accomplish this, we must find the number of aliases, synonyms and homonyms present in the system.

TDM 101 Alias redundancy factor

Aliases are data elements mapping to the same physical data with the same definition (same length, type, occurrences), but having a different data name. This metric measures the ratio of aliases to the total number of data elements.

TDM 013 / TDM 009

TDM 102 Synonym redundancy factor

Synonyms are data elements mapping to the same physical data, but having a different definition and a different data name. Synonyms are usually valid re-definitions of data elements. This metric measures the ratio of synonyms to the total number of data elements.

TDM 014 / TDM 009

 

TDM 103 Homonym redundancy factor

Homonyms are data elements mapping to different physical data, but with the same data name. They often will have different attributes such as length and type. This metric measures the ratio of homonyms to the total number of data elements.

TDM 015 / TDM 009

 

TDM 104 Data element redundancy factor

Primary data elements are the elements passed between programs or systems through linkage, file or data base structures. These are resulting fields created for use in other programs or as final outputs. When the ratio of primary data elements to total data elements is low, it reflects that many other variables were used as work fields to derive primary elements. This may indicate a high degree of maintenance that has added complexity to the data.

TDM 011 / TDM 009

 

TDM 105 Element to program factor

This score helps derive information about how complex average program data usage is in terms of the size of the average data division. Size is an issue because it complicates element location efforts.

TDM 009 / TEM 052

 

TDM 106 I/O (primary) record redundancy factor

Redundancy found at the element level in systems can be traced back to redundancy at the record level. This score identifies redundancy (aliases) at the record level. The greater the redundancy, the lower this metric will be.

TDM 002 / TDM 001

 

TDM 107 Copy usage factor

TDM 008 / TDM 003

This metric shows the level of adherence to the use of copy members in the current system(s).

0 - 10% assign 1

11 - 20% assign 2

21 - 30% assign 3

31 - 40% assign 4

Over 41% assign 5

 

TDM 108 Relational transformation factor

TDM 005 / TDM 003

Presence of repeating records in a system tends to increase the time required to transition the system to a relational design (through increased analysis time and the time required to delineate such repeating structures in a relational model). This metric measures the level of repeating records.

0 - 10% assign 5

11 - 20% assign 4

21 - 30% assign 3

31 - 40% assign 2

Over 41% assign 1

TDM 109 Literal redundancy factor

This score demonstrates need to stabilize or externalize hard coded data.

TDM 019 / TDM 018

TDM 110 Data length mismatch factor

TDM 020 / Total source programs

Data length mismatches occur when a data element of one defined length is moved to, compared with, or in other ways involved with a data element of a different defined length. This can cause truncation of the data values, and potentially other problems as well. This metric measures the level of data length mismatches in the system(s).

No mismatches assign 5

Average 0 - 1 per program assign 4

Average 1 - 2 per program assign 3

Average 2 - 3 per program assign 2

Over 3 per program assign 1

 

TDM 112 Date transformation factor

This score helps to distinguish the level of effort which will be required to modify the data for century date change or for any other change involving dates.

TDM 025 / TDM 009

 

TDM 115 Data usage redundancy rating factor

1 - ((TDM 104 + TDM 106 + TDM 118 + TDM 109) / 4)

Aggregate average redundancy factor (note that, in this case, lower % of redundancy is rated as being better based on above calculation) of:

0 - 20 % assign 1

21 - 40 % assign 2

41 - 60 % assign 3

61 - 80 % assign 4

Over 80% assign 5

 

TDM 116 Data usage inconsistency rating factor

An aggregate average data usage inconsistency factor based upon the level of usage of copy members and the level of data length mismatches.

(TDM 107 + TDM 110) / 2

 

TDM 117 Data transformation difficulty factor

An aggregate average data transformation factor based upon the level of repeating records, data usage redundancy and data usage inconsistency.

(TDM 108 + TDM 115 + TDM 116) / 3

 

TDM 118 Total record redundancy factor

This metric measures the ratio of logical records to total records.

TDM 004 / TDM 003

TDM 121 Data dictionary coverage to actual data name usages indicator

Represents how well data dictionary definitions reflect what is actually in the system.

Score 1 (poor) through 5 (excellent)

 

TDM 122 Ratio of standard abbreviations of current data names to total of syllables in data names.

Standard consistent abbreviation used in conjunction with descriptive multi-part syllables (i.e. MASTER-FILE-BIRTH-DATE). Assign the following scores:

Score 1 (poor) through 5 (excellent)

 

TDM 123 Average data name length within the system

Total of all data name lengths within a system / the number of data names in that system

Score 1 (poor) through 5 (excellent)

 

TDM 125 Average number of syllable abbreviations making up data names

Assign the following scores where a score of 1 is poor and a score of 5 is excellent.

If average number of syllables is 1 assign 1

If average number of syllables is 2 assign 2

If average number of syllables is 3 assign 3

If average number of syllables is 4 assign 4

If average number of syllables is 5 assign 5

 

For example, MASTER-FILE-EMPLOYEE-BIRTH-DATE would be assigned a score of 5.

 

TDM 129 Data naming summary score

Based on TDM 121 through 125, rate the quality of current data name quality on a 1 (poor) through 5 (excellent) scale.

TDM 130 Total (estimated) unreferenced data element score

While TDM 030 was a calculated count of unreferenced elements and TDM 133 is a score reflecting the actual percentage of unreferenced elements, TDM 130 is a score reflecting how bad analysts believe the unreferenced problem is. This is the score that should be used if 133 cannot be calculated.

Under 10% assign 5

10 - 15% assign 4

16 - 20% assign 3

21 - 30% assign 2

Over 30% assign 1

TDM 133 Unreferenced data element score

TDM 030 / TDM 009

This metric measures the number of unreferenced data elements against the total number of data elements in the system(s). This number will likely increase as a system grows older and undergoes more maintenance changes and the data becomes, therefore, more difficult to manage.

Under 10% assign 5

10 - 15% assign 4

16 - 20% assign 3

21 - 30% assign 2

Over 30% assign 1

 

TDM 134 Unreferenced local data element factor

Total number of unreferenced data elements hard coded in the program / total number of data elements hard coded in the program

TDM 135 Unreferenced I/O element factor

Total unreferenced data elements in I/O records / total number of data elements in I/O records. Unreferenced data elements read in from files can cause problems by allowing a program to make unknown or unauthorized modifications. Relational databases make this type of unreferenced element obsolete because each program will read only what it needs. This metric is calculated at the program level.

TDM 136 Deepest logical nesting of levels in a record

The largest number of unique nesting levels for a single record. In the following example, the deepest logical nesting level would be 40.

05, 10, 10, 10, 15, 20, 25, 25, 30, 35, 40

TDM 137 Average logical nesting of levels in a record

The total number of logical nesting of levels in a record / the number of records.

TDM 138 Greatest number of data elements in a record

Count of the largest number of data elements in a record.

TDM 139 Average number of data elements in a record

The total number of data elements per record / number of records.

TDM 140 Greatest number of redefines in a record

Count of the largest number of redefines in a record.

TDM 141 Average number of redefines in I/O record definitions

The total number of redefines per I/O record / the number of I/O records.

TDM 151 Program level data quality factor

Based on TDM scores 131 through 143, rate program level data quality, which directly impacts maintainability, for each source module on a 1 (poor) through 5 (excellent) basis

TDM 152 Program average data maintainability score

Total of all TDM 151 scores for each program / number of programs

TDM 250 Data maintainability factor

(TDM 115 + TDM 116 + TDM 121 + TDM 129 + TDM 133) / 5

This metric attempts to pull together various other aggregate metrics to establish an overall rating for data maintainability.

(TDM 115 + TDM 116 + TDM 121 + TDM 129 + TDM 130) / 5

 

TDM 251 System level data usage quality

(TDM 250 + TDM 117) / 2

This metric attempts to pull together a few other aggregate metrics to establish an overall rating for data usage quality.

TDM 300 Physical data quality measure

The percentage of data which is defective. This measure may be taken at the system level and/or for each data store.

TDM 301 Physical data quality standard

Percentage of defective data considered acceptable. Data stores with defect levels below or equal to this will not be targeted for purification.

TDM 302 Data stores to be purified

Number of data stores which will undergo purification. This number is calculated by determining how many data stores have a higher defect level than the quality standard (TDM 300 > TDM 301).

TDM 305 Physical data defect score

Variance between accepted standard and actual defect rating. This measure may be taken at the system level and/or for each data store. (TDM 300 - TDM 301)

Under 5% assign 5

6 - 10% assign 4

11 - 15% assign 3

16 - 20% assign 2

Over 20% assign 1

Procedural metrics focus on the complexity, structure and defects derived in process logic. These metrics can be captured with sophisticated automated collection capabilities. Certain software products have significant market penetration, particularly in the COBOL arena, which was the first to develop process analysis tools. Most of the metrics below were developed for COBOL, and are specific to it. As automated analysis tools are developed for other languages, procedural metrics relevant to those languages will be developed and incorporated here.

VISION:Assess, PATHVU and Battlemap produce basic procedural counts for COBOL programs along with the McCabe Essential and McCabe cyclomatic complexity metrics. PATHVU also produces three scores to summarize a program's complexity, structure and defect rating. VISION:Assess additionally produces the Halstead metric. More information about these products can be obtained from the vendors identified in the Tool / Technology section of Comsys-TIM.

The metrics shown below represent the desired set of procedural metrics used in determining the quality of process logic within a program and / or application. They are subdivided into counts, defect counts and scores.

TPM 001 Alter Verbs

Alter verb count. Alter verbs affect control flow by dynamically resetting the destination of Go To statements.

TPM 002 Control Verb

A reserved word, IF, Go To, Perform, Alter, etc., used in a program control transfer.

TPM 003 Data Division Lines

Total Data Division lines of code not including comments and space lines.

TPM 004 Entry Points

A count of PROGRAM-ID statements and ENTRY verbs in a program.

TPM 005 Fall Through

A count of implicit transfers of control from one paragraph to the succeeding paragraph in the Procedure Division.

TPM 006 Go To Verbs

A count of the actual number of Go To verbs in the Procedure Division of a program.

TPM 007 Non-Standard Fall Through

A count of non-standard implicit transfers of control from a procedure in one Perform range to a succeeding Perform range.

TPM 008 Non-Standard Go To Verbs

A count of Go To verbs that do not conform to the standard established during static analysis. A Go To Exit where Exit verbs are not allowed is a non-standard Go To verb.

TPM 009 Non-Standard Perform Paragraph

Count of performed paragraphs that exist in a program when standard used during static analysis is Perform ... Thru or Perform Section.

TPM 010 Non-Standard Perform Section

A metric parameter that provides a count of Performed Sections that exist in a program when the standard used during static analysis is Perform or Perform ... Thru.

TPM 011 Non-Standard Perform ... Thru

A count of Perform ... Thru ranges that exist in a program when the standard used during static analysis is Perform or Perform SECTION.

TPM 012 Perform Total

A count of Perform verbs, including Performed paragraphs, Performed Sections, and Perform Thrus, in the program.

TPM 013 Procedural Verbs

Reserved words, such as MOVE, ADD, and SET, used to manipulate data in a program.

TPM 014 Procedure Division Lines

A count of lines in the Procedure Division that include Copy members, but not comment lines, blank lines, SKIP statements and EJECT statements.

TPM 015 Procedures

A count of paragraph names and Sections in the Procedure Division.

TPM 016 Splitting Node

A decision point, such as an IF statement or an AT END phrase, where control in a program can flow along more than one path. A reducible splitting node can be placed in a separate paragraph and Performed from the original location. Non-reducible splitting nodes represent unstructured code.

TPM 017 Termination Verbs

A count of program termination verbs including Stop Run, Goback and Exit Program statements and CICS Return statements.

TPM 018 Total Loops

A count of all Go To and Perform-based loops in the Procedure Division.

TPM 019 Total Perform Ranges

A count of unique Perform ranges, including Perform paragraph ranges, Perform Section ranges and Perform ... Thru ranges.

TPM 020 Total Program Lines

A count of lines of programming code. These lines include Copy members, blank lines and comments in the program.

TPM 021 Total Verbs

Count of reserved words in the Procedure Division of a program.

TPM 022 Total Unique I/O Types

Count of unique I/O types (verbs) for the current language in use.

TPM 025 Total Simple and Structured Programs

Count of programs in the system(s) which are rated Simple and Structured in the procedure analysis reports, or if an analysis tool is not available, are rated as such by the current system SME.

TPM 026 Total Simple and Unstructured Programs

Count of programs in the system(s) which are rated Simple and Unstructured in the procedure analysis reports, or if an analysis tool is not available, are rated as such by the current system SME.

TPM 027 Total Complex and Structured Programs

Count of programs in the system(s) which are rated Complex and Structured in the procedure analysis reports, or if an analysis tool is not available, are rated as such by the current system SME.

TPM 028 Total Complex and Unstructured Programs

Count of programs in the system(s) which are rated Complex and Unstructured in the procedure analysis reports, or if an analysis tool is not available, are rated as such by the current system SME.

TPM 029 Number of Programs to be Revised

Total number of programs selected for detailed analysis and manual adjustment.

TPM 030 Number of Programs with Duplicate Perform Action Stacks

A duplicate perform action stack is an occurrence of two or more performs, one after the other, which may be repeated in different locations throughout the source logic. This metric counts the number of programs containing at least one duplicate perform action stack.

TPM 031 Number of Programs with Procedural Coupling

Procedural coupling is a measure of association established by a connection from one procedure to another. It occurs when three or more interlocking Go To paths target two or more interlocking loops. This metric counts the number of programs containing an unacceptable level of procedural coupling.

TPM 032 Number of Programs with Paragraph Size Problems

Count of the number of programs containing at least one unacceptably large paragraph.

TPM 033 Number of Programs with Logic Spikes

A logic spike is created when a long series of related conditional statements is spread across several paragraphs instead of being contained in a single conditional statement. This metric counts the number of programs containing at least one logic spike.

TPM 034 Number of Programs with Switch Problems

Programs with many switches suffer from serious design flaws. This metric counts the number of programs containing an unacceptably high number of switches.

TPM 035 Number of Programs with Table Problems

Count of the number of programs containing table handling areas needing improvement and clarification.

The following metrics are high risk constructs that signal potential logic errors within a program.

TPM 051 Lines Dead Code (VISION:Assess)

All unexecutable lines of code including syntactically unreachable procedures and source lines.

TPM 052 Looping Perform Range Violations (PATHVU)

A count of Perform range violations that form Go To invoked loops branching to a point in a logic path that is prior to the Perform being violated. Unique to PATHVU. VISION:Assess signals this as a Recursive path.

TPM 053 Perform Range Violations (PATHVU) / Jumps (Via/ Recap)

A count of Perform range violations in the Procedure Division. A Perform range violation occurs when a Go To statement within the range of the Perform branches outside the controlling Perform.

TPM 054 Recursive Performs (PATHVU & VISION:Assess - counted differently in each product)

A count of Perform statements that lead to recursion. A recursive Perform occurs when a logic path is conditionally or unconditionally Performed by itself.

TPM 055 Runaway Logic (PATHVU)

A count of the paths that eventually lead to a point where they can fall out of the bottom of the Procedure Division.

TPM 056 Unexecutable Statements (PATHVU)

A count of statements that are not in dead paragraphs and that cannot be executed in active paragraphs in the Procedure Division.

TPM 057 Un-entered Procedures (PATHVU)

A count of procedures that cannot be reached by any logic path in the Procedure Division.

TPM 058 Unresolved Performs (PATHVU) / Premature Exits (VISION:Assess) / Live exits (Via/Recap)

A count of Perform or Perform ... Thru verbs left unresolved by reaching the Exit of another active Perform range through either a Perform or Go To control transfer.

TPM 061 Architecture Score

A metric score, unique to PATHVU, that reflects a weighted measure of a program's physical structure based on a ratio of procedures to non-standard Go To verbs, non-standard Perform verbs, Alter verbs, termination verbs and non-standard fall through occurrences. A perfectly structured program is rated at zero and becomes unacceptable when it goes over 100.

TPM 062 Average Procedure Name Length Score

A metric parameter that identifies the average length of all paragraph and Section names referenced in a program's Procedure Division. It is calculated by dividing the total length, in characters, of all paragraph and Section names by the number of procedures and Sections.

TPM 063 Code Flaw or Defect Score

A metric parameter that provides a weighted measure that indicates the severity of unexecutable statements, looping Perform range violations, Perform range violations, recursive Performs, runaway paths, un-entered procedures and unresolved Performs. A perfect rating is zero and becomes unacceptable when it exceeds 100.

TPM 064 Complexity Score

A metric parameter that produces an overall measure of the difficulty of a program's logic flow. Unique to PATHVU. It is a weighted score based on total verbs, deepest nesting level, level coefficient and percentage of control verbs. The complexity score ranges between 1 and 1000 with 1000 being very complex.

TPM 065 Cyclomatic complexity *

Measure of a program's testability. This McCabe metric measures the complexity of control flow through a program by summing the IF and OR count plus the number of components.

TPM 066 Deepest Nesting Level

Maximum depth within a logic path at which procedure references reside.

TPM 067 Diagnostics Total

Count of unexecutable statements, looping Perform range violations, Perform range violations, recursive Performs, runaway paths, un-entered procedures and unresolved Performs.

TPM 068 Essential Complexity *

Measure of how structured a program is. This McCabe metric is calculated by summing the non-reducible count and the component count.

TPM 069 Level Coefficient

The distance, in terms of nesting levels, that procedure references reside from the program's mean nesting level.

TPM 070 Mean Nesting Level

The arithmetic average of the depth within a logic path at which procedure references reside.

TPM 071 Nesting Level

The depth within a logic path at which references to procedures reside in a program.

TPM 073 Percent Unstructured

This McCabe based VISION:Assess metric shows the percentage of unstructured logic for a given program.

TPM 074 Complex Percent

This McCabe based VISION:Assess metric shows the percentage of complex logic in a given program.

TPM 080 Halstead Metrics

Two Halstead metrics assist in determining program complexity. The first is the number of total and distinct operators in a program and looks at the number of keyword verbs to determine these scores. The second is the number of operands and uses the number of variables within a program.

The following scores are based on information provided by various process analyzers. Since collecting this information is difficult without at least one of these products, the scores are summarized using product provided scores and product provided summary level reports.

The following metrics are developed as 1 to 5 rating factors based on the structure (architecture), complexity and defect rating given a program by various products. These scores are recorded uniquely on Forms 003B-3, 003B-5 or 003B-7 by calculating the rating below for each program.

TPM 250 Program Level Architectural Rating

Using Form 003B-2 / 003B-3 (PATHVU users) or 003B-4 / 003B-5 (Via/Recap users) or 003B-6 / 003B-7 (Battlemap users), calculate and enter the architecture rating for each program on the respective form.

TPM 251 Program Level Complexity Rating

Using Form 003B-2 / 003B-3 (PATHVU users) or 003B-4 / 003B-5 (Via/Recap users) or 003B-6 / 003B-7 (Battlemap users), calculate and enter the complexity rating for each program on the respective form.

TPM 252 Program Level Defect Rating

Using Form 003B-2 / 003B-3 (PATHVU users) or 003B-4 / 003B-5 (Via/Recap users), calculate and enter the defect rating for each program on the respective form.

TPM 254 System Average Architectural Rating Score

Using Form 003B-2 / 003B-3 (PATHVU users) or 003B-4 / 003B-5 (Via/Recap users) or 003B-6 / 003B-7 (Battlemap users), calculate and enter system average architectural / structural rating score on the respective form.

TPM 250 / Total # programs

 

TPM 255 System Average Complexity Rating Score

Using Form 003B-2 / 003B-3 (PATHVU users) or 003B-4 / 003B-5 (Via/Recap users) or 003B-6 / 003B-7 (Battlemap users), calculate and enter the system average complexity rating score on the respective form.

TPM 251 / Total # programs

 

TPM 256 System Average Defect Rating Score

Using Form 003B-2 / 003B-3 (PATHVU users) or 003B-4 / 003B-5 (Via/Recap users), calculate and enter the system average defect rating score on the respective form.

TPM 252 / Total # programs

 

TPM 253 Procedural Composite Score

This system average metric is calculated for each system being assessed. There is only one TPM 253 metric value per system and it is derived differently depending on whether one uses PATHVU, or another McCabe based product like VISION:Assess, Via/Recap or Battlemap as shown below.

VISION:Assess Users (using Form 003B-1), calculate as follows:

VISION:Assess Composite score 90 - 100 assign 5

VISION:Assess Composite score 80 - 89 assign 4

VISION:Assess Composite score 70 - 79 assign 3

VISION:Assess Composite score 60 - 69 assign 2

VISION:Assess Composite score 50 - 59 assign 1

PATHVU users (using Form 003B-3), calculate as follows:

(TPM 254 + TPM 255 + TPM 256) / 3

Via/Recap users (using Form 003B-5), calculate as follows:

(TPM 254 + TPM 255 + TPM 256) / 3

Battlemap users (using Form 003B-7), calculate as follows:

(TPM 254 + TPM 255) / 2

 

TPM 260 Program Average Procedural Composite Score

PATHVU Users:

For each program, calculate TPM 260 by summing TPM 250, 251 and 252 and dividing the total by 3. This is a weighted average of the PATHVU architecture, complexity and defect score and calculated using Form 003B-3 for each program. Enter TPM 260 on Forms 008A and 008B.

(TPM 250 + TPM 251 + TPM 252) / 3

VISION:Assess Users:

Calculate TPM 260 by using Average Procedural Composite Score from VISION:Assess and applying the proper 1 through 5 value as shown below for each program in the system and enter on Forms 008A and 008B.

Composite score 90 - 100 assign 5

Composite score 80 - 89 assign 4

Composite score 70 - 79 assign 3

Composite score 60 - 69 assign 2

Composite score 50 - 59 assign 1

 

Via/Recap Users:

For each program, calculate TPM 260 by summing TPM 250, 251 and 252 and dividing the total by 3. This is a weighted average of the Via/Recap essential complexity, cyclomatic complexity and defect score and calculated using Form 003B-5 for each program. Enter TPM 260 on Forms 008A and 008B.

(TPM 250 + TPM 251 + TPM 252) / 3

Battlemap Users:

For each program, calculate TPM 260 by summing TPM 250 and 251 and dividing the total by 2. This is a weighted average of the Battlemap essential and cyclomatic complexity and calculated using Form 003B-7 for each program. Enter TPM 260 on Forms 008A and 008B.

(TPM 250 + TPM 251) / 2

 

McCabe defined Cyclomatic Complexity of a program as the sum of Cyclomatic Complexity for all components in that program. He defined Essential Complexity for components, but not for whole programs. PATHVU computes Essential Complexity of a program the same way McCabe defined program-level Cyclomatic Complexity. That is, Essential Complexity of a program is the sum of Essential Complexity for all components in that program. This sum is always greater than zero.

VISION:Assess, however, sums Essential Complexity for all components in a program, then subtracts the number of components. Since Essential Complexity of every component in a perfectly structured program is one, VISION:Assess scores Essential Complexity for a perfectly structured program as a zero.

PATHVU's method shows large programs to be different from small programs - even when both are perfectly structured. The VISION:Assess method makes large and small programs look equally good if they are perfectly structured.

PATHVU computes Structured Percent by dividing the number of non-reducible splitting nodes by total splitting nodes. VISION:Assess computes Unstructured Percent by dividing the number of reducible splitting nodes by total splitting nodes. Consider the following example:

PATHVU's Structured Percents and VISION:Assess’s Unstructured Percents agree that both programs are perfectly structured. However, PATHVU's Essential Complexity indicates that Program A is larger (and therefore more difficult to maintain), while VISION:Assess’s Essential Complexity gives no such indication.

Certain metrics are useful in determining the maintainability and translatability of IBM assembler language code (ALC). The following counts and scores are calculated on a program by program basis.

TPM 501 Lines of ALC code

Count of the total number of lines of assembler language code. Includes comments but not blank lines.

TPM 502 Number of macros

Count of the number of abbreviated coded routines which, in their expanded state, designate certain procedural actions.

TPM 503 Number of non-standard macros

Count of the number of code macros which are not standard to ALC, but have been created for a specific program.

TPM 504 Number of occurrences of non-standard macros

Count of the number of times non-standard macros are used in the program.

TPM 505 Number of operation codes

Count of the unique code words specifying an operation, or action, in the program.

TPM 506 Number of operation codes using register notation

Count of the unique operation codes which directly refer to a specific address location, rather than a data name.

TPM 507 Percentage of non-standard macros

Percentage of code macros which are not standard to ALC, but have been created for a specific program.

TPM 503 / TPM 502

 

TPM 508 Percentage of register notation

Percentage of unique operation codes which directly refer to a specific address location, rather than a data name.

TPM 506 / TPM 505

 

TPM 551 Total lines of ALC code

Count of the total number of lines of assembler language code in the system(s). Includes comments but not blank lines.

TPM 552 Total number of macros

Count of the number of abbreviated coded routines which, in their expanded state, designate certain procedural actions.

TPM 553 Total number of non-standard macros

Count of the number of code macros which are not standard to ALC, but have been created for a specific system or environment.

TPM 554 Total occurrences of non-standard macros

Count of the number of times non-standard macros are used in the system(s).

TPM 555 Total number of operation codes

Count of the unique code words specifying an operation, or action, in the system(s).

TPM 556 Total number of operation codes using register notation

Count of the unique operation codes which directly refer to a specific address location, rather than a data name.

TPM 557 Percentage non-standard macros

Percentage of code macros which are not standard to ALC, but have been created for a specific system or environment.

TPM 553 / TPM 552

 

TPM 558 Percentage of register notation

Percentage of unique operation codes which directly refer to a specific address location, rather than a data name.

TPM 556 / TPM 555

 

Determining functional quality has always been a subjective task and will remain so to at least some degree. Comsys-TIM brings a level of objectivity to determining the functional condition of an application that previously did not exist. Many attempts in determining functional quality of an application have been to ask a user what they do not like about the current system. While this is one aspect, it forces the user to couch their comments in terms of the current system's architecture. Current system architectures are the main reason driving redevelopment criteria and therefore limit a user's ability to articulate what they really need.

The metric categories below support short and long term user views of requirements. While the metrics alone do not specify requirements, they do provide a snapshot of the current situation and provide insights into how it can be remedied in the short and long run.

Quality metrics focus on current system performance in terms of reliability, backlog requests, performance and IS responsiveness. Conformance metrics focus on how well the current system meets strategic functional requirements. Again, this category deals with functionality (what the system does) and not architecture (how the system is put together to achieve functional requirements).

Assessing current application issues, such as productivity, quality and costs, provides essential input to the Interim Maintenance Plan, a key deliverable of Inventory / Analysis. Most IS shops do not keep these statistics. But to assess current needs and build a plan to meet those needs, baseline statistics must be established.

If these statistics are already in place, and there is no reason that an organization should not routinely record this information, the effort will be greatly simplified. Quality metrics for an application include user responsiveness and reliability counts and scores.

FQM 001 Average number of service requests submitted per month

The total number of service requests submitted during any one year period / 12

FQM 002 Average number of service requests processed per month

The total number of service requests processed during any one year period / 12

FQM 003A Total number of service requests outstanding - short term

Count of the total number of outstanding service requests which have been submitted but not yet fulfilled for the short term.

FQM 003B Total number of service requests outstanding - long term

Count of the total number of outstanding service requests which have been submitted but not yet fulfilled over the long term.

FQM 003 Total number of service requests outstanding

Count of the total number of outstanding service requests which have been submitted but not yet fulfilled - for both the short term and long term.

FQM 003A + FQM 003B

FQM 004A Total number of service request hours estimated outstanding - short term

Count of the total number of hours for outstanding service requests to be fulfilled in the short term.

FQM 004B Total number of service request hours estimated outstanding - long term

Count of the total number of hours for outstanding service requests to be fulfilled in the long term.

FQM 004 Total number of service request hours estimated outstanding

Count of the total number of hours for outstanding service requests to be fulfilled for both the short and long term.

FQM 004A + FQM 004B

FQM 005 Average number of hours per service request

The total number of hours per month spent on functional, corrective, and perfective service requests / the average number of service requests processed per month

FQM 008 + FQM 009 + FQM 010 / FQM 002

FQM 006 Function points added per month

Function points were created as a measurement to quantify the functionality provided by IS organization relative to the end user's perspective. This metric is a count of the total function points added to the system(s) each month.

FQM 007 Number of maintenance support personnel

Should reflect logical persons assigned per month to reflect part-time support.

FQM 008 Hours per month spent on functional (adaptive) enhancements

The number of hours spent each month on functional enhancements. Includes major and minor functional requests to adapt software to data / processing requirements.

FQM 009 Hours per month spent on corrective maintenance

The number of hours spent each month on corrective maintenance. Includes all failure correction and other errors reported by users.

FQM 010 Hours per month spent on perfective changes

The number of hours spent each month on perfective changes. Includes performance tuning and technological upgrades.

FQM 011 Fully costed programmer / analyst per annum.

Includes all overhead charges, salary and benefits.

FQM 012 Average hours per work day

The total number of hours worked in a given month / the number of days worked in a given month.

FQM 013 Average work days per month

The total number of days worked per month / the number of months worked

FQM 014 Hours of system "down time" per month

Number of hours system is not functioning when it should be.

FQM 015 Hours of system "up time" per month

Number of total hours system functions per month.

FQM 016 Percentage of time system exceeds batch run time window

When the amount of time required to complete the system processing is greater than the amount of time available for that system (i.e., 24 hours for the daily cycle)

FQM 017 Percentage of time on-line transactions exceed desired window

When the amount of time required to complete an on-line transaction is greater than the time considered to be acceptable (i.e., 7 seconds per transaction when the application requires a 3 second response time)

FQM 051 Average cost per programmer / analyst per hour

The average cost for each programmer / analyst on an hourly basis

FQM 011 / (FQM 012 * FQM 013 * 12 months)

 

FQM 052 Average cost per service request

The average cost for each service request

FQM 005 * FQM 051

 

FQM 054 Functional improvement rating

FQM 008 / (FQM 009 + FQM 010)

A higher rating, 4 or 5, indicates that support activity is focused on adding business related enhancements to a system. A low number means that support teams are just trying to keep the system working. Shifting this to a higher ratio may require code improvement activities, found in Comsys-TIM Positioning stage, or other infrastructure related adjustments may be required.

Under 1 assign 1

1 to 1.99 assign 2

2 to 2.99 assign 3

3 to 3.99 assign 4

over 4 assign 5

 

FQM 055 Mean time to failure

The average number of days between implementing a new or changed system or group of related systems, and its subsequent failure (abnormal end) in processing.

FQM 056 Mean time to repair

The average number of days between a failure (abnormal end) in processing, and the implementation of the corrective change to resolve the problem.

FQM 057 Number of person days per function point

Count of the number of days required for a programmer to complete coding the functionality contained in a single function point.

FQM 058 Service request backlog rating

Use the larger resulting number of the following:

FQM 003 / FQM 002 or

FQM 004 / (FQM 012 * FQM 013)

Service request backlogs indicate the amount of time required to fulfill all outstanding user requests. When hidden backlogs are accounted for, backlogs should be less than one year. The industry average is 36 months. Poor scores indicate potential Positioning or major architectural improvements based on findings in related assessment tasks.

Over 36 months assign 1

25 - 35 months assign 2

18 - 24 months assign 3

12 - 18 months assign 4

Under 12 months assign 5

 

FQM 059 System down time rating

FQM 014 / FQM 015

System down time means that users are not receiving information and money is being lost. A low score indicates that, based on related technical findings, the system or the processes supporting the system may need to be improved.

Under 1 % assign 5

1.1 - 3 % assign 4

3.1 - 5% assign 3

5.1 - 6% assign 2

Over 6 % assign 1

 

FQM 060 System performance rating

(FQM 016 + FQM 017) / 2

A low score indicates that the system requires either performance tuning or adjustments to an architecture that is processing data in an inappropriate method for the business requirement being addressed. Beyond tuning, organizations should look to the data architecture and functional implementation used to build the current system.

Under 5 % assign 5

5 - 10 % assign 4

11 - 20 % assign 3

20 - 30 % assign 2

Over 30 % assign 1

 

FQM 061 User functional rating

If realistically attainable, a user view of the current system's functionality should be scored as shown below. This is a user satisfaction rating that should confirm more sophisticated user backlog and functional conformance (FCM) ratings.

Excellent - meets all user needs today assign 5

Good - meets needs well / enhancements needed assign 4

Average - would meet needs with much work assign 3

Fair - meets minimal needs for now assign 2

Poor - does not meet needs at all assign 1

 

FQM 150 Functional quality rating

This is a summary average of all functional quality ratings. It is merely an indicator, based on a score lower than 4, to look at the individual factors used to calculate the composite score Management should take no action based on this metric alone.

(FQM 054 + FQM 058 + FQM 059 + FQM 060 + FQM 061) / 5

These metrics identify the level of conformance between the current system and target replacement system. These numbers are based on an entity mapping and functional mapping process defined in the Inventory / Analysis stage of Comsys-TIM. The effort required to gather this information is significant and will typically be performed only when a replacement initiative is fairly certain.

FCM 001 Number of primary entities identified in the target application

A primary entity may be defined as a person, place, thing or concept that has characteristics of interest to the enterprise, and is something about which we store data. This metric counts the number of primary entities in the target (final) application.

FCM 002 Number of primary entities identified in the current application

Count of the number of primary entities contained in the current (existing) application.

FCM 003 Number of overlapping entities

Count of the number of entities which appear in both the current and the target applications

FCM 004 Number of target system entities not defined in current application

Count of the number of entities which are contained in the target (final) application, but not the current (existing) application.

FCM 005 Number of current primary entities not defined in target but required

Count of the number of entities which are contained in the current (existing) application, but not in the target (final) application.

FCM 006 Percentage of current primary entities defined in target

The percentage of primary entities defined in both the current and target (final) applications.

FCM 003 / FCM 001

 

FCM 007 Entity overlap factor

This metric measures the level of overlap between the current system entities and the target system entities. A high level of overlap will indicate that many of the current system entities may be reused in the target system.

FCM 006 value of 90 -100% assign 5

FCM 006 value of 80 -89% assign 4

FCM 006 value of 70 -79% assign 3

FCM 006 value of 60 -69% assign 2

FCM 006 value of 50 -59% assign 1

 

FCM 008 Percentage of current entities not defined in target but required

The percentage of entities defined in the current application, not in the target application. These entities are required in the target application.

FCM 005 / FCM 001

 

FCM 009 Entity value added factor

This metric attempts to quantify the added value of the current system to the redevelopment process in terms of the percentage of current entities not described in the target system, but required. These entities can be reused from the current application if the target model is revised to contain them.

FCM 008 value of 50 -100% assign 5

FCM 008 value of 30 - 49% assign 4

FCM 008 value of 20 - 29% assign 3

FCM 008 value of 10 - 19% assign 2

FCM 008 value of 0 - 9% assign 1

 

FCM 010 Data reusability factor

This metric measures how much of the current data/entity relationships may be reused in the target system.

(FCM 007 + FCM 009) / 2

 

FCM 015 Number of function hierarchy diagrams to be created for this project

A function hierarchy diagram charts the breakdown of the activities of an enterprise into progressively increasing detail. Depending on the redevelopment objectives, a project may entail a single function hierarchy diagram of the enterprise, or multiple diagrams of smaller portions of the enterprise, segmented by system, department, business area, etc. This metric counts the number of function hierarchy diagrams necessary for this project. (It is assumed that this number will also represent the number of function dependency diagrams required, as they stand in a one-to-one relationship with function hierarchy diagrams.)

FCM 016 Number of systems being merged

Count of the number of systems which will be consolidated into the target system.

FCM 021 Number of functions identified in the target application

A function is a group of business activities, which together completely support one aspect of furthering the mission of the enterprise. This metric identifies the total number of functions defined in the target (final) application.

FCM 022 Number of functions identified in the current application

Count of the number of functions present in the current (existing) application.

FCM 023 Number of overlapping functions

Count of the number of functions present in both the current and target applications.

FCM 024 Number of target system functions not defined in current application but identified

Count of the number of functions which are contained in the target (final) application, but not the current (existing) application.

FCM 025 Number of current functions not defined in target but required

Count of the number of functions which are contained in the current (existing) application, but not in the target (final) application.

FCM 026 Percentage of current functions defined in target application

The percentage of functions defined in both the current and target applications.

FCM 023 / FCM 021

 

FCM 027 System-level functional reusability factor

This metric measures the level of overlap between the current system functions and the target system functions. A high level of overlap will indicate that many of the current system functions may be reused in the target system.

FCM 026 value of 80 -100% assign 5

FCM 026 value of 60 -79% assign 4

FCM 026 value of 40 -59% assign 3

FCM 026 value of 20 -39% assign 2

FCM 026 value of 0 -19% assign 1

 

FCM 028 Percentage of functions not defined in target but required

The percentage of functions defined in the current application, but not in the target application. These functions are required in the final application.

FCM 025 / FCM 021

 

FCM 029 Functional value added factor

This metric attempts to quantify the added value of the current system to the redevelopment process in terms of the percentage of current functions not described in the target system, but required. These functions can be reused from the current application if the target model is revised to contain them.

FCM 028 value of 50 -100% assign 5

FCM 028 value of 30 - 49% assign 4

FCM 028 value of 20 - 29% assign 3

FCM 028 value of 10 - 19% assign 2

FCM 028 value of 0 - 9% assign 1

 

FCM 035 System functional reusability factor

This metric measures how many of the current system functions may be reused in the target system.

(FCM 027 + FCM 029) / 2

 

FCM 036 Number of logical functions in current application

Count of the total number of unique(non-redundant) functions in the current application.

FCM 037 Number of current functions to be consolidated

Count of the total number of complementary functions focusing on the same business activities which must be combined together to create the target application.

FCM 038 Total number of current functions to be eliminated

Count of the total number of current functions which are redundant and must be removed to create the target application. This number does not include the one version of each function which will be retained.

FCM 039 Total number of current functions to be re-aggregated

Typically, programs which map to multiple functions or multiple programs which map to the same function are candidates for re-aggregation. This metric is a count of the total number of current functions which should be removed or re-located to ensure the least level of functional overlap in the target system.

The following metrics define the potential for functional reuse on a module basis.

FCM 041 Number of complete functions in module also identified in the target application

Count of the total number of complete functions in a source module identified in the current application that have also been identified in the target (final) application.

FCM 042 Number of partial functions in module also identified in the target application

Count of the total number of partial functions in a source module identified in the current application that have also been identified in the target (final) application.

FCM 043 Source program reusability factor

This score identifies the level of reusability at the source module level for a target application. The higher the metric, the more reusability.

FCM 041 > zero and FCM 042 > zero assign 5

FCM 041 > zero and FCM 042 = zero assign 4

FCM 042 > zero and FCM 041 = zero assign 3

FCM 041 = zero and FCM 042 = zero assign 1

Function points were created as a measurement to quantify the functionality provided by IS organization relative to the end user's perspective. The SPQR implementation of function points is based on the original 1979 Function Point methodology by Alan Albrecht. The SPQR implementation allows us to derive the amount of Procedural Code associated with delivering a project in any given language.

The SPQR method requires answers to eight questions:

Number of inputs to the program

Number of outputs from the program

Number of inquiries by users of the program

Number of data files created or updated by the program

Number of interfaces to other programs

The complexity of the problem (on a scale of 1 to 5)

The complexity of the code (on a scale of 1 to 5)

The complexity of the data (on a scale of 1 to 5)

 

By answering these questions we can compute the number of function points associated with this program. The SPQR method has a table of 30 common source code language multipliers, which when provided with the number of function points can compute the number of source lines necessary to complete this problem.

Then we can compare this with actual lines of code written in the following metric. Ratio of the number of function points counted to the actual lines or source code - excluding comments and counting copy and macro expansions as reused code. This could provide a judgment about the quality of the code in the program or system being studied.

The architecture of an application defines how it is assembled. Older systems typically evolve over time and reflect no cohesive plan or structure. This is what generally causes much of the user dissatisfaction with older applications. Information cannot be made available on an ad hoc basis, sharing of data between functional areas is cumbersome, interfaces or bridges are required to adequately share data and processing windows may be drawn out over a lengthy batch cycle.

The metrics in this section are designed to help determine what is wrong with an application that is not strictly technical or functional in nature. Categories include general conformance to a strategic architecture, data usage analysis and user views of information.

General architectural metrics rate the overall quality of how a system was designed.

AGM 001 Deepest Call linkage structure

If PGM A calls B and PGM B calls C, this is a Call linkage structure depth of 3. This metric indicates a worst case occurrence of this metric where a programmer must trace logic across many Call transfers to ascertain how a program functions. (This could be a symptom of functional scatter.)

AGM 002 Average Call linkage structure

Average transfer depth (see Call linkage definition for AGM 001) is where the deepest point of each Call path is determined and averaged across the system. This gives an indication of how many programs, on average, an analyst must search across in order to research a change.

AGM 003 Call linkage rating factor

Rate Call linkage factor reflects inadequate Call usage by tracing through too many modules, tracing recursive Call paths or non-use of the Call. The first two cases create analyst confusion while non-use of the Call verb increases source program size.

No problems assign 5

Minor issue assign 4

Recurring issue assign 3

Major linkage issues assign 2

Urgent linkage problem assign 1

 

AGM 004 Data entry to summary level factor

This indicates average length of time it takes to get executive or management level summary information after the information is entered into the system. This "information aging" metric indicates if decisions are being made based on inaccurate or out of date information.

Immediate update of information with summary level data available immediately assign 5

Nightly extract availability assign 4

2 - 5 day availability assign 3

6 - 10 day availability assign 2

Over 11 days to receive summary level data assign 1

 

AGM 005 Batch versus on-line update factor

This indicates how long data takes to get into the central data base where it is available to users and other systems. This is another information aging metric that indicates if users are using old or outdated information.

Immediate update of information assign 5

Over night update assign 4

Second day / weekend update assign 3

3 - 4 day update of data assign 2

Over 4 days to update master file / DBMS assign 1

 

AGM 006 Platform conformance factor

This metric indicates how well the current hardware and software environment conforms to ideal or strategic requirements. This is important since obsolete indicates need for immediate action while acceptable means that management can divert funding to other activities.

Excellent / strategic assign 5

Acceptable assign 4

Acceptable for interim assign 3

Unacceptable assign 2

Obsolete - must migrate assign 1

 

AGM 007 Functional scatter factor

This indicates whether functions, defined and mapped on Comsys-TIM Form 004, are split across multiple programs or systems. Functional scatter increases the amount of time required to research and implement a change. If Form 004 is not completed yet, review scatter factor with functional technical analysis and estimate findings as shown below.

No problem assign 5

Minimum scattering assign 4

Minimum scattering / impacts maintenance assign 3

Major scattering / impacts maintenance assign 2

Major scattering / strategic application assign 1

 

AGM 008 Functional clumping factor

Clumping is when many functions or processes map to a few single large modules. This is indicated on Form 004 or may be determine if many people are working on a few, very large modules - this is typically a large scale maintenance problem.

No problem assign 5

Minimum scattering assign 4

Minimum scattering / impacts maintenance assign 3

Major scattering / impacts maintenance assign 2

Major scattering / strategic application assign 1

 

AGM 009 I/O isolation factor

This indicates if I/O logic is tightly coupled with business logic or if it is isolated into sub-functions. Minimal isolation, as found in some IBM IMS systems for example, increases maintenance research efforts and complicates migration sensitive factors such as re-hosting, data architecture migration or business rule review and extraction.

I/O isolated at system level assign 5

I/O isolated at a program level assign 4

I/O partially isolated assign 3

Minimum I/O isolation assign 2

No I/O isolation assign 1

 

AGM 030 Total number of non-standard technologies in use

This is a count of all non-standard technologies in use, such as home-grown TP monitors, obscure languages, custom-built functions or interfaces, etc.

AGM 035 Total number of interface systems

This is a count of all systems which will not in the current project be assessed, but which share data with systems being assessed.

AGM 040 Total number of user-supported systems

This is a count of all systems which are maintained by their users instead of the standard MIS support groups.

AGM 100 General architectural summary rating

This is a summary rating for all architectural findings that indicates if the current system has been implemented in such a way as to ease or complicate upgrade or migration efforts. A low number means that factors used to calculate this number be researched further based on the Comsys-TIM scenario being pursued.

(AGM 003 + 004 + 005 + 006 + 007 + 008 + 009) / 7

 

Presentation layer metrics help assess the quality of how a user inputs and receives information from a given application. This includes data entry, report access, on-line interface ability, executive information systems and end user query ability.

APM 001 Data entry quality score

Identifies level of conformance between the way data currently enters the system and the way it should ideally enter the system. A low score means that major redesign of not only the input facility itself, but the underlying core system, must be pursued.

Conforms exactly assign 5

Requires cosmetic upgrades assign 4

Requires redesign assign 3

Requires redesign & platform change assign 2

Requires complete overhaul assign 1

 

APM 002 Presentation layer isolation factor

If business logic is tightly coupled with presentation management logic, as is the case where home grown TP monitors are in use, maintenance and migration of the system could be highly complicated. If application programmers are not even aware of the presentation interface or related requirements, a score of 4 or 5 may apply.

Isolated at system level assign 5

Isolated at a program level assign 4

Partially isolated assign 3

Minimum isolation assign 2

No isolation assign 1

 

APM 003 Presentation platform conformance factor

This metric reflects the acceptability of current TP monitor and presentation platform. Home grown or highly proprietary TP monitors score low, while portable interfaces or the use of middleware capability would score closer to a 4 or 5.

Functions presented on strategically acceptable platforms assign 5

Acceptable for long term (over 3 years) assign 4

Acceptable for mid term (1 - 3 years) assign 3

Unacceptable mid term (1 - 3 years) assign 2

Obsolete - must change near term assign 1

 

APM 004 Presentation layer summary factor

This metric summarizes all presentation analysis findings. A low score indicates that further research into the factors comprising this score should be reviewed in light of the scenario driving this assessment.

(APM 001 + 002 + 003) / 3

 

APM 010 Report Count

This is a count of the total number of reports in the system(s) to be assessed.

Data usage layer metrics assist in determining the quality of an application's data usage and representation. Some of the information may be gleaned from the data definition metrics (TDM) but additional information must be derived from applications support or data administration personnel.

ADM 001 Data redundancy factor

Identify level of physical data redundancy within the current system based on the same physical data (customer for example) being stored in multiple places. Redundancy leads to poor or inadequate tracking of information, conflicting user results or other risks to the business.

Little or no redundancy assign 5

Requires cosmetic upgrades assign 4

Requires redesign assign 3

Requires redesign & platform change assign 2

Requires complete overhaul assign 1

 

ADM 002 Data consistency factor

Identify level of data inconsistency within a given system based on users always getting the same information to a given query regardless of system are providing the information. Inconsistency reflects poorly on IS organizations and can lead to hidden costs to the business.

Consistent within the system assign 5

Minimum inconsistency assign 4

Data sharing within system is difficult assign 3

Users receive different answers to identical queries assign 2

Urgent inconsistency problems assign 1

 

ADM 003 Data integration factor

Rate ability to share data between system and various interface systems or sub-systems. If data sharing is difficult or impossible, whole sub-systems may emerge to deal with the problem, while cross functional data sharing remains a problem. Indicates a need to develop and migrate to a new data architecture.

Consistent - no problem assign 5

Minimum inconsistency in shared data assign 4

Data sharing among systems is difficult assign 3

Data sharing is impossible assign 2

Data sharing issues are causing severe assign 1

business problems

 

ADM 004 Data architecture conformance score

Identifies level of conformance between existing data bases and files and the ideal data architecture. This should reflect inaccessible, non-integrated, redundant data views versus target requirements that include consistent enterprise wide data definitions, that support distributed and other strategic architectures as required.

Conforms to strategic architecture assign 5

Partially conforms to strategic architecture assign 4

Does not conform but positioned to conform assign 3

Does not conform assign 2

Does not conform / no clear migration path assign 1

 

ADM 005 End user summary information availability access rating

If end users can easily access and update data based on rapidly changing requirements, the organization will have achieved many of its strategic data architecture goals. A user acceptability rating of 1 or 2, for example, indicates need to implement a GUI front-end (middleware) or other interim solution while core systems can be redesigned.

Functions presented on strategically acceptable platforms assign 5

Acceptable for long term (over 3 years) assign 4

Acceptable for mid term (1 - 3 years) assign 3

Unacceptable mid term (1 - 3 years) assign 2

Obsolete - must change near term assign 1

 

ADM 006 Data architecture summary rating

This metric summarizes data access layer findings. A score of 1, 2 or 3 indicates a need to review each ADM rating to determine specific areas to be researched and added to the interim and strategic plans based on scenario the driving this assessment.

(ADM 001 + 002 + 003 + 004 + 005) / 5

 

ADM 007 Total number of batch conversion bridges to be built

This metric determines how many batch modules must be built to process data to allow it to pass between data stores which have been expanded and data stores which have not, in a field and record size expansion project. These batch conversion bridges are run at break points in the job execution stream.

ADM 008 Total number of API’s to be built and utilities to be modified

This metric defines the number of application programming interfaces, or called subroutines, which must be built to process data to allow it to pass between data stores which have been expanded and data stores which have not, in a field and record size expansion project. These API’s and utilities are run concurrent with the execution of the application.

AAM 001 Architecture summary rating

This metric summarizes general, presentation layer and data access layer architecture assessment findings. It is an indicator to look back at various other metrics within the architecture assessment tasks should a system score below a 4 or 5. No decisions should be based on this metric.

(AGM 100 + APM 004 + ADM 006) / 3

In order to gain a complete picture of an application area, it is important to assess the support team assigned to supporting that area. It is also important to determine the acceptance and usage of project management skills, testing methodology, development methodology and related subject matter. These issues are all critical to the success of any redevelopment effort.

ISM 001 Individual tally

Total individuals assigned to the application team.

ISM 002 Maintenance support tally

Total individuals assigned to the application maintenance.

ISM 003 Total support staff tally

Total individuals assigned to the application development.

ISM 004 Average years experience of maintenance staff.

The total years of experience for each maintenance employee / number of maintenance employees.

ISM 005 Average years experience of development staff.

The total years of experience for each development employee / number of development employees.

ISM 006 Development methodology rating

This rating indicates that an application support area is highly skilled or alternatively lacks skill in formal development techniques. No formal method signals potential runaway projects, inadequate analysis or design focus and the mismanagement of IS resources.

Single methodology accepted and in use assign 5

Single methodology accepted and being implemented assign  4

Multiple methodologies in use assign 3

No methodology accepted / evaluation underway assign 2

No methodology / see no need for a methodology assign 1

ISM 007 Project management (P/M) rating

This rating indicates that an application support area either uses or does not use project management tools and techniques. A 1, 2 or 3 rating opens up an IS area to project mismanagement, budget overruns and resource mismanagement.

P/M approach accepted, automated and in use assign 5

Single approach accepted and being implemented assign 4

Multiple approaches in use assign 3

No approach accepted / evaluation underway assign 2

No P/M approach and see no need for a such

an approach assign 1

 

ISM 008 Testing maturity factor

Late projects or broken systems can be tied back to inadequate testing in many cases. A score of less than 4 may signal potentially unreliable systems and the need to rework maintenance or enhancement projects while making any redevelopment activities a high risk endeavor.

Multi-platform unit and system testing methodology in place assign 5

Production environment unit and system test methodology in place assign 4

Unit testing approach only assign 3

Testing done at individual discretion of programmer assign 2

No / limited testing assign 1

 

ISM 009 Test bed availability factor

Production test beds (or suites) tend to cover less than 30% of a given systems functionality during a typical run and process for extended periods. Tools and techniques should be pursued to create test beds that dramatically decrease execution time while increases logic execution coverage.

Streamlined system test bed in place - with over

80% coverage assign 5

Partially streamlined test bed in place - with over

70% coverage assign 4

Production test bed in use - with over 60% coverage assign 3

Production test bed in use - with 40 - 60% coverage assign 2

Production test bed in use - under 40% coverage assign 1

ISM 010 Testing tool usage factor

Testing is the most widely automated area in the IS market. Not having any tools in this area opens up an organization to increase testing time, increased cycles and reduced reliability of the system once in production.

On-line & batch test coverage monitors in widespread use assign 5

On-line & batch test coverage monitors in limited use assign 4

Batch test coverage monitors in widespread use assign 3

Testing tools in limited / isolated use assign 2

No testing tools in use assign 1

 

ISM 011 CASE planning tool factor

Skill levels with planning workstations

Integrated CASE planning workstation in widespread use assign 5

Component CASE planning tool in widespread use assign 4

CASE planning tool in limited use assign 3

Own CASE planning tool - no training and no use assign 2

No CASE planning tool at all assign 1

 

ISM 012 CASE analysis tool factor

Skill levels with analysis workstations and modeling facilities

Integrated CASE analysis workstation in widespread use assign 5

Component CASE analysis tool in widespread use assign 4

CASE analysis tool in limited use assign 3

Own CASE analysis tool - no training and no use assign 2

No CASE analysis tool at all assign 1

 

ISM 013 CASE design tool factor

Skill levels with design workstations and modeling facilities

Integrated CASE design workstation in widespread use assign 5

Component CASE design tool in widespread use assign 4

CASE design tool in limited use assign 3

Own CASE design tool - no training and no use assign 2

No CASE design tool at all assign 1

 

ISM 014 Application generation tool factor

Skill levels with application generation tool(s)

Integrated CASE generator in widespread use assign 5

Component application generation in widespread use assign 4

Application generation in limited use assign 3

Own application generation - no training and no use assign 2

No application generation at all assign 1

 

ISM 015 Maintenance/Redevelopment tool summary rating factor

The level of maintenance and redevelopment tool utilization is summarized below. A rating of 3 or lower signals that there may be opportunities to further automate current tasks and that any planned redevelopment efforts must accommodate a technology learning curve.

(ISM 060 + 061 + 062 + 063 + 064 + 065 + 066 + 067) / 8

 

ISM 016 Coding standards rating

Since the majority of systems are still developed or maintained using 3rd generation languages, coding standards remain critical to long term stability of systems. This includes COBOL, C, C++ and any other sanctioned languages. Lack of standards indicates one more area of potential breakdown in the development cycle.

Coding standards accepted and in use assign 5

Coding standards accepted and being implemented assign 4

Multiple coding standards in use assign 3

No coding standards in place / evaluation underway assign 2

No coding standards / see no need for coding standards assign 1

 

ISM 020 Quality assurance (Q/A) factor

Organizations having implemented Total Quality Management (TQM) have shown improvement in the reliability and integrity of their systems. This metric reflects the use of quality reviews in the analysis, design, coding, maintenance and production control aspect of the IS organization.

Formal Q/A team / process in place assign 5

Formal Q/A process in place assign 4

Informal Q/A process in place assign 3

Limited Q/A reviews assign 2

No Q/A reviews assign 1

 

ISM 021 Measurement Maturity Factor

Measurement covers every aspect of an IS organization including system attributes (Comsys-TIM forms 003 and 006), functional attributes (Comsys-TIM form 005), skills (form 007), user satisfaction (form 005), quality and financial payback. Without formal measurement programs, IS cannot establish feedback and improvement loops to required to improve quality and control or reduce costs.

Comprehensive measurement process in place assign 5

Limited measurement / feedback process in place assign 3

No measurement / feedback process in place assign 1

 

ISM 022 SEI Rating

The Software Engineering Institute of Carnegie Mellon performs an IS process maturity assessment to ascertain the maturity level of an IS department. This metric may be used to average other ratings in the IS Infrastructure Assessment task.

SEI rating of 5 assign 5

SEI rating of 4 assign 4

SEI rating of 3 assign 3

SEI rating of 2 assign 2

SEI rating of 1 assign 1

 

ISM 030 CASE tool summary rating factor

The level of development and maintenance tool utilization is summarized below. A rating of 3 or lower signals that there may be opportunities to further automate current tasks and that any planned redevelopment efforts must accommodate a technology learning curve.

(ISM 011 + 012 + 013 + 014) / 4

 

ISM 031 Testing summary rating factor

This summary rating indicates overall maturity of the testing process. A score of less than 4 indicates an immediate need to review the overall testing process, establish procedures, procure tools and implement standards. Testing can have a major impact on near term system reliability and quality.

(ISM 008 + 009 + 010) / 3

 

ISM 035 LTM population indicator

This metric is used to denote the use of a legacy transition meta-model. If populating and using an LTM, set this metric to 1. If not, set to 0.

ISM 040 Number of support teams

This is a count of the number of application support teams involved in maintaining current system(s). Do not include teams whose applications are not within the scope of the current project.

ISM 041 Members per support team

This is an estimated average number of people per maintenance support team. When estimating this average, do not include teams whose applications are not within the scope of the current project.

ISM 050 IS infrastructure maturity rating

A summary rating factor of less than 4 indicates a need to examine each of the meters that went into calculation of this composite score. This metric should not, however, be used as a basis for taking any action in and of itself. It does signal that much catching up may be required if redevelopment efforts are anticipated near term.

(ISM 006 + 007 + 030 + 020 + 021 + 022 + 31) / 7

 

Note: If no SEI (ISM 022) in place, omit 022 and divide by "6"

 

ISM 060 Migration tool factor

Skill levels with migration tools

Integrated Migration tool in widespread use assign 5

Component Migration tool in widespread use assign 4

Migration tool in limited use assign 3

Own Migration tool - no training and no use assign 2

No Migration tool at all assign 1

 

ISM 061 Code analysis tool factor

Skill levels with code analysis tools

Integrated Code analysis tool in widespread use assign 5

Component Code analysis tool in widespread use assign 4

Code analysis tool in limited use assign 3

Own Code analysis tool - no training and no use assign 2

No Code analysis tool at all assign 1

ISM 062 Environmental analysis tool factor

Skill levels with environmental analysis tools

Integrated Environmental analysis tool in widespread use assign 5

Component Environmental analysis tool in widespread use assign 4

Environmental analysis tool in limited use assign 3

Own Environmental analysis tool - no training and no use assign 2

No Environmental analysis tool at all assign 1

 

ISM 063 Data Definition analysis tool factor

Skill levels with data definition tools

Integrated Data Definition analysis tool in widespread use assign 5

Component Data Definition analysis tool in widespread use assign 4

Data Definition analysis tool in limited use assign 3

Own Data Definition analysis tool - no training and no use assign 2

No Data Definition analysis tool at all assign 1

 

ISM 064 Code improvement tool factor

Skill levels with code improvement tools

Integrated Code improvement tool in widespread use assign 5

Component Code improvement tool in widespread use assign 4

Code improvement tool in limited use assign 3

Own Code improvement tool - no training and no use assign 2

No Code improvement tool at all assign 1

 

ISM 065 Data definition migration tool factor

Skill levels with data definition migration tools

Integrated Data definition migration tool in widespread use assign 5

Component Data definition migration tool in widespread use assign 4

Data definition migration tool in limited use assign 3

Own Data definition migration tool - no training and no use assign 2

No Data definition migration tool at all assign 1

 

 

ISM 066 Repository technology

Skill levels with repository technology tools

Integrated Repository technology in widespread use assign 5

Component Repository technology in widespread use assign 4

Repository technology in limited use assign 3

Own Repository technology - no training and no use assign 2

No Repository technology at all assign 1

 

ISM 067 Reverse engineering tool factor

Skill levels with reverse engineering tools

Integrated Reverse engineering tool in widespread use assign 5

Component Reverse engineering tool in widespread use assign 4

Reverse engineering tool in limited use assign 3

Own Reverse engineering tool - no training and no use assign 2

No Reverse engineering tool at all assign 1

Technical Client/Server Metrics specify information about a client/server system. This information will be useful for developing work estimates for tasks involving client/server applications.

TCS 001 Number of unique client/server environments and platforms

Count of the total number of unique client/server environments and platforms. Count each environment/platform only once.

TCS 010 Number of GUI front-ends

Count of the total number of GUI front-ends. Count each GUI front-end only once.

TCS 011 Number of SQL accesses specified under GUI front-ends

Count of the total number of SQL accesses specified under the GUI front-ends.

Technical Validation Metrics specify information about a system to be validated after some positioning activity. This information will be useful for developing work estimates for validating positioning tasks.

TVM 005 Number of validation data stores

This is a count of the number of data stores to be validated and/or used in a validation effort.

The Integrated Assessment Metrics summarize the analysis completed during prior assessments. Metric summaries are produced at varying levels depending on application breakdown and the specific objectives of the assessment. For example, data definition quality is viewed at a system level for redevelopment reusability but at the program level for maintainability.

The metrics below are divided into maintenance ratings and redevelopment ratings. Maintenance ratings are vital to developing the interim plan and tend to drive Positioning efforts. Redevelopment ratings provide input to the strategic redevelopment plan and drive Positioning and Transformation efforts.

The maintenance metrics are divided into program and system and / or sub-system categories.

IMM 001 Program maintainability rating factor

Utilizing maintenance / redevelopment rating summary Form 008A and the following equation, calculate the program maintainability rating factor for each source program in the application that has a TDM 151 and TPM 260 rating metric. Where TDM 151 ratings have not been recorded on Form 003A, use the TPM 260 value for IMM 001.

(TDM 151 + TPM 260) / 2

This metric is a summary of maintainability for each program in the application. Unfortunately, the technology does not yet exist to calculate the TDM 151 metric for a large number of programs.

IMM 002 System maintainability rating factor

This metric is a summary of maintainability for each system.

(TEM 250 + (TDM 250 * 2) + (TPM 253 * 2) + (FQM 150 * 2)

+ AAM 001 + ISM 050) / 9

Redevelopment reusability is determined at a module and system level for functional processes and only at the system level for data. Module reusability factors are developed for each source program within the system being assessed.

IRM 001 Source module redevelopment reusability factor

Summary of quality of source module as it pertains to identification and reuse of logic in target system.

(TDM 151 + TPM 260 + (FCM 043 * 2)) / 4

 

IRM 002 System data redevelopment feasibility factor

This metric rates the viability of reusing some of the existing data definitions as input into the target data architecture.

((FCM 010 * 2) + TEM 250 + TDM 251 + AAM 001 + ISM 050) / 6

 

IRM 003 System process redevelopment feasibility factor

Summary of the quality of system in terms of reusing business rules in target system.

((FCM 035 * 2) + TEM 250 + TPM 253 + AAM 001 + ISM 050) / 6