Using Verification Metrics to estimate DO-178B/C Projects PDF Print E-mail
Article Index
Using Verification Metrics to estimate DO-178B/C Projects
Page 2
All Pages

By Gustavo Cubas, Engineering Manager, Avionyx.

While all software projects are unique by definition, they share commonalities that allow companies and project managers to learn from previous experiences.  Looking at past project performance is the first step towards having a well-planned and controlled project.  In order to learn from previous projects, a systematic look at your metrics database is a must.  This paper summarizes many valuable lessons learned that can increase your estimation accuracy in your next DO-178C compliant software project.

The challenge

Before any project starts, we all want to know how much it will cost, how long it will last, how many people will be needed and in general what resources are going to be necessary to finish it on time and under budget.

Deciding Which Metrics to Use

It is tempting to characterize project productivity and performance by tracking the number of hours per requirement.  While such metrics can be useful as a rule-of-thumb tool, it is not enough for budgeting and project programming at a task level.

Productivity is the result of a complex equation that must consider quantitative and qualitative variables.  A partial list of such variables includes the following:

Qualitative Variables  Quantitative Variables
 Table 1. Factors that affect verification effort
  • Schedule constraints
  • Team Experience (application, tools, process, customer)
  • Project Manager Experience & mentoring skills
  • Tool capabilities
  • Testing method (manual, automated, high-level, low-level, black-box, white-box etc.)
  • Level of abstraction and quality of requirements
  • Application complexity
  • Software Criticality Level
  • Process complexity (plans & standards)
  • Code & Requirements stability
  • Software methodology, language
  • # of Requirements
  • # of test cases
  • Total statements
  • Total executable statements
  • # of decision points (cyclomatic complexity)
  • Source lines of code per requirement
  • # of test stations per engineer

A project effort estimate must consider the qualitative factors to ensure that previous metrics are relevant.  For instance, when estimating the effort for a graphical library running on a 32-bit RISC processor with a Real-Time Operating System, metrics from a previous project based on a modest 16-bit processor may not be accurate due to differences in algorithmic complexity, development environment, or concurrency.

The Approach

First, Commit

Before you start collecting metrics, you will need your team’s commitment to collect them.  If they are not willing to spend 5 minutes per day filling a timecard or entering their hours in a time tracking system, then you will need to start by creating the awareness in your team about the importance of this endeavor.

Then, Plan

The next step is to plan what metrics you need to track.  Detailed activity-level metrics are good for controlling task performance, but may be too detailed for tracking overall project productivity or future reference for estimation.  This means that if you collect metrics from a detailed project schedule, you also need to have a second layer of metrics for the category of that activity.  This mapping is illustrated in the following figure.


Task Classification

AddThis Social Bookmark Button