These notes are taken on Oct. 20, 2010 from a "webinar" given by DDC-I (provider
of DeOS) and LDRA (testing tools). Supposedly, the talk will be made available
on http://ecast.opensystemsmedia.com/, with registration.
- D0-178B cert consists of 5 processes
- Planning
- Development
- Verification - the focus, and the focus of this talk
- Configuration
- Quality Assurance
- Includes 66 objectives (e.g. documenting your requirements, testing every
requirement, etc.). Requirements "with independence" means that different
aspects of the requirements are satisfied by different PEOPLE (e.g.
developer is not the same person doing code review).
- Level A: 66 objectives, 25 w/ independence
- Level B: 65 objectives, 14 w/ independence
- Level C: 57 objectives
- Level D: 28 objectives
- Level E: 0 objectives (no effect on safety)
- Purpose of verification = Detecting and reporting any errors introduced in
development. Consists of three parts:
- Review = qualitative assessment of accuracy/completeness/correctness
- Testing = Demonstrating requirements are met. Includes three kinds of
tests.
- Requirements-based
- Hardware/Software Integration
- Software/Software Integration (i.e. different pieces of software)
- Analysis = Quantitative assessment of accuracy/completeness/correctness
- Test result analysis (e.g. pass/fail for each test)
- Traceability analysis: e.g. each LOC must map to at least 1
requirement; usually done with a tool. Common traceability gaps:
- Requirement has no tests
- Requirement has no code
- Code has no requirement
- Coverage analysis = Demonstrates that test set covers/exercises all
code. Different levels of coverage analysis required for different
criticality levels:
- Level A example: The F-16 tail rudder controls the roll of the plane (even
allowing it to flip all the way over), so if the software that controls the
rudder messes up, the plane could crash. Therefore, that software is Level
A. In a simulation (during the testing phase), it was discovered that the
software would cause the plane to flip over onto its back when the plane
flew over the equator (0 degrees lattitude) at a speed that would kill the
pilot due to the Gs. The F-16 is not actually subject to D0-178B, but
makes a good example.
- Hazard Analysis consists of three steps:
- What failures can occur?
- What is the severity of each failure?
- What is the probability of the failure occuring?
- For hardware, computing the probability of failure is an "exact science." For
software, this is impossible to compute, which is why Levels A-E were
invented. However, these levels of software are considered "good enough" for
hardware with the following likelihood of failure PER HOUR OF OPERATION:
- Level A: 10^-9 (once in a billion hours)
- Level B: 10^-7
- Level C: 10^-5
- Level D: 10^-3
- Level E: N/A
(That explanation is to the best of my understanding based on what the
presenter said.)
Source given as "FAA System Safety Handbok, Ch. 3, Dec. 2000"
- D0-178C will come out in 2011; retains core of DO-178B but addresses three new
technologies which can make software development and certification must
faster.
- Formal Methods: Validating mathematically that the system meets design
criteria
- Model-based Development: Allows code generation, so verification can be
done at the model level.
- Object-Oriented Technology
- DO-178C also addes new "Tools support" in addition to the above three areas.
- DO-178C requires more stringent requirements tracing. You have to be able to
trace forwards and backwards at nearly every point in the development
hierarchy, i.e. from requirements to models/formal methods/hand-modified code
to implementation (the main body of source code). For Level A, you must trace
all the way to object code (e.g. from source code to object code and vice
versa). This half of the presentation is being given by a company called LDRA
and their main product appears to be tools to allow you to do this tracing.
- How much "extra work" does it take to develop D0-178B software?
- Level D: 1.5 to 2x over level E
- Level C: Up to 5x over level E
- Level B: Not given by presenter
- Level A: One order of magnitude over level E