Determining The Right Assurance Mix

Jay Kahn

23 October 1996

Panel Description

What is the right mix of assurance approaches for your organization?

How much does the right mix depend on factors such as your environment, reliance on technology, value of reputation, impact of security breaches, and connectivity needs?

The Joint Commission on Security faulted the traditional process as expensive, slow, and an unbalanced use of resources.

Panelists

Joe Alexander, Sun Microsystems Federal, Product Manager for Trusted Solaris

- Prior work at DISA -- responsible for building working security solutions for the DOD MLS Program

Dan Gambel, Director of INFOSEC Engineering at GRC International

- Prior work as systems integrator for Grumman Corp

- Experience as a Certifier/Accreditor for DIA

David Chizmadia is a member of the research staff in the NSA INFOSEC organization

- 10 years of Trusted Systems Criteria experience

- US Technical Liaison to the UK IT Security Evaluation and Certification Scheme

Jay Kahn (Chair)

- INFOSEC Engineer for the MITRE Corporation

Alternative Assurance Methods

Testing

Pedigree

Criteria-based assurance

System Analysis & Software Engineering Techniques Used

Tools to validate the product

Warranty

Documentation

Training

Vendor Support

Conforming to industry standards

On-the-job experience with the product

Marketplace factors - Share held by the product

Panel Conclusions: Popular & Promising Methods

Some assurance methods seem to offer more "bang for the buck" than others

- Testing

- Warrantee --> Vendor and/or integrator

- Pedigree

People (Degrees, licenses, clearances)

Process (CMMs, ISO)

Product (TPEP)

- Good Housekeeping is important

Configuration Management

Good product documentation

Training of people

Support Desk

Panel Conclusions: Business Observations

The commercial system owner does not want to be in the business of managing risk. They lay off many other risks through insurance and warrantee. The primary question is one of liability

- In business, this is an issue of law and money

What constitutes reasonable and customary precautions?

- In Government, it is often an issue of career risk

The Government is using more COTS than GOTS and custom software.

Panel Conclusions: Process

We would not advocate use the same methods for a Doctor's office and a pacemaker

The appropriate mix should depend on the system, the system environment, and the people skills available

Final mix will be risk and cost driven

The role of management is critical

- We have to sell management on what we are doing

- Management must be prepared in case something goes wrong

Panel Conclusions: Cost Benefit

All methods were potentially applicable

Not all methods seem to be cost effective

Producing assurance for its own sake is an expensive and meaningless exercise. If the assurance evidence doesn't give insight into the security mechanisms in the product/system, it has not meaningful purpose.

We don't have good records about which methods have produced secure systems. We have even less data on security failures.

There will be continuing costs in managing risk

- Both people cost and infrastructure costs

There are lots of "rice bowl" issues that can force the selection of assurance methods

Panel Conclusions: Metrics

Process "measurements" are good screening values but they cannot be used to make fine-grained judgments

The less complex the product and the closer it is to something that people have had experience with, the better the predictor value of the assurance evidence

Having assurance evidence does not predict success, but the lack of assurance evidence may predict failure

As of this time, there does not seem to be sufficient evidence to rank the value of various processes such as the CMMs, ISO, etc. If we could get some real data, we may find these processes are better than we think they are today.

Reminder

1996 WITAT Conference Notes:

- gnissc96.html

1994-95 WITAT Conference Notes:

- http://www.cse.dnd.ca/~formis/witat/ (Being changed. Check at above URL)

Backup Slides for the WITAT Web Pages

Joe Alexander

Joe Alexander of Sun Microsystems Federal is the product manager of Trusted Solaris, Sun's B1+ operating system. Joe came to Sun after retiring from the US Army in Jan 1995. While on active duty he had several challenging information technology assignments. He delivered the first computer system to the US Army JAG School and followed that with assignments in Korea managing wargaming and simulation computer systems and US Central Command where he was the strategic automation architect from 1987-1992, supporting both Ernest Will (Kuwait tanker reflagging) to Desert Shield/Storm. His final assignment was with DISA where he was responsible for building working security solutions for the DOD multilevel security program.

David Chizmadia

David Chizmadia is a member of the research staff in the NSA INFOSEC organization. His perspectives on assurance derive from 10 years of Trusted Systems Criteria experience. This experience includes three years of TCSEC/TNI evaluations, two years of developing and helping develop TCSEC guidance, two years as technical editor for and member of the technical working group that produced the US Federal Criteria, and 3 years as US Technical Liaison to the UK IT Security Evaluation and Certification Scheme. He has recently moved out of the Trusted System and Criteria arena to take a position researching the security issues surrounding Object and Mobile Agent Management Systems.

Dan Gambel

Dan Gambel, Director of INFOSEC Engineering at GRC International has a long record in information systems security programs and managing security professionals, introducing innovation into the use of commercial off the shelf security products. Over 30 years of experience in computer systems, and 20 years of experience in a wide variety of security applications. Has been the security architect for four accredited MLS systems and numerous system high integration efforts. He serves on the System Security Engineering Capability Maturity Model Steering Group and teaches computer security integration for the George Mason University Information Security Institute. Regular contributor to the National Computer Security Conference and Computer Security applications conference.

Jay Kahn

Jay Kahn has been an INFOSEC Engineer with the MITRE Corporation for six year. A graduate of the University of California at Berkeley, he did graduate work at Texas A&M University. After serving as a computer specialist for the USAF Air Weather Service, he worked for Sperry Corp in Europe for 14 years on commercial site including Shell Oil, the Swedish State Police, a Swiss Bank, and the Yugoslavian Army. He has worked INFOSEC issues for many components of the US Defense establishment for the past 15 years.