DOT/FAA/TC-xx/xxFederal Aviation AdministrationWilliam J. Hughes Technical CenterAviation Research DivisionAtlantic City International AirportNew Jersey 08405Understanding the OverarchingProperties: First StepsSeptember 2018Final Report (Limited Release)This document is currently under publicationreview and will be available to the U.S. publicthrough the National Technical InformationServices (NTIS), Springfield, Virginia 22161.U.S. Department of TransportationNOT FAA POLICY OR GUIDANCE - LIMITED RELEASE DOCUMENT
Federal Aviation AdministrationNOT FAA POLICY OR GUIDANCE - LIMITED RELEASE DOCUMENT
NOTICEThis document is disseminated under the sponsorship of the U.S.Department of Transportation in the interest of information exchange. TheUnited States Government assumes no liability for the contents or usethereof. The United States Government does not endorse products ormanufacturers. Trade or manufacturer's names appear herein solelybecause they are considered essential to the objective of this report. Thisdocument does not constitute FAA certification policy. Consult your localFAA aircraft certification office as to its use.Disclaimer: This document is a draft deliverable of the research. This draftreport is being made available as a “Limited Release” document by the FAASoftware and Digital Systems (SDS) Program and does not constitute FAApolicy or guidance. This document is being distributed to selectedorganizations only with express written permission by the ContractingOfficer’s Technical Representative (COTR). The research information inthis document represents only the viewpoint of authors.The FAA is concerned that its research does not get released to outsideFAA organizations before proper and full review is completed. However, aLimited Release distribution under select conditions does allow immediateexchange of research knowledge in a way that will benefit the partiesreceiving the documentation and, at the same time, not damage perceptionsabout the quality of FAA research. When the FAA releases such researchdocumentation, the FAA strives to insure that the receiver knows that thedocumentation is incomplete, limited in distribution, and should not furtherdistribute without the express written permission by the COTR.
This report will be available at the Federal Aviation Administration WilliamJ. Hughes Technical Center’s Full-Text Technical Reports page:actlibrary.act.faa.gov in Adobe Acrobat portable document format (PDF).
Technical Report Documentation Page1. Report No.2. Government Accession No.3. Recipient's Catalog No.DOT/FAA/TC-xx/xx4. Title and Subtitle5. Report DateUnderstanding the Overarching Properties: First StepsSeptember 2018 (draft Pub)6. Performing Organization Code7. Author(s)8. Performing Organization Report No.C. Michael Holloway9. Performing Organization Name and Address10. Work Unit No. (TRAIS)NASA Langley Research Center, 100 NASA Road, Hampton VA 2368111. Contract or Grant No.IAI-140712. Sponsoring Agency Name and Address13. Type of Report and Period CoveredDraft report, under publicationreview, limited releaseFederal Aviation AdministrationWilliam J. Hughes Technical CenterAviation Research DivisionAtlantic City International Airport, NJ 0840514. Sponsoring Agency CodeBarbara Lingberg, AIR-6B415. Supplementary NotesThe FAA William J. Hughes Technical Center Aviation Research Division Technical Monitors were John Zvanyaand Srini Mandalapu.16. AbstractThe Overarching Properties are the product of a multi-year, international effort to develop a minimum set of properties sufficientfor use in the approval process. In other words, if an entity for which approval is sought is shown to possess these properties intheir entirety, then granting approval for the entity to be used on an aircraft is warranted. The work is not finished.This report explains the Overarching Properties as they are currently constituted, including the philosophical foundationunderlying them, the specific text and meaning of each property, and the relationships the properties have to each other and totime. The report also discusses the remaining issues that must to be resolved in the future.NASA Langley Research Center’s participation in the effort was supported in substantial part through an annex, “StreamliningAssurance Processes”, to a Reimbursable Interagency Agreement (Numbered IA-1407 by NASA and DTFAWA-14-C-00019by the FAA), “Enhancement of Aeronautical Research and Technology Development”. C. Michael Holloway was the primaryNASA person conducting the work, with occasional assistance from Patrick Graydon.17. Key Words18. Distribution StatementApproval, assurance, certification, properties, philosophy19. Security Classif. (of this report)UnclassifiedForm DOT F 1700.7This document will be available to the U.S. public throughthe National Technical Information Service (NTIS),Springfield, Virginia 22161. This document will also bealso available from the FAA William J. Hughes TechnicalCenter at actlibrary.tc.faa.gov. Limited release.20. Security Classif. (of this page)21. No. of PagesUnclassified(8-72)Reproduction of completed page authorized(The document is paginated assuming 2-sided printing. Hence, some blank pages are included to provide a back side where needed.)22. Price
ACKNOWLEDGEMENTSMike DeWalt (the retired Chief Scientist and Technical Advisor for Software) motivated the effortresulting in the Overarching Properties. Barbara Lingberg and George Romanski (the current ChiefScientist and Technical Advisor for Software) kept it going after Mike left the scene. SriniMandalapu provided program management support for the Interagency Agreement which partiallyfunded NASA Langley Research Center’s involvement throughout. Thanks to all of you.i
TABLE OF CONTENTSPageEXECUTIVE SUMMARYvii1. PRELUDE11.1 Brief history1.2 Presentation style122. PHILOSOPHY33. PROPERTIES53.1 Statements3.1.1 Intent3.1.2 Correctness3.1.3 Acceptability3.1.4 Colloquial summary3.1.5 Relationship to each other3.1.6 Relationship to time3.2 Prerequisites3.3 Assumptions3.4 Constraints789101111121314154. PRACTICALITIES174.1 Supplant or supplement?4.2 Who has DIBS?4.3 Does sufficiency even matter?4.4 Can anything go?181819205. POSTLUDE216. REFERENCES21iii
LIST OF FIGURESFigurePageFIGURE 1: THE OVERARCHING PROPERTIES6iv
LIST OF TABLESTablePageNone.v
LIST OF ACRONYMSACEassurance case evaluationALEapproved list evaluationAPEapplicant process evaluationAVEapplicant varying evaluationDeBdesired behaviorDiBdesired intended behaviorEUROCAEEuropean Organisation for Civil Aviation EquipmentFAAFederal Aviation AdministrationNASANational Aeronautics and Space AdministrationOPsOverarching Propertiesvi
EXECUTIVE SUMMARYThe Overarching Properties are the product of a multi-year, international effort to develop aminimum set of properties sufficient for use in the approval process. In other words, if an entityfor which approval is sought is shown to possess these properties in their entirety, then grantingapproval for the entity to be used on an aircraft is warranted. The work is not finished.This report explains the Overarching Properties as they are currently constituted, including thephilosophical foundation underlying them, the specific text and meaning of each property, and therelationships the properties have to each other and to time. The report also discusses the remainingissues that must to be resolved in the future.NASA Langley Research Center’s participation in the effort was supported in substantial partthrough an annex, “Streamlining Assurance Processes”, to a Reimbursable Interagency Agreement(Numbered IA-1407 by NASA and DTFAWA-14-C-00019 by the FAA), “Enhancement ofAeronautical Research and Technology Development”. C. Michael Holloway was the primaryNASA person conducting the work, with occasional, always valuable, assistance from MalloryGraydon.vii
1. PRELUDEThe purpose of the Overarching Properties is to constitute a set of properties that are sufficient towarrant receiving approval for use on aircraft. That is, if an entity for which approval is soughtpossesses1 these properties in their entirety, then granting approval is appropriate. They are calledproperties because they encapsulate the “characteristic qualities”  that a product must have tojustify approval. They are called overarching because they “encompass all”  of the necessaryproperties.The purpose of this document is to explain the Overarching Properties 2 as they currently exist,including their philosophical foundation, the specific details of each property, the relationshipsamong them, and some practical considerations that attach to their use. Readers of this documentare assumed to be at least somewhat familiar with current laws, regulations, and processesgoverning certification of airborne systems, software, and electronic hardware. Because theOverarching Properties are expressed at a much higher level of abstraction than is common today,however, readers without intimate knowledge of current practice may find understanding theOverarching Properties easier than readers with such knowledge. Readers of the document are alsoassumed to be aware that what is described herein is a work still in progress.The document’s structure is as follows. The remainder of this introduction presents somebackground information. §2 explains the philosophy underlying the Overarching Properties. TheOPs themselves are then explained in detail in §3. Comments about issues that may arise in practicewhen the OPs are used are made in §4. The document concludes in §5 with brief speculativeremarks about the future of the OPs.1.1 BRIEF HISTORYThat which are now called the Overarching Properties originated in a workshop in December 2015.The workshop was sponsored by the Federal Aviation Administration (FAA), who selected theinvitees to this workshop, seeking to ensure industry and governmental participation from acrossa wide area of technical disciplines, countries, and assurance viewpoints. The effort continued withtwo more invitation-only meetings in April and July 2016, periodic virtual meetings, and an onlineforum, resulting in a set of three Overarching Properties.These OPs were presented to the public in September 13-15, 2016 at the 2016 FAA StreamliningAssurance Processes Workshop in Richardson, Texas. The Overarching Properties work was onlyone of the activities discussed, along with the other ongoing activities collected under the“streamlining assurance processes” banner. A handout containing the Overarching Properties was1 Two notes are appropriate here. First, henceforth for simplicity of expression the word product will be used as a shorthand for “an entityfor which approval is sought.” Second, the use of the word possessed instead of satisfied may strike some readers as odd. It is common in somecircles to talk of ‘satisfying’ properties; such usage cannot be deemed wrong, but ‘conditions’ are better said to be ‘satisfied’ and ‘properties’ tobe ‘possessed’.2 The abbreviation OPs (pronounced “oh-peas”) will be used in place of the full phrase from time to time, but not always, as it seemsinappropriate in some sentences.
distributed to attendees without any additional printed explanatory material. To supplement thewritten material, oral presentations were delivered and several discussion sessions held.Most workshop participants who expressed opinions about the OPs were favorable to the ideas asthey understood them, with the level of enthusiasm ranging from tepid to euphoric. In the categoryof less-than-favorable comments, some participants expressed confusion over how theOverarching Properties work fits in with the other streamlining activities. In particular, theseparticipants doubted that adopting a new certification regime based on the OPs would immediately,or perhaps ever, result in faster or cheaper certification. The response to these concerns was, andcontinues to be, that the FAA’s streamlining activities are not just about reduced cost and time,but also about increasing flexibility without compromising safety, which is the emphasisunderlying the Overarching Properties effort.Other less-than-favorable comments came from participants who indicated a desire forsubstantially more details about how the OPs might be used in practice, especially questioning thefeasibility of evaluating whether a product possesses the properties. Finally, the question wasraised by a few attendees of whether the OPs as written were complete. The response to commentsof these type was to acknowledge much work remained to be done.To accomplish this remaining work, virtual meetings and forum activity continued through theremainder of 2016, resulting in some relatively minor changes to the OPs. In early 2017 the teamwas dubbed the Overarching Properties Working Group (OPWG). New people joined the team,and some original team members left. Three physical meetings were held in 2017 (February, May,and September), with continuing virtual meetings throughout the year. The emphasis of the effortin 2017 was on addressing the question of evaluation. The evaluation approach pursued during thistime involved the creation of a set of criteria. At first these criteria were considered to be a meansby which assessors could determine whether a product possesses the OPs. Later, a switch inemphasis led to the criteria being considered as the set against which may be assessed thesufficiency of proposed processes for ensuring that a product possesses the OPs. Do not worry ifthis distinction seems unclear at this point; the issue of evaluation is discussed in more detail in§4.4.The version of the Overarching Properties described in this document was finished during aphysical meeting in February 2018, and refined slightly through July 2018. The changes from theversion presented at the public workshop are mostly not substantial but rather subtle or editorial.The change in format from three separa