Jerry Krasner, Ph.D., MBA
American Technology International, Inc.
EMF is the premier market intelligence and advisory firm in the embedded technology industry. Embedded technology refers to the ubiquitous class of products which use some type of processor as a controller. These products include guided missiles, radars, and avionics as well as robots, automobiles, telecom gear, and medical electronics.
Embedded Market Forecasters (EMF) is the market research division of American Technology International, Inc. EMF clients range from startups to Global 100 companies worldwide. Founded by Dr. Jerry Krasner, a recognized authority on electronics markets, product development and channel distribution, EMF is headquartered in Framingham, Mass.
About the Author
Jerry Krasner, Ph.D., MBA is Vice President of Embedded Market Forecasters and its parent company, American Technology International. A recognized authority with over 30 years of embedded industry experience, Dr. Krasner was formerly Chairman of Biomedical Engineering at Boston University, and Chairman of Electrical and Computer Engineering at Wentworth Institute of Technology and Bunker Hill Community College. In addition to his academic appointments, Dr. Krasner served as President of Biocybernetics, Inc. and CLINCO, Inc., Executive Vice President of Plasmedics, Inc. and Clinical Development Corporation, and Director of Medical Sciences for the Carnegie-Mellon Institute of Research. Earlier, he was Senior Engineer at the MIT Instrumentation Laboratory. Dr. Krasner earned BSEE and MSEE degrees from Washington University, a Ph.D. in Medical Physiology / Biophysics from Boston University and an MBA from Nichols College.
Copyright 2010 by American Technology International, Inc, 638 Main St, Ashland, MA 01721 All rights reserved. No part of this book covered by copyright hereon may be reproduced or copied in any manner whatsoever. Every effort has been made to provide accurate data. To the best of the editor’s knowledge, data is reliable and complete, but no warranty is made for this.
EMF conducts annual detailed and comprehensive surveys of embedded developers in a manner so as to maintain statistical accuracy. Between March 2009 and April 2010, EMF gathered data from 1325 developers to look at design outcomes (time-to-market, design completions ahead of or behind schedule, closeness of final design results to pre-design expectations, etc.) and to look at the relative levels of lines-of-code for different application verticals.
This data has also been used to look at project costs and design outcomes measured by comparing final design results to pre-design expectations for the software, tools, merchant boards and processors used by embedded developers.
For many military and aerospace/avionics applications, application software requires adherence to such standards as DO 178B/Level A, ARINC 653, Common Criteria and MILS, among others. Developers are exposed to a myriad of claims and counter claims. Green Hills Software, for example, touts their EAL 6+ certification as an example of their product superiority – but a reading of their certification shows a very limited coverage.
In this report, EMF data is used to determine project costs and design outcomes for several RTOSes certified to DO 178B Level A. This should certainly be of interest to OEMs and systems integrators who make OS and processor choices for their specific applications.
Recently, smaller companies have had their OSes certified under DO 178 B Level A – and it raises the question whether they offer, in addition to their intrinsic capabilities, a comparable return on investment (ROI). In this report we compare project costs and design outcomes for well known certified RTOSes (Integrity, VxWorks and LynxOS) as well as with Micrium uC/OS II, a lesser known RTOS.
EMF’s comprehensive embedded developer surveys and our unique Dashboard tool enable detailed comparisons to be made between competing DO 178B OSes.
Calculating the ROI
The following parameters are used to calculate the ROI:
a) Number of software developers per project
b) Number of months from design start to product shipment
c) Percent of designs completed behind schedule
d) Number of months behind schedule
Multiplying a) and b) provides the total average number of man-months expended in a project. Multiplying a), c) and d) provide the average number of man-months lost to schedule. Adding these calculations provides the total number of project man-months.
Table I presents a side-by-side comparison between the secure OSes from Wind River, Green Hills, LynuxWorks and Micrium. The Industrial Average is for all OSes.
|Devel time Months||13.9||12.5||17.9||16.3||13.7|
|% behind schedule||47.0%||43.1%||43.0%||51.1%||36.4%|
|Ave Delay Months||1.79||1.21||1.55||1.69||0.91|
|Average Developer months/project||204.3||76.3||272.1||231.5||168.5|
|Developer months lost to schedule||26.3||7.4||23.5||23.9||11.2|
|Total developer months/project||230.6||83.6||295.6||255.4||179.7|
|At $10K/developer month|
|Average developer cost/project||$2,043,300||$762,500||$2,720,800||$2,314,600||$1,685,100|
|Average cost to delay||$262,542||$73,615||$235,296||$239,455||$111,930|
|Total developer cost/project||$2,305,842||$836,115||$2,956,096||$2,554,055||$1,797,030|
This Table was derived from data that was based on the following number of lines of code:
- Green Hills: 1054 thousand lines of code
- LynxOS – 840 thousand lines of code
- Micrium – 829 thousand lines of code
- VxWorks – 577 thousand lines of code
- Industry average – 787 thousand lines of code
Design outcomes are another contributor to the total cost of ownership. The EMF survey asks developers to report on how close to their pre-design expectation their final design outcome was for Performance, Systems Functionality and Features & Schedule. EMF believes that acceptable design outcomes are those that are within 30% of pre-design expectation.
Table II presents the survey results for these certified RTOSes:
|Features & Schedule||79.3%||78.8%||70.2%||73.2%||81.4%|
Note that all of the certified RTOSes outperformed the industry average.
It is interesting to note that a lesser known OS compared very favorably compared to the more established DO 178B OSes. In a previous report such non-certified (but frequently used) OSes as ThreadX and Nucleus also produced more favorable results than the better known OSes.
It is clear from the presented analysis that embedded developers looking for a certified RTOS take into account the design particulars of their applications and the compute power required. EMF hopes that this analysis will help developers to look at certified software from a different perspective.