Aerospace Software

Expanding Scope of DO-178C

DO-178C now includes Auxiliary Power Units, whereas DO-178B is not specific about APU. It merely indicated "engines". But what is interesting is that DO-178C qualifies the scope as follows: "... and, by region, Auxiliary Power Units." I am aware that an Indian company has worked on a DO-178B compliant APU Controller for the B787. I therefore wonder what relevance the phrase "by region" has.

Reading DO-178C -- An Advice

The single most important piece of advice that I could give as you attempt to read DO-178C is this:
Resist the temptation to read DO-178C with a copy of DO-178B open on the side.
It may seem like a good idea to begin by comparing the changes in DO-178C against what you already know, i.e., DO-178B, but I found - very quickly - that this is also extremely boring task. And anything that is boring will never stick to your mind.
Instead read the document as if you are reading an aerospace software standard for the first time. Just flow with it. And enjoy reading it. There is lots to learn.
And yes, as I have been emphasizing in this blog, reading a standard can be enjoyable.

Change!

The eagle-eyed among you may have already detected a slight change in the Blog Header. Yeah! Time to start exploring DO-178C too. That also explains the long hiatus.
I wish to thank all those who have sent me mails demanding that I start posting again. I will start posting soon and more regularly. I promise.

Interpreting Object Code Verification

The best way to interpret paragraph 6.4.4.2.b of the DO-178B is by using an "AND" logic.

Just to be clear what I am talking about, I quote the document: "The structural coverage analysis may be performed on the Source Code, unless the software level is A and the compiler generates object code that is not directly traceable to Source Code statements."

My interpretation:

The following 3 conditions apply:

(A) Structural Coverage analysis has to be performed on Object Code,

(B) The software level is A, and

(C) Object Code cannot be directly traceable to the Source Code

A == B AND C

Meaning,

We have to provide 100% Structural Coverage on the Object Code

If

The Software Level is A

AND

The Object Code does not have a direct correspondence to the Source Code

Thus,

If the Software Level is B, C, D or E, we can do Unit Level using any Qualified Testing Tool available in the market, since the condition (B) is False. Please note that  (i) most OTS tools instrument the source code, and (ii) you have to qualify the tool for the environment you are testing in.

If the Software Level is A, but we can prove that Object Code has a one to one correspondence with the Source Code, then condition (C) is False. This happens when you have an interpreter which trranslates each line of your source code to a set of object code. (Proprietary languages have this feasture). In this case again, go ahead and use OTS tools without any problem.

However, since compiler use is more wide-spread, condition (C) is never False. It is therefore easier to just go ahead and carry out an Object Code Verification whenever the software level is A.

A word of caution: it is fine to take credit from a previous project, as long as the the environment is identical AND the constructs of the language being used is identical.

Object Code Verification is applicable even when someone claims that s/he has set the compiler option to no-optimisation. After all the idea of verification is to doubt every claim. 

Independence = Outsourcing

I have always held the view that in order to achieve high quality software development and verification should be done by two independent companies. This belief is from experience.

In the very early years of our existence we became the sole supplier for verification of a Level A software for a UK based company. The initial verification was carried out internally by our customer. I am sure there must have been misgivings on our taking over the verification work, but our customer quickly realized the value we added. We generated Problem Reports by tons. Some of the PRs were trivial but most added value. Since this was very early in the development such huge PRs did not cause any major slippage in the end date of the project.

Why did this happen? Because we had to deliver quality. And quality to a verification-focused company is: 'no item that passes our hand should have a defect". We did not care if the designer will feel bad if we generate many PRs. All we cared that the customer should not come back to us saying you did a poor job at verification. (by the way, after all these years, I am still terrified that some customer will accuse us of doing a poor job at verification; and so all our outputs are gold-plated).

However, the above view I hold is self-serving. If companies outsource verification, then some of it will definitely come our way! I was therefore pleasantly surprised when I read in CAST-26 discussion that one of three perspectives with regard to what was meant by independence is called "Organizational Verification Independence Perspective". Finally, I have a documented support to my view. According to this perspective, "This perspective proposes that to truly achieve independence, the personnel performing the verification activity (i.e., review) should actually be in a different organization (or company) than the organization which developed the data being reviewed/verified."

The CAST paper does indicate that this perspective is beyond the DO-178B objectives' intent, I do wish this becomes the standard interpretation of independence in DO-178C or D or E.

Now, how is that for wishful thinking?

CAST - Independence of Test Case & Procedures Developer and Executor

In order to meet independence objectives A6-3 (Level A and B), A6-4 (Level A), and A7-1 (Level A), CAST-26 takes the following position:

"Test Execution: The person responsible for executing the tests should not be the same person who developed the requirements or code being verified by the tests, nor the developer of the test cases and procedures being executed."

I can understand that the test executor should not be the person who wrote the requirements or the code, but cannot really understand why the developer of test cases and procedures cannot execute the test cases. There are occassions when the development of cases and procedures is separated from test execution to ensure that the requirements of gate audits or SOI audits. But I have never come across any software plan or DER insisting on having an independent executor. Is this recommendation an overkill or is there a deeper thought that went into this? I wonder.

DO-178B and Aircraft Accidents

One may claim a decade (or more) of DO-178B experience, but try speaking about aircraft and you might as well be talking to a wall. At least that has been my experience. Because getting that piece of code out of the door is so important - and exhausting - aircraft knowledge among aerospace software engineers is near zero. And this has nothing to do with the fact that most engineers working of a DO-178B project are not aerospace engineers.

Not every one will agree that knowing about aircraft is that important for a software engineer. I wonder why? I have faced similar arguments from car/mobike owners: "Just because I drive a car does not mean I need to read the maintenance manual." I wonder if similar situation exists in the the automobile industry. Would a software engineer working in the automobile domain claim that s/he couldn't care less about how a differential works? May be distributing free copies of Zen and the Art of Motorcycle Maintenance will help :)

Difference in perspective has a deep influence on the quality of work. Who do you think enjoys the work more (and as a result does quality work): the brick layer or the cathedral builder? And why do you suppose this does not apply to the aerospace software engineer? Or for that matter to the software engineer in any domain?

Now, if you are convinced that you need to learn more about aircraft, let us work out the best way of gaining such knowledge. You could start with a Flight Mechanics text book - I guarantee you will never finish that book; Flight Without Formula is perhaps an exception. 

I, on the other hand, prefer reading about accidents. After all, the whole idea behind DO-178B is to ensure that the aircraft does not suffer a catastrophic/major/minor failure. So, here is what I recommend. Visit the Aviation Safety Network. It has a listing of all known accidents since 01-Jan-1943. Browse through the accident records and pick up the ones that relate to Bowing or Airbus. Each record has a detailed description of the accident and results of the investigation. Some accidents are due to software deficiencies, like this one involving Boeing 737 in Amsterdam. Want more details? Click through to the detailed investigation report (opens a PDF): Your nerves start tingling as you read:

Not all certified Boeing 737 operating software versions for the autothrottle and flight control computers respond to an erroneous radio altitude signal in the same way. This situation is undesirable, especially in cases where an airline is using several versions that respond differently and without having informed its pilots.

As you explore the reports and google for the words you do not understand, you will see your knowledge grow leaps and bound.

By the way, the lastest air accidents - like the Qantas A380 T900 blowout - are listed too (the site is well maintained and regularly updated) but obviously do not have much details as of now.

A regular reading habit of this website will turn a 'brick layer' into a 'cathedral builder'.

Data and Control Coupling

The most confusing aspect of DO-178B the verification of Data Coupling and Control Coupling. In fact, a study of the clarification in DO248B and CAST Paper 19 leaves you even more confused. Is it a feature of good design or code review or verification testing. Clarification indicates that it is a combination of all three. That is no good. No wonder "CAST believes that additional guidance in the analyses of data coupling and control coupling will be required in the future."

This is one aspect that I would like to see resolved in DO-178C. What engineers look for actionable items; not some vague philosophical statements.

Review Checklists

Tell your quality team that checklists are not required for reviews and you are likely to get beaten up. Severely. Most people cannot look beyond checklists when it comes to review and review evidence.

Checklists are no doubt the most powerful tool know to mankind to compensate for human memory and (lack of) attention. But are they useful everywhere?

Consider this checklist point: Are the inputs correct and as per requirements? Straight forward, right? Afterall RBT demands this checklist point. But wait! Be honest and tell me if you would check this point for every test case in the document, say there are 20 test cases to be verified in the test cases and procedures document. Ideally, it should be ...

Test case 1 - checked Yes

Test case 2 - checked Yes

Test case 3 - checked No

...

Test case 20 - checked Yes

Have you ever seen a filled checklist that remotely resembles this. Normally, there would be one point and that is supposed to cover all the test cases (or whatever you are reviewing). I can understand if there is one checklist per test case, but one checklist for 20 test cases, not effective.

This is a trivial point, you would say. We do not need to look at the checklist for things that are basic. Checkling for correct inputs is one such. Then why have a checklist point? Besides, there are likely to be 15 more checklist points that you need to verify. Remember ... human memory and attention ...

That brings be to the second point. To be effective, the checklists need to be the right size. Not too big not too small. To be effective the checklists need to comprehensive. To be effective the checklists need to evolve incorporating lessons learnt from previous project. Hmmm... sort of contradicting, don't you agree? Before long, we have a 2-page checklist built on lessons learnt. Taking the example of 20 test cases, how many reviewers have you seen verifying every checklist point on the 2-page checklist for each of the 20 test cases?  The reviewers depend on memory for that. But we started with the assumption that checklists are memory compensators. Catch 22!

Let's see what DO-178B says about checklists. Refer 11.3.c(1). "Review methods, including checklists or other aids." Let's repeat it together: "or other aids" So there is life beyond checklists? Now what could DO-178B possibly mean by other aids? I am tempted to convert this into a quiz, but no, let me carry on. Have you ever considered 'Template' as an aid? And no, templates are not just a listing of contents in a pre-formatted document. Well designed templates are not only powerful aid to review but also to development. A thoughtfully developed template will have a choice of actual contents embedded within "<" and ">" to pick from. This restricts the imagination of the developer to the choices available, but guarantees compliance. I can think of the software plans developed and reviewed using good templates. all the reviewer has to ensure that the template has been followed (besides, checking if the plans make sense with respect to the project, of course, but we don't need template for that).

Hmmm.. what are the other aids that I can think of? What about tools? I would love to have a tool that scans my test scripts to get rid of silly errors. The tool can be used to parse the test script and tell me if things are ok, by and large. Of course, depending on what is being checked, the tool needs to be qualified.

I wish DO-178C would give examples instead of leaving it entirely to our imagination: like this, Review methods, including checklists or other aids, such as, templates, parsing tools, ..., etc.

What other aids do you use besides checklists for review?

 

Flying the A380

I have been travelling.

My return flight from London Heathrow to Dubai will remain with me for life. I flew back on an A380!!!

My first major project as a Project Manager was a Level A Software Development for the A380. (Unfortunately, not huge publicity is given to the fact that AK Aerotek has worked on multiple A380 systems / subsystems - the Indian newspapers, on the other hand, go hyperbolic when it comes to one of the bigger competitors ... that's PR job for you! Hmmmm... I won't deny traces of envy here.)

In any case, flying on an A380 reminded me of the Project Manager who boldly volunteered to sit on the first trial of an aircraft designed by his team, when rest of the team turned pale at the idea. His logic: he was safe since he was sure that the aircraft which his team designed wouldn't even fly.

In my case, the flight took off smoothly and was uneventful. How I wished there was someone sitting next to me to whom I could boast! (The couple sitting next to me wouldn't even talk to each other!) A380 is noticeably quieter than all other airplanes I have flown in. I could actually enjoy the movies on the seven-hour flight (I still do not have a QC2/3). All our attempts (there were 3 of us) to get an upgrade in exchange of flight miles were a failure. The flight was apparently full.

Coming back to the A380 Level A project. It was a concurrent development and everything was developed from scratch (except the sensors). It was also a distributed system. Concurrent development has its own unique set of challanges that are not addressed in DO-178B; for that matter not many software development books address concurrent development. It is worth writing a book on this. That may be because not many projects are of this type. Most projects build on pre-existing products and most projects turn into a re-engineering project. Which is a pity from an engineer's point of view.