[SystemSafety] A Fire Code for Software?

[Steve Tockey]

Les wrote:

“But let us not despair. Looking to the future, my preferred metaphor is: software as literature; where ideas of permanent and universal interest are expressed with enlightened meaning and form; creating something of value that endures - authored by software developers who get it substantially right on
the first draft.”

Have you had a chance to read Donald Knuth’s “Literate Programming”? As written, it’s a bit out of date being primarily paper-based. On the other hand, read it in light of Hypertext technology and a new, very interesting capability emerges. I have a brief discussion of it in Chapter 22 of the new book, lines 138 through 363.

As well, one of the guiding philosophies in the new book is, “… change the nature of programming from a private, puzzle solving activity to a public, mathematics based activity of translating specifications into programs … that can be expected to both run and do the right thing with little or no debugging”



“From my experience on the management team of a fixed price software house my paper offers 24 searching questions that may help you smoke out the "real" software engineers in a job interview. For each question I explore the answers most frequently offered by the craftsmen and the "right" answers you will often get from a software engineer.”

That’s a pretty good set of questions. Having not read your paper yet, I thought I would take a stab at answering them:

1. What do you feel is your mission when you set out to capture user requirements?

To understand, as precisely and completely as possible, what their real need is (both spoken and unspoken).

2. How do you solve wicked problems?

First, decompose a large system into distinct “domains”, and then—as needed—further decompose those into distinct subdomains. Abstraction, high cohesion, loose coupling, etc. are guiding principles in that decomposition. Second, for each (sub-) domain get as much agreement as possible on the 'functional requirements’ through building a semantic model. If prototyping the (user) interface before building a semantic model is needed, that is acceptable.

To the extent the above can be done in a sequential fashion, try to do so because it helps minimize project cost by reducing overhead and rework. To the extent that it needs to be done incrementally, do so (aka, “agile”). There is no one-size-fits-all process, use a process that best meets the needs of that specific project.

3. How do you make design decisions?

The two primary consideration in design are 1) preserving the policy and process semantics in the agreed-to semantic model, and 2) satisfying the 'non-functional’ requirements economically.

4. How do you go about innovating?

By applying ‘Fudd's First Law of Creativity': Come up with as many ideas as you can, then throw away the bad ones

5. Can you describe your creative design process?

It is a combination of my answers to Q3 and Q4, namely considering many different semantics-preserving mapping strategies.

6. What is the secret to developing great software?

It should not be a secret. Somebody should write a book on that. Oh, wait . . .   :^)

7. What is your view on the role of documentation in software projects?

There is no such thing as self-documenting code. I believe only in 'self-coding documentation'.

8. Do you try to reuse code?

No, not really. I focus mainly on reusing all or parts of semantic models (typically 60% of a project’s technical effort). Then I focus on reusing mapping strategies (typically 30% of the technical effort). The actual code is very highly sensitive to a large number of project- and environment-specific characteristics and tends to be not very reusable (and it’s only 10% of the technical effort anyway, so why bother?).

9. How you feel about working in a team?

For anything beyond a trivial project, teamwork is essential. Besides, the other team members are an incredibly valuable resource in peer reviews.

10. The fox knows many little things, but the hedgehog knows one big thing.
- Archilochus (Greek poet, 680 BCE - 645 BCE)
Which do you think is the best approach, specialisation (the hedgehog) or having a broad knowledge
of your application domain (the fox)?

Any non-trivial project needs both: some who can see the big picture across the project, and others that are highly focused on specific parts.

11. How do you recognise and manage risk?

I start with a generic risks list and customize it to that one specific project, keeping in mind that I need to involve most (if not all) of the team in ongoing risk identification and analysis. Prioritization and control follow straightfoward approaches. The only other twist is to continue actively applying risk management throughout the project (I.e., it’s not a one-time deal only at the beginning, it is re-done multiple times as the project progresses).

12. If your software has the potential to harm life and property, how you make it safe?

By following a process that has been designed to a) avoid defects in the first place, and b) make unavoidable defects as visible as possible as soon as possible. To the extent that there are existing safety requirements (both in terms of product- and process-), make sure to take those into account.

13. Is software security something we should be concerned about? If so, how do you secure a software
product?

We absolutely need to be concerned about security in the vast majority of software. We can make systems secure by a) risk-based application of the ‘Web Applications Security Consortium’s Threat Catalog’, and b) frequent and appropriate peer review of requirements, design, and code to help avoid the stupid mistakes that lead to most security flaws (like buffer overrun).

14. When writing software, how do you manage your projects? Is planning a valuable exercise?

Planning is invaluable. How any one specific project is planned and managed depends on key attributes of that project, like team size, team co-location or not, how knowable & stable the requirements are, whether or not the stakeholders get value out of early delivery of partial functionality, etc . It won’t be the same every time. I’m a firm believer in ‘rolling wave planning’ (aka ‘progressive elaboration’). You don’t need to—and often can’t—plan out the entire project at the very beginning. Have fine-grained plans to manage near-term work, more coarse-grained plans are OK for work that is farther in the future. Just make sure to do the necessary fine-grained planning as the future work approaches the present. The subtext is, ‘Agile development needs to be seen as simply rolling wave planning on steroids’.

15. Is formal scope definition useful? If so, how do you go about defining the scope and extent of your
software product?

Yes, very useful if it can be agreed on up-front. We would first get general agreement through a ‘Project Charter’. Later agreement can be (but not necessarily) based on (sub-) domain level use case specifications. If it can’t be agreed on up-front, then use a more open-ended process (aka rolling wave planning) and adjust on the fly as more becomes known.

16. Are you happy to commit to an up-front cost? How do you estimate software costs?

On day one? No, not at all. But by 10-15% of the way into the project we can predict costs to within about +/-10%. Costs are best estimated as derived from counts of the number of classes in the semantic models of each of the (sub-) domains.

17. Are you happy to commit to a deadline for software delivery?

Insofar as cost is derived from an estimate of project effort, and insofar as schedule is largely also derived from that same effort, then when I am in a position to commit to cost I am also willing to commit to any reasonable schedule that is consistent with the effort and the staffing profile.

18. Are you an efficient software developer?

I believe so, yes.

19. How productive are you?

My productivity rate is on the order of 7 person-days per class in a semantic model, plus or minus about 10%.

20. What is software quality and how do you deliver it?

That's not a simple question, so it doesn’t have a simple answer. We need to consider not only product quality, we also need to consider process quality. Product quality depends on getting appropriate stakeholder involvement in agreeing on both ‘functional' and 'non-functional' requirements. Functional requirements are captured in one or more semantic models. Non-functional requirements are based on applying (a possibly customized version of) ISO 25010.  Process quality comes from applying a time-tested approach that has been designed to a) avoid defects in the first place, b) expose as many unavoidable defects as early as possible, and c) refining and tuning the time-tested process as the project progresses.

21. How did you learn to develop software?

It took many separate steps: 1) self-taught in high school (1975), 2) BA in Computer Science from UC Berkeley (while paying the bills working as a developer, 1981), 3) Masters in Software Engineering from Seattle University (1993), 4) working for a series of companies that all encourage continuing education, and 5) seeing what really does & does not work on actual projects and learning from that.

22. How do you know that your work will delight your customer?

Because the customer is as involved in the project as I can get them to be, while respecting their need to get their own job done.

23. How you handle maintenance after your software is delivered?

See Q7 above. To the extent that any maintenance is needed, it must be driven by changing the documentation first to have a full understanding of what code changes might be needed as a result of the necessary documentation changes.

24. Are your actions guided by any particular moral code? Do you feel responsible for how your systems
are used?

Yes. If not me—the one who built it—then who?


How did I do? Would you consider hiring me?



Cheers,

— steve



From: Les Chambers <les at chambers.com.au<mailto:les at chambers.com.au>>
Date: Saturday, March 10, 2018 at 7:15 PM
To: Steve Tockey <Steve.Tockey at construx.com<mailto:Steve.Tockey at construx.com>>
Cc: "systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>" <systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>>
Subject: RE: [SystemSafety] A Fire Code for Software?

Steve
I'm with you Steve; our descendants will look back on much of today's software as over-priced pulp fiction; poor quality, untrimmed and ragged; redolent with crime, science-fiction, fantasy, romance and horror - authored by amateurs.
But let us not despair. Looking to the future, my preferred metaphor is: software as literature; where ideas of permanent and universal interest are expressed with enlightened meaning and form; creating something of value that endures - authored by software developers who get it substantially right on
the first draft.

A while ago I gave this subject some serious thought, specifically what separates the software engineer from the other ... commonly known as software craftsmen.
Definition (?) of software engineering:
Applying a systematic, disciplined, quantifiable approach to concieving, developing, operating and maintaining software intensive systems.
More at:
http://www.chambers.com.au/glossary/software_engineering.php
In this paper I examine the discipline of software engineering, from its definition in the literature to the belief systems that make it work. I examine the attitudes of mind it embraces (and the ones it discards), how intuition must give way to formalism and how upfront analysis must replace expensive downstream trial and error. Put simply, how we must think more before we act.

Much of this paper is given over to the current imperfect situation where the quality of software is almost wholly dependent on the individual and collective skills of the project team - I cite Woody Allen who commented, "95 percent of successful moviemaking is in casting" - one must resort to tricky questions in job interviews. From my experience on the management team of a fixed price software house my paper offers 24 searching questions that may help you smoke out the "real" software engineers in a job interview. For each question I explore the answers most frequently offered by the craftsmen and the "right" answers you will often get from a software engineer.

Summary (Refer paper, chapter 9):
1. What do you feel is your mission when you set out to capture user requirements?
2. How do you solve wicked problems?
3. How do you make design decisions?
4. How do you go about innovating?
5. Can you describe your creative design process?
6. What is the secret to developing great software?
7. What is your view on the role of documentation in software projects?
8. Do you try to reuse code?
9. How you feel about working in a team?
10. The fox knows many little things, but the hedgehog knows one big thing.
- Archilochus (Greek poet, 680 BCE - 645 BCE)
Which do you think is the best approach, specialisation (the hedgehog) or having a broad knowledge
of your application domain (the fox)?
11. How do you recognise and manage risk?
12. If your software has the potential to harm life and property, how you make it safe?
13. Is software security something we should be concerned about? If so, how do you secure a software
product?
14. When writing software, how do you manage your projects? Is planning a valuable exercise?
15. Is formal scope definition useful? If so, how do you go about defining the scope and extent of your
software product?
16. Are you happy to commit to an up-front cost? How do you estimate software costs?
17. Are you happy to commit to a deadline for software delivery?
18. Are you an efficient software developer?
19. How productive are you?
20. What is software quality and how do you deliver it?
21. How did you learn to develop software?
22. How do you know that your work will delight your customer?
23. How you handle maintenance after your software is delivered?
24. Are your actions guided by any particular moral code? Do you feel responsible for how your systems
are used?

The Way Out of the Crisis
The level of formality we use to build a system must increase in proportion to its size.
We have already reached the stage where human beings are not capable of manually (by intuition) building and validating a large system that is correct by design. There is too much variation in attitudes and skill levels in human beings to guarantee a predictable outcome.

My solution is to do what humanity has always done to deal with largeness and complexity where consistency of outcome is critical. We increase the level of formality in what we are doing and support construction with tools. So my response to those who complain about "plug and pray" is, "try harder because it is the only way out."

 I'd encourage people currently working on yet another industry standard to accept that we have enough of them, what we need is better tools to assist with compliance. Industry standards drive culture, tools ensure that cultural imperatives are actually realised in end products.

Les

From: systemsafety [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of Steve Tockey
Sent: Friday, March 9, 2018 5:01 AM
To: Andrew Banks; 'Andy Ashworth'
Cc: systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>
Subject: Re: [SystemSafety] A Fire Code for Software?


Andrew,
I agree 100%. But I would actually take it even farther.

I am already on record as having said (referring to the software industry as a whole),

“We are an industry of highly paid amateurs”

Claiming that one is an engineer simply because the words “software engineer” are printed on one’s business card are simply not sufficient. I strongly recommend that we start a parallel effort to the recent “don’t call them bugs, call them defects” movement (Are you paying attention, Chris Hills?). In this nw movement, anyone who uses the term “software engineer” is required to:

A) provide a reference to a definition of the term “engineer(ing)” that has been accepted by already-recognized engineers (e.g., Civil, Chemical, Mechanical, Industrial, . . .)

B) Show how what they are doing on a day-to-day basis on their projects is consistent with that legitimate engineer-accepted definition

The vast majority of people in this industry can’t do either A) nor B).


What happens most of the time in the software industry is no better than what I call “Resume Driven Development”. Major technical decisions are not made in the best interest of the organization, they are made based on what looks good on the developer’s resume.


Regarding this so-called “paradigm shift to agile methods”, I claim that if you really understand what “agile” means then you recognize that it is nothing more and nothing less than a project management paradigm. And a project management paradigm—any, including waterfall—alone is not sufficient to either warrant calling “engineering” or assure delivery of reasonable software at a reasonable cost and schedule. A true engineer would know when a waterfall project management paradigm is and is not appropriate. A true engineer would know when an agile project management paradigm is and is not appropriate. A true engineer would make a decision on which project management paradigm to use based on what provides the best return on investment for the organization, and not what looks sexy on their resume. A true engineer would also know that a project management paradigm alone is not sufficient to assure delivery of  high-quality software in a cost-effective way. Delivering high-quality software cost-effectively requires:

*) a sane and rational approach to how the project is being managed
*) a sane and rational approach to how requirements, design, and construction (coding) work is being done on the project
*) a sane and rational approach to how quality work (inspections and/or other peer reviews, as well as testing) is being done on the project


My second book is not titled “How to Engineer Software” by accident.



Cheers,

— steve




From: Andrew Banks <andrew at andrewbanks.com<mailto:andrew at andrewbanks.com>>
Date: Wednesday, March 7, 2018 at 10:12 PM
To: Steve Tockey <Steve.Tockey at construx.com<mailto:Steve.Tockey at construx.com>>, 'Andy Ashworth' <andy at the-ashworths.org<mailto:andy at the-ashworths.org>>
Cc: "systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>" <systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>>
Subject: RE: [SystemSafety] A Fire Code for Software?

Hi Steve

And here is the rub:

>> My definition of “model based” involves creating and maintaining precise specifications of semantics:
>> policies that need to be enforced and processes that need to be carried out.

It is the absence of this up-front work that is so prevalent in software (and systems-) engineering… even in formal development environments, engineers need to “get on with it” and let the requirements catch up.  Then throw in the paradigm shift to more Agile methods and it gets even more unpredictable.

But The Authorities seem to not care: Eg in the automotive world, despite standards such as ISO 26262 there is no statutory requirement to follow a formal development process… only “conformity of production” matters – and the type approval process doesn’t even mention the existence of software (or involve any checking of how it came into being), and just concerns itself with the physical characteristics of the vehicle.

Compare with civil engineering, where the detailed plans form part of the planning process, and implementation is controlled by strict building regulations, and independently monitored – and all components have to comply with appropriate standards.


Regards
Andrew


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20180311/209e70dd/attachment-0001.html>