[SystemSafety] What measures do you use (was:Koopman replies to concerns over Toyota UA case)

[Steve Tockey]

Paul E. Bennett wrote:

³Sometimes it takes no more than a brief look to know that a code sample is
not good enough to include in the final product. At other time you do need
a very definite measure as an indicator to how good or bad it might be. The
thresholds are always negotiable on a function by function basis.²

My concern is how subjective a review becomes. Different reviewers may
pass or fail the exact same code. The same reviewer may pass or fail the
exact same code on different days. The goal should be to get as much
subjectivity out of the review as possible so the developer has a ³Clear,
Concise, Correct, Coherent, Complete and Confirmable² (your words)
understanding of the minimum acceptance criteria. Likewise the reviewer(s).


³Firstly, as I have indicated elsewhere, the original requirements
specification
is where some concentration of effort is needed to reduce its complexity
before
the developer begins to use it to write the code. I have, in the past,
ripped up
a requirements specification and re-asked the client what he really
needed.²

I wholeheartedly agree.


³Measures for requirements specification I usually phrase as meeting the
six Cs.
Clear, Concise, Correct, Coherent, Complete and Confirmable. In order to
meet
those goals, the complexity of the requirements has to be managed and
whatever
techniques anyone has to do that should be deployed.²

I also claim that this means the requirements cannot be written in a
natural language, like English. UML State Chart? Sure (Right, Les?)?
English? No way.


³As an aside, one function I saw had a McCabe score of 47. However, it was
a very
simple structure that was essentially a non-nested CASE selector method
and came
with a very reasonable justification as to why it was created that way.²

Agreed. I argue, however, that CASE needs to be treated different than
IF-THEN, for exactly the reason that it is so regular in structure. I
don¹t have a definitive alternative right now, research is required on
that one. But for complexity assessment purposes, it does need to be
treated differently (not so for testing purposes, tests cases must
exercise each CASE.



‹ steve





-----Original Message-----
From: systemsafety <systemsafety-bounces at lists.techfak.uni-bielefeld.de>
on behalf of "paul_e.bennett at topmail.co.uk" <paul_e.bennett at topmail.co.uk>
Date: Sunday, December 31, 2017 at 4:45 AM
To: "systemsafety at lists.techfak.uni-bielefeld.de"
<systemsafety at lists.techfak.uni-bielefeld.de>
Subject: Re: [SystemSafety] What measures do you use (was:Koopman replies
to concerns over Toyota UA case)

On 31/12/2017 at 4:24 AM, "Steve Tockey" <Steve.Tockey at construx.com> wrote:

>Derek,
>Are you saying that there should be NO constraints whatsoever on
>the code
>a developer writes? Are you willing to accept the following
>because I have
>seen all of these with my own eyes:
>
>*) A single C++ class that had over 3400 lines of code, but all of
>that
>code was in one single method. Further, the Cyclomatic complexity
>of that
>one method was over 2400, meaning that 2 out of every 3 lines of
>code was
>a decision of some sort.
>
>*) A function that had 57 input parameters.
>
>*) A single if() statement with 36 conditions in the boolean
>expression.
>
>We may end up having to agree to disagree, but having no limits on
>structural complexity whatsoever completely removes an
>organization¹s
>ability to say ³NO!² to the kinds of schlock code that today¹s
>³highly
>paid amateur programmers² spew out every day. Are you OK with the
>current
>state of affairs in the software industry? I¹m definitely not.

Sometimes it takes no more than a brief look to know that a code sample is
not good enough to include in the final product. At other time you do need
a very definite measure as an indicator to how good or bad it might be. The
thresholds are always negotiable on a function by function basis.

>You then wrote:
>
>³I can reduce the complexity (by creating lots of smaller
>functions) and
>nobody will be any the wiser. Where does the 'missing' complexity
>go? It
>now exists in the dependency relationships between all the
>functions I
>have created; these dependency relationships are not included in
>the book
>keeping (as it exists today, somebody may invent a way of
>measuring it
>tomorrow).
>
>This is an accounting fraud.²
>
>Again, if all you ever measure is Cyclomatic complexity then yes,
>I agree.
>But that is explicitly not what I said, was it? I said that we
>need a
>collection of measures (again, ³Software complexity is not a
>number, it's
>a vector²). I didn¹t say it explicitly in my last reply, there
>need to be
>at least two categories of software structural complexity metrics:
>
>*) ³Local" complexity metrics--like Cyclomatic complexity and
>Depth of
>decision nesting, these measure the complexity inside of a single
>function.
>
>*) ³Global" complexity metrics--like Number of parameters and Fan
>out,
>these measure how functions fit into their larger environment.

Firstly, as I have indicated elsewhere, the original requirements
specification
is where some concentration of effort is needed to reduce its complexity
before
the developer begins to use it to write the code. I have, in the past,
ripped up
a requirements specification and re-asked the client what he really needed.

Measures for requirements specification I usually phrase as meeting the
six Cs.
Clear, Concise, Correct, Coherent, Complete and Confirmable. In order to
meet
those goals, the complexity of the requirements has to be managed and
whatever
techniques anyone has to do that should be deployed.

Secondly, Factoring of Functions, is a worthwhile activity. You will often
get the
benefit of having a function that can be reused many times over within the
application, and perhaps become a component of your company library. I have
the rule that all library components must have a published data-sheet
against which
full certification of conformity can be accomplished. Some of the role of
the
developer has to be selection of the most suitable existing components to
use in
producing the solution.

Finally, Refactoring is an activity that the developer performs to remove
the
complexity from the solution despite the complexity in the problem. If one
has a measure for requirements complexity and a separate measure for the
solution complexity then the ratio between the two would likely be a
useful metric
of developer performance.

I have a number of measures for my own developments.

Complexity (by McCabe: preferred maximum is 7)
Number of Issues raised at each review (there are three reviews per cycle)
Post development stage problems reported (preferred as low as possible)
Level of decision nesting (preferred maximum is 3)
Comment Quality (Six Cs compliant?)
Correlation of Function to Requirements (as revealed by Fagan Inspection)

Whatever measures you decide put in place, you should have a means by which
a test would prove them being met or not. Not met tests from my list would
require
some very intense justification (as I said, some measures have to be
negotiable on
a case by case basis). All the above are raised during reviews and would
deal with
code that was already compiled with zero warnings or errors.

As an aside, one function I saw had a McCabe score of 47. However, it was
a very
simple structure that was essentially a non-nested CASE selector method
and came
with a very reasonable justification as to why it was created that way.

Regards

Paul E. Bennett IEng MIET
Systems Engineer
Lunar Mission One Ambassador
-- 
********************************************************************
Paul E. Bennett IEng MIET.....
Forth based HIDECS Consultancy.............
Mob: +44 (0)7811-639972
Tel: +44 (0)1392-426688
Going Forth Safely ..... EBA. www.electric-boat-association.org.uk..
********************************************************************

_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE