[SystemSafety] Does "reliable" mean "safe" and or "secure" or neither?
Dewi Daniels
dewi.daniels at software-safety.com
Thu Apr 28 14:54:35 CEST 2016
Mike,
I expect Peter Ladkin will be along shortly to satisfy your expectations.
Dewi
On 28 April 2016 at 13:51, Mike Ellims <michael.ellims at tesco.net> wrote:
> Good afternoon Michael,
>
> Well that's a bit of a damp squib then, I was expecting a fiery return
> comment!
>
> Cheers.
>
> -----Original Message-----
> From: systemsafety
> [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of
> Michael J. Pont
> Sent: 27 April 2016 18:41
> To: 'The System Safety List'
> Subject: Re: [SystemSafety] Does "reliable" mean "safe" and or "secure" or
> neither?
>
> Mike (Ellims),
>
> Thanks for your reply.
>
> My original email was intended to make two points, the first of which C.
> Michael Holloway expressed much more clearly and succinctly than I did in a
> later post:
>
> "One would surely expect professional engineers to be less cavalier in
> frequently adding new definitions to common words, which change the
> meanings
> in confusing, unhelpful ways."
>
> I (still) believe that this is what has happened here in the application of
> the phrase reliability to software.
>
> I'm happy to accept your argument that - when following particular
> standards
> - we may be obliged to accept the definitions that are associated with
> these
> standards (but my original point was intended to be more general).
>
> Michael.
>
> -----Original Message-----
> From: Mike Ellims [mailto:michael.ellims at tesco.net]
> Sent: 25 April 2016 13:25
> To: M.Pont at SafeTTy.net; 'The System Safety List'
> <systemsafety at lists.techfak.uni-bielefeld.de>
> Subject: RE: [SystemSafety] Does "reliable" mean "safe" and or "secure" or
> neither?
>
>
> Good afternoon Michael,
>
> First sorry about the length of the reply and the delay!
>
> >>
> >> We can choose to define any terms that we use in a particular domain
> >> in
> any way that we like.
> >>
>
> First who is we?
>
> Second, no we can't; to maintain consistency with existing standards,
> practice and research we have to maintain constancy with existing
> definitions, for example IEC 61508 Part 4 states "see IEV 191-12-01 for a
> definition of reliability".
>
> IEV 191-12-01 defines reliability as, "probability that an item will
> perform
> a required function under given conditions for a given time interval
> (t1.t2)". ISO 26262 doesn't give a definition of reliability but as it
> purports to be a sector specific derivative of IEC 61508 I assume it will
> inherit definitions not otherwise defined (does anyone have an answer to
> that?).
>
> >>
> >> Where these terms conflict with general use of the same word, we make
> >> it
> more likely that people reading the
> >> documents - etc - will be confused (in my view).
> >>
>
> Which is why standard definitions such as that given in BS 4478 and IEV
> 191-12-01 exist because there is a disparity between what the man in the
> street means when they say reliable and what reliability engineers have
> decided that term reliable means. The same problem exists with the use of
> the word theory, in science is it a reasonably precise meaning and
> scientists have other words such as conjecture and hypothesis for concepts
> that are not well established. However in common use the word theory has a
> different less precise meaning and in general represents a plausible
> scenario as may be argued in court of law.
>
> >>
> >>Confusion tends to result in Bad Things Happening in this business.
> >>We
> (therefore) want to avoid confusion.
> >>
>
> Which is why we have an exist precise definition... actually I'll concede
> that as stated it's not actually very precise i.e. it doesn’t specify how
> to
> perform the statistical evaluation buy that’s expanded on in textbooks such
> as O'Conner and IEC 61508 etc. Note this is not an argument for the
> usefulness of the term as applied for software it's only an argument
> against
> redefining the meaning of the term. In general I would argue that if we
> have
> a problem with the way a term is defined then we should find a similar term
> that could be explicitly defined to mean what we think is necessary. So
> rather than try and redefine "software reliability" or "reliability" we
> should use a different term.
>
> >>
> >> Two seconds on Google gives me this definition of reliability:
> >>
>
> Sorry but I don't think that appeal to Google is a reasonable approach.
> Google can be used as a mechanism for locating specific information in a
> field of study e.g. reliability engineering. For example if I put the
> search
> term "reliability" into Google the first entry is a link to the Wikipedia
> disambiguation page that lists a set specialist used for the term
> reliability (https://en.wikipedia.org/wiki/Reliability) the second of
> which
> is reliability engineering. The second sentence of the entry for
> reliability
> engineering gives the following definition "Dependability, or reliability,
> describes the ability of a system or component to function under stated
> conditions for a specified period of time" which is cross referenced to
> IEEE
> Standard Computer Dictionary: A Compilation of IEEE Standard Computer
> Glossaries.
>
> Various generic dictionaries produce the following set of definitions in
> order of appearance;
>
> dictionary.com : the ability to be relied on or depended on, as
> for
> accuracy, honesty, or achievement.
> merriam-webster.com : 1. the quality or state of being reliable 2:
> the
> extent to which an experiment, test,
> or measuring procedure yields the same results on
> repeated trials
> thefreedictionary.com : The ability of an item to perform a required
> function under stated conditions for a
> specified period of time.
> en.wiktionary.org : 1. The quality of being reliable, dependable or
> trustworthy. 2. In education - the ability to
> measure the same thing consistently (of a
> measurement indicating the degree to which the
> measure is consistent); that is, repeated
> measurements would give the same result (See also
> validity). 3. In engineering - measurable time of
> work before failure
>
> Several definitions are a close match the IEC and IEEE definition and the
> last makes a clear distinction as to where different definition apply.
> However there is wide variation and taking this approach it's pot-luck
> whether any particular person would derive anything useful or usable. For
> example the Concise OED has no separate definition for reliability and the
> Oxford Dictionary of Computing does and it matches the IEC/IEE/BS standard.
>
> Note that if we search for the term "software reliability" we usually get
> something more akin the standard meaning as used in reliability engineering
> e.g. "A software quality aspect that is measured in terms of mean time to
> failure or failure intensity of the software", or from ANSI " the
> probability of failure-free software operation for a specified period of
> time in a specified environment".
>
> >> "Definition of reliability. 1 : the quality or state of being
> >> reliable. 2
> :
> >> the extent to which an experiment, test, or measuring procedure
> >> yields
> the same results on repeated trials."
> >>
> >> I see "2" as the useful (and general) definition here. It may even
> >> be
> compatible with O'Conner.
>
> I don't think that this is compatible with O-Conner nor the standard
> definitions of engineering reliability, and to my mind the second
> definition
> would seem to be a closer match to testing or program corretness.
>
> As an extreme contrived counter example to the usefulness of definition 2
> we
> could consider the case of a software system that always halted 1/2 a
> second
> after start producing no output. We could run this any number of times and
> get the same output (whatever the input) and meet the criteria above that
> repeated trails produced the same output. Thus the software system is
> reliable. Useless but reliably useless... which is perhaps not useful.
>
> >>
> >> My issue is that software (whether good or bad) doesn't change over
> time.
> >>
>
> This statement uses the starting position that all reliability issues are a
> result of wear out, i.e. X changes over time.
>
> My original point was that reliability engineering is NOT just about wear
> out. An object (mechanical, electric, software) can fail to deliver the
> specified function for a number of different reasons. A simplified
> mechanical example may help, consider if a part were made from aluminum as
> specified, rather than stainless steel as was intended (i.e. we have a
> specification error) then the part could fail more quickly than desired for
> several reasons;
>
> i) it was not strong enough,
> ii) it melted,
> iii) it suffered badly from corrosion,
> iv) or because it wore more quickly.
>
> Reliability isn't only about "wear-out"; as defined it's "about failure to
> deliver functions", whether that function be ability of hold a load,
> support
> a beam, maintain a current, measure a value, monitor a parameter or to
> perform a calculation. In addition you can stretch the analogy of wear out
> to Niklaus Wirth's concept of accumulated state. That is the longer the
> program runs the more (possibly redundant) state information builds up
> within in memory, the more probable the system is to go bonk (important
> technical term). The patriot missile system springs to mind as a simple
> example of catastrophic state accumulation and while possibly reliable when
> used with the assumed timeframe (t1,t2) it proved unreliable the used in
> the
> timeframe (t1,t3) where t3 >> t2.
>
> >>
> >> Talking about "software reliability" (therefore) doesn't make sense
> >> and
> is likely to lead to confusion.
> >> (Hardware reliability is fine; System reliability is also fine, where
> System = Hardware + Software.)
> >>
>
> At the current time, given the limits associated with research in software
> reliability prediction that is not that an unreasonable position to take in
> practical terms. However as stated above I don't think that gives a green
> light to appropriate the term reliability.
>
> As a topic of theoretic interest software reliability is a valid area of
> research and some of the concepts have at least some utility if the
> limitations of the techniques are taken into account. For example
> reliability growth theory indicates that over time with a good development
> process the rate of failures observed should decline; so it isn't then we
> are in an effluent stream with no propulsive device (grossly simplified of
> course). There is also practical application to the evaluation of systems
> fielded and to comparisons between different software running on identical
> platforms.
>
> In summary:
> 1. Reliability is a existing defined term, even for software.
> 2. For software the term may have little practical predictive power,
> however
> it can't just be redefined.
> 3. If necessary a new term should be defined that captures the idea of
> reliability in a manner that matches common usage.
> 4. I suspect 3 could be really, really, really hard.
>
> Please feel free to take pot-shots.
>
> Cheers.
>
>
> -----Original Message-----
> From: systemsafety
> [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of
> Mike Ellims
> Sent: 24 April 2016 13:23
> To: 'Coq, Thierry' <Thierry.Coq at dnvgl.com>; 'The System Safety List'
> <systemsafety at lists.techfak.uni-bielefeld.de>
> Subject: Re: [SystemSafety] Does "reliable" mean "safe" and or "secure" or
> neither?
>
> > as software does not "fatigue" or randomly "break" the way hardware
> > does
>
> Reliability engineering is much more than just fatigue or randomly breaks,
> it encompasses everything about the control of variation and error in a
> system, most commonly using statistical methods.
>
> BS 4478 defines reliability as "The ability of an item to perform a
> required
> function under stated conditions for a stated period of time".
>
> O'Conner states that "reliability is usually concerned with failures in the
> time domain. This distinctions marks the difference between traditional
> quality control and reliability engineering".
>
> He goes on to list a number of reasons why a failure may occur as follows
> (abridged)
>
> - design: which may be inadequate
> - overstress: i.e. analysis of condition was incomplete and/or incorrect
> - variation: which includes manufacturing variation
> - wear out, which is what everyone seems to think of...
> - error, such as errors in specification
>
>
> Henley and Kumamoto give a potted history of the development of reliability
> engineering and track its roots to work done by Lusser on the V2 (V1?)
> missile which was spectacularly unreliable at first. But obviously not a
> wear out issue...
>
> This is of course separate from whether in any context reliability is
> useful
> concept e.g. software. You can obviously measure the reliability of
> Windows98 is terms of mean time between crashes (failures in time) likewise
> you can measure the reliability of Linux in the same manner. Whether that
> has any deep meaning aside from the fact that it shows Windows98 to be
> pants
> compared to Linux for some distribution of uses/input/outputs a different
> question.
>
> Cheers.
>
> -----Original Message-----
> From: systemsafety
> [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of
> Coq, Thierry
> Sent: 24 April 2016 07:24
> To: The System Safety List
> Subject: Re: [SystemSafety] Does "reliable" mean "safe" and or "secure" or
> neither?
>
> Hi
> I find this list hugely informative. In particular, I find PBL's posts
> factual, useful, interesting and intriguing. As well as the many other
> debaters who agree with him or challenge him. As it should be. I wish to
> express my gratitude to all debaters.
>
> However, this last exchange seems to me a debate on authority.
> On our left, we have DO-178. B now C.
> On our right, we have IEC, IEEE, Musa, etc.
>
> To go further, it is plain fact that the aeronautics industry has
> demonstrated it doesn't need "software reliability" to deliver highly
> reliable automated systems, or systems of systems.
> It seems evident with the knowledge we have of the aeronautics success that
> in order to use "software reliability" in other industries, or in
> aeronautics, there needs to be a clear use case where the value of
> "software
> reliability" is demonstrated, compared to other methods or techniques, in
> order to apply "software reliability". The analogy of software and hardware
> does not seem valid, as software does not "fatigue" or randomly "break" the
> way hardware does, which is the basis for all probabilistic reliability
> theories for hardware. The analogy that does seem valid between hardware
> and
> software is the presence of systematic faults, in design, manufacturing,
> installation, testing, misuse, etc. Which in hardware also does not have a
> probability number. Formal methods can be used to identify such systematic
> faults in software. If one can be found, then a test environment can be
> devised in which the software will fail 100% of the time. Random hardware
> faults do not behave like t hat.
>
> Best regards,
> Thierry Coq
> The opinions reflected here are my own and are not necessarily those of my
> employer
>
> -----Original Message-----
> From: systemsafety
> [mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de] On Behalf Of
> Peter Bernard Ladkin
> Sent: dimanche 24 avril 2016 03:27
> To: The System Safety List
> Subject: Re: [SystemSafety] Does "reliable" mean "safe" and or "secure" or
> neither?
>
> On 2016-04-23 19:43 , Nick Tudor wrote:
> > DO-178C
>
> In the absence of a complete sentence, let me suggest one.
>
> ---- DO178C sees no need to assign any meaning to the term "software
> reliability".
>
> It's fine for some industry consortium to find it has no use for a specific
> concept. RTCA likely has no use for the notion of a cup of tea, either
> (BS6008). But that doesn't mean it makes any sense to argue that there
> isn't
> any such thing as a cup of tea.
>
> PBL
>
> Prof. Peter Bernard Ladkin, Faculty of Technology, University of Bielefeld,
> 33594 Bielefeld, Germany Je suis Charlie
> Tel+msg +49 (0)521 880 7319 www.rvs.uni-bielefeld.de
>
>
>
>
>
>
>
> ****************************************************************************
> **********
> This e-mail and any attachments thereto may contain confidential
> information
> and/or information protected by intellectual property rights for the
> exclusive attention of the intended addressees named above. If you have
> received this transmission in error, please immediately notify the sender
> by
> return e-mail and delete this message and its attachments. Unauthorized
> use,
> copying or further full or partial distribution of this e-mail or its
> contents is prohibited.
>
> ****************************************************************************
> **********
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
>
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
>
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
>
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
> _______________________________________________
> The System Safety Mailing List
> systemsafety at TechFak.Uni-Bielefeld.DE
>
--
Yours,
Dewi Daniels | Director | Software Safety Limited
Telephone +44 7968 837742 | Email d <ddaniels at verocel.com>
ewi.daniels at software-safety.com
Software Safety Limited is a company registered in England and Wales.
Company number: 9390590. Registered office: Fairfield, 30F Bratton Road,
West Ashton, Trowbridge, United Kingdom BA14 6AZ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/mailman/private/systemsafety/attachments/20160428/102470ba/attachment-0001.html>
More information about the systemsafety
mailing list