[SystemSafety] Analysis of some Work Breakdown Structure projects

[Steve Tockey]

Derek,

“The rework task information is the same as all the other task information, e.g., date, time, person-id, etc. But there is no field specifying what previous tasks have been reworked.”

You don’t need that for an R% calculation. You only need total time spent on rework divided by total time spent on the project. I looked at CESAW_defect_facts.csv and saw that each row has a project_key column and a defect_fix_time_minutes column.

CESAW_project_summary.csv has the overall effort data for each project in column “Effort [hours]”.

Overall R% for the entire data set should be calculate-able by adding up all defect_fix_time_minutes in CESAW_defect_facts.csv and dividing by 60 to convert to hours, then dividing that by the total of the Effort [Hours] column in CESAW_project_summary.csv.

R% on a project-by-project basis should be calculate-able by adding up all defect_fix_time_minutes in CESAW_defect_facts.csv that have the same project_key and dividing by 60 to convert to hours, then dividing that by the Effort [Hours] value for the row with the same project_key in CESAW_project_summary.csv.



Martyn,

“I'd be interested in data on the defects injected and fixed.  How many per KLOC, how variable between individuals, what relationship to component size (LOC), what percentage injected in each phase, where they were found, how long a defect typically remains before being found and fixed ...”

Table CESAW_defect_facts.csv has columns including:

*) wbs_element_key
*) person_key
*) injected_phase_name
*) injected_phase_type
*) defect_found_date
*) defect_removed_phase_key
*) removed_phase_name
*) removed_phase_type

It seems like you could at least get some of the data you’re looking for, e.g., % injected by phase, % detected by phase, how they were found (failure vs. construction vs. appraisal). You should be able to match the WBS element key in CESAW_defect_facts.csv to the corresponding task in CESAW_task_fact.csv to get calendar dates for when the original work on the deliverable was done.


— steve





-----Original Message-----
From: Derek M Jones <derek at knosof.co.uk<mailto:derek at knosof.co.uk>>
Organization: Knowledge Software, Ltd
Date: Wednesday, June 9, 2021 at 7:13 PM
To: Steve Tockey <Steve.Tockey at construx.com<mailto:Steve.Tockey at construx.com>>, "systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>" <systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de>>
Subject: Re: [SystemSafety] Analysis of some Work Breakdown Structure projects

Steve,

Use of a Work Breakdown Structure (WBS) is simply a good project management practice. It’s orthogonal to Waterfall (or other “plan-based”) lifecycles vs. Agile ones.

I was surprised to see how many non-software projects use it.

The WBS is no more and no less than simply a catalog of the work (to be) done on that project...
A running battle I have with the agilista crowd is their insistence that, “our product backlog is our requirements”. No it’s not. Your product backlog is the Agile manifestation of your WBS. The requirements live elsewhere, but that’s another story for another time.

Everybody has their own best practices.  What they all
share is a lack of evidence showing almost anything other
than the fact that they are used.

“I'm not sure of the kind of questions that those involved might ask about such data.  Suggestions welcome.”

There is an implicit "Assuming the data has the information needed to
answer the question."

To me, an extremely useful thing to look at would be “Rework Percentage” (R%). Add up total labor spent repairing defects (it’s a
The WBS structure in the data is essentially parent links,
which enables things like figure 2 to be created.

The rework task information is the same as all the other task
information, e.g., date, time, person-id, etc.  But there is
no field specifying what previous tasks have been reworked.

I was hoping to be able to spot the iterations, but could not
see a simple way to do it.  Somebody familiar with using WBS
may be able to see some tell-tale patterns that indicate iterations.

defect if it is a shortcoming found after the creator of that work claimed that that work was complete, e.g., going into a review or inspection of some sort) and divide that by the total labor on that project.

The most common task in the data are inspections of one sort or
another.

Is reworking a short coming, or an intrinsic part of the process
of building something new?

Writing software is a process of discovery, and accepting reworking
may be the most cost effect way of progressing.

The limited but reliable data I have from 5 organizations of between 50 and 350 developers shows a weighted average of 61.2%. In other words, 61.2 out of every 100 labor hours spent in those organizations goes to fixing defects discovered after the worker claimed their work was complete. This leads directly to the obnoxious but unavoidable conclusion that,
“Rework is not only the single largest driver of cost and schedule on a typical software project; it is bigger than all other drivers combined!”

And what approach would be less costly?  Those who sing the benefits
of complete upfront design are guilty of hand-waving, just as
much as everybody else.

Having said that, R% is highly manageable. I don’t have published, reference-able data but I do have experience on software projects that makes me highly confident R% can be well under 10%. I also have anecdotal data that suggests “Acceptance Test Driven Development” can cut R% in half. R% is the net effect of what Martin Thomas is looking for in timing differences between defect injection and detection. The bigger the difference, the higher R%.

Why is 10% better than 90%?  It depends on the nature of
the problem.

I saw in your paper that the CESAW data set does include rework data. So I think looking at R% both overall and on a project-by-project basis would be interesting and useful.

There is lots of interesting data in there.
'All' that is are good ideas to connect it all
together :-)

— steve
-----Original Message-----
From: systemsafety <systemsafety-bounces at lists.techfak.uni-bielefeld.de<mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de><mailto:systemsafety-bounces at lists.techfak.uni-bielefeld.de>> on behalf of Derek M Jones <derek at knosof.co.uk<mailto:derek at knosof.co.uk><mailto:derek at knosof.co.uk>>
Organization: Knowledge Software, Ltd
Date: Tuesday, June 8, 2021 at 3:07 AM
To: "systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de><mailto:systemsafety at lists.techfak.uni-bielefeld.de>" <systemsafety at lists.techfak.uni-bielefeld.de<mailto:systemsafety at lists.techfak.uni-bielefeld.de><mailto:systemsafety at lists.techfak.uni-bielefeld.de>>
Subject: [SystemSafety] Analysis of some Work Breakdown Structure projects
All,
Until recently data on the use of Agile in safety critical software
was almost non-existent.
I'm not sure if the use of work breakdown structure counts as
Agile, but here is an analysis of lots of data (project 615
was one of the safety critical projects):
http://arxiv.org/abs/2106.03679
Never having used WBS, I'm not sure of the kind of questions
that those involved might ask about such data.  Suggestions
welcome.
--
Derek M. Jones           Evidence-based software engineering
tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com
_______________________________________________
The System Safety Mailing List
systemsafety at TechFak.Uni-Bielefeld.DE<mailto:systemsafety at TechFak.Uni-Bielefeld.DE><mailto:systemsafety at TechFak.Uni-Bielefeld.DE>
Manage your subscription: https://lists.techfak.uni-bielefeld.de/mailman/listinfo/systemsafety

--
Derek M. Jones           Evidence-based software engineering
tel: +44 (0)1252 520667  blog:shape-of-code.coding-guidelines.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.techfak.uni-bielefeld.de/pipermail/systemsafety/attachments/20210610/19fb527b/attachment-0001.html>