As a part of MSA (Measurement System Analysis), we analyze repeatability and reproducibility of a measurement system in a Lean Six Sigma project. It’s called Gage R&R. The Gage R&R can be categorized into two types – Continuous Gage R&R and Attribute Gage R&R.
The Continuous Gage R&R is mainly used by the Lean Six Sigma project to analyze a measurement system with a continuous variable such as resistance, voltage and dimension.
The Attribute Gage R&R is rarely used in the Lean Six Sigma project. It typically measures a discrete variable such as Pass/Fail criterion and numerical rating. But the applications of Attribute Gage R&R are surprisingly wide. So I tried to use the Attribute Gage R&R in a Lean Six Sigma project.
Motivation
There are many opportunities to make a decision in a group such as a decision on schedule, a specification, or a target cost for example. Even in a software development, the size of backlog is determined by the SCRUM team.
The Lean Six Sigma project often uses FMEA (Failure Mode and Effects Analysis) with RPN (Risk Priority Number) which is determined by the team in a meeting.
Such examples use a discrete variable, they are a group decision, and the decision makes a great impact on the business.
Does the decision made by an individual have consistency? Does the decision made by the group have consistency? Are repeatability and reproducibility on the judgments good enough? If not, the business could have a problem. Such questions became a motivation to study the Attribute Gage R&R with FMEA.
Preparation
The RPN (Risk Priority Number) is calculated with the three factors – Severity, Occurrence and Detectablity. I used the Severity for the Attribute Gage R&R.
First, we as a team (four people) discussed all failure modes and the severity rating to understand the meaning of each failure mode and the rating system.
Then I created twelve tables with the failure modes in randomized order for each person in the team for three days execution (12 table = 4 people x 3 days)
Execution
I handed out the randomized table to each person every day. Each person rated the Severity for each failure mode in the table. We repeated the rating for three days.
Analysis
We collected the 12 results (4 people in 3 days). Also we rated the Severity in the team. The result of team evaluation became the standard rating.
Attribute Agreement Analysis
I put all results and the standard rating on Minitab and execute the Attribute Agreement Analysis on it. Then I saw the agreements were around 60% in both “Within Appraisers” and “Appraiser vs Standard”. Some Kappa values were less than 0.6. The result was pretty bad.
The reasons why the agreements (consistences) were low could be:
- Understanding of failure modes was not good enough (and/or inconsistent in the team)
- Understanding of rating system was not good enough (and/or inconsistent in the team)
They might come from lack of discussion before the execution and lack of training for the FMEA.
Countermeasure
Not only in the FMEA but in other group decision, we need to have enough time in a meeting to reach common understanding of each item. And we need to have good understanding of tool for group decision making.
Conclusion
If we have to make an important decision in a team with a rating system, the Attribute Agreement Analysis can be a good tool to measure our understanding of the items.