Kappa Calculation
There are three steps to calculate a Kappa coefficient:
Step one, rater sheets should be filled out for each rater. In the example rater sheet below, there are three excerpts and four themes. Enter 1 in the corresponding cell if the rater thought the theme was present in that excerpt; enter 0 if the rater thought the theme was absent.
Rater Sheet: RATER 1
| Excerpt # | Theme(1=Present 0=Absent) | |||
| Theme W | Theme X | Theme Y | Theme Z | |
| Excerpt 1 | 1 | 1 | 1 | 1 |
| Excerpt 2 | 0 | 1 | 0 | 1 |
| Excerpt 3 | 0 | 1 | 1 | 1 |
| Excerpt 4 | 1 | 0 | 0 | 1 |
| Excerpt 5 | 1 | 1 | 1 | 1 |
Rater Sheet: RATER 2
| Excerpt # | Theme(1=Present 0=Absent) | |||
| Theme W | Theme X | Theme Y | Theme Z | |
| Excerpt 1 | 1 | 1 | 0 | 0 |
| Excerpt 2 | 0 | 0 | 1 | 1 |
| Excerpt 3 | 1 | 1 | 1 | 1 |
| Excerpt 4 | 1 | 0 | 0 | 1 |
| Excerpt 5 | 1 | 1 | 1 | 1 |
Step two, you will need to fill out a matrix for each theme. For theme W for example, Rater 1 and Rater 2 agreed 3 times that theme W was present in the excerpts, so enter 3 in box A of the matrix. One time Rater 1 thought theme W was absent and Rater 2 thought theme W was present, so enter 1 in box B. Rater 1 never thought a theme was present that Rater 2 thought was absent, so enter 0 in box C. One time Rater 1 and Rater 2 agreed that theme W was absent, so enter 1 in box D.
Key
| Rater 1 | |||
| Present | Absent | ||
| Rater 2 | Present | A | B |
| Absent | C | D |
Theme W
| Rater 1 | |||
| Present | Absent | ||
| Rater 2 | Present | 3 | 1 |
| Absent | 0 | 1 |
Theme X
| Rater 1 | |||
| Present | Absent | ||
| Rater 2 | Present | 3 | 0 |
| Absent | 1 | 1 |
Theme Y
| Rater 1 | |||
| Present | Absent | ||
| Rater 2 | Present | 2 | 1 |
| Absent | 1 | 1 |
Theme Z
| Rater 1 | |||
| Present | Absent | ||
| Rater 2 | Present | 4 | 0 |
| Absent | 1 | 0 |
Enter your theme matrix values in the calculator to determine your kappa values.
| Theme | A | B | C | D | Observed | Expected | Kappa |
| W | 3 | 1 | 0 | 1 | 0.8 | 0.56 | 0.00 |
| X | 3 | 0 | 1 | 1 | 0.8 | 0.56 | 0.00 |
| Y | 2 | 1 | 1 | 1 | 0.6 | 0.52 | 0.00 |
| Z | 4 | 0 | 1 | 0 | 0.8 | 0.8 | 0.00 |
If you’re like others, you’ve invested a lot of time and money developing your dissertation or project research. Finish strong by learning how our dissertation specialists support your efforts to cross the finish line.