Kappa Calculation
There are three steps to calculate a Kappa coefficient:
Step one, rater sheets should be filled out for each rater. In the example rater sheet below, there are three excerpts and four themes. Enter 1 in the corresponding cell if the rater thought the theme was present in that excerpt; enter 0 if the rater thought the theme was absent.
Discover How We Assist to Edit Your Dissertation Chapters
Aligning theoretical framework, gathering articles, synthesizing gaps, articulating a clear methodology and data plan, and writing about the theoretical and practical implications of your research are part of our comprehensive dissertation editing services.
- Bring dissertation editing expertise to chapters 1-5 in timely manner.
- Track all changes, then work with you to bring about scholarly writing.
- Ongoing support to address committee feedback, reducing revisions.
Rater Sheet: RATER 1
| Excerpt # | Theme(1=Present 0=Absent) | |||
| Theme W | Theme X | Theme Y | Theme Z | |
| Excerpt 1 | 1 | 1 | 1 | 1 |
| Excerpt 2 | 0 | 1 | 0 | 1 |
| Excerpt 3 | 0 | 1 | 1 | 1 |
| Excerpt 4 | 1 | 0 | 0 | 1 |
| Excerpt 5 | 1 | 1 | 1 | 1 |
Rater Sheet: RATER 2
| Excerpt # | Theme(1=Present 0=Absent) | |||
| Theme W | Theme X | Theme Y | Theme Z | |
| Excerpt 1 | 1 | 1 | 0 | 0 |
| Excerpt 2 | 0 | 0 | 1 | 1 |
| Excerpt 3 | 1 | 1 | 1 | 1 |
| Excerpt 4 | 1 | 0 | 0 | 1 |
| Excerpt 5 | 1 | 1 | 1 | 1 |
Step two, you will need to fill out a matrix for each theme. For theme W for example, Rater 1 and Rater 2 agreed 3 times that theme W was present in the excerpts, so enter 3 in box A of the matrix. One time Rater 1 thought theme W was absent and Rater 2 thought theme W was present, so enter 1 in box B. Rater 1 never thought a theme was present that Rater 2 thought was absent, so enter 0 in box C. One time Rater 1 and Rater 2 agreed that theme W was absent, so enter 1 in box D.
Key
| Rater 1 | |||
| Present | Absent | ||
| Rater 2 | Present | A | B |
| Absent | C | D |
Theme W
| Rater 1 | |||
| Present | Absent | ||
| Rater 2 | Present | 3 | 1 |
| Absent | 0 | 1 |
Theme X
| Rater 1 | |||
| Present | Absent | ||
| Rater 2 | Present | 3 | 0 |
| Absent | 1 | 1 |
Theme Y
| Rater 1 | |||
| Present | Absent | ||
| Rater 2 | Present | 2 | 1 |
| Absent | 1 | 1 |
Theme Z
| Rater 1 | |||
| Present | Absent | ||
| Rater 2 | Present | 4 | 0 |
| Absent | 1 | 0 |
Enter your theme matrix values in the calculator to determine your kappa values.
| Theme | A | B | C | D | Observed | Expected | Kappa |
| W | 3 | 1 | 0 | 1 | 0.8 | 0.56 | 0.00 |
| X | 3 | 0 | 1 | 1 | 0.8 | 0.56 | 0.00 |
| Y | 2 | 1 | 1 | 1 | 0.6 | 0.52 | 0.00 |
| Z | 4 | 0 | 1 | 0 | 0.8 | 0.8 | 0.00 |