Purpose: Results from a pilot study demonstrated moderate to high agreement between reviewers for determining cause of cytologic-histologic diagnostic errors using a “No Blame Box” compared to traditional blame-oriented adjudication methods. The “No Blame Box” is a decision aid to assess the specimen interpretability through identifying tumor presence and specimen quality. Although the “No Blame Box” uses a continuous categorization scale, we created a “box” separated into 4 quadrants: A) no tumor, poor quality, B) no tumor, good quality, C) tumor, poor quality, and D) tumor, good quality. Our goal was to determine if actual reviewers could exhibit the same high level of agreement as seen in the pilot study. Methods: A new study set of cytologic-histologic errors from 40 patients was circulated to 7 reviewers to determine the level of agreement. Reviewers provided the interpretability of the specimen by indicating A, B, C, or D corresponding to the tumor presence and quality of the specimen from the “No Blame Box” as well as a diagnostic assessment for both the cytology and surgical specimens. From the review diagnoses, a reason for discrepancy of cytology/surgical interpretation/sampling was assigned. Agreement between reviewers was measured using a pair-wise kappa statistic, and a value over 0.4 was considered to be of moderate agreement. Results: The median kappa between the 7 reviewers when categorizing cytologic interpretability of the specimen as ABC versus D (which represents poor quality or low tumor versus excellent quality with tumor) was 0.474, ranging from 0.262 to 0.875 with fewer than 29% of the kappas falling below 0.4. The median kappa for the same “No Blame Box” classification of ABC versus D for the surgical specimens was 0.573, ranging from 0.420 to 0.775. The median kappa for assigned reason for discrepancy based on review diagnoses was 0.507, ranging from 0.254 to 0.702 with fewer than 10%of the kappas falling below 0.4. Conclusion: The use of the “No Blame Box” decision aid maintained the inter-observer agreement seen in the pilot study, which indicates it is a reliable tool to assess the interpretability of the specimens. This decision aid would be very beneficial to incorporate in to the cytologic-histologic review process to provide a more reliable measure for reasons for discrepant cases.