Updated ProChair, Review, and Diversity pages - #368
Conversation
|
|
||
| Keeping the review form reasonably stable across editions can help | ||
| authors and reviewers calibrate and interpret its ratings. | ||
| One possible model is Nierstrasz's Identify the Champion scoring process, |
There was a problem hiding this comment.
So Identify the Champion is actually more general than this and does not prescribe A-D scoring. (Personally, I prefer to have two 'champion' levels so you can champion without giving the topmost rating.) I would just tweak the language here a little.
There was a problem hiding this comment.
Here is another scoring system, for example: https://andrumyers.wordpress.com/2022/02/10/headroom-for-champions-a-scoring-system-for-identify-the-champion/
There was a problem hiding this comment.
Not proposing to add a link to this, btw.
I do wonder if we want to reinforce the right interpretation of the scores though. I stand by my critique that the A-D scoring system has a centralizing effect that has to be combated somehow.
| submission, evaluation, archival availability, licensing, and any | ||
| requirements for persistent identifiers or external repositories. | ||
|
|
||
| ### Auxiliary Material |
There was a problem hiding this comment.
This is what we've typically called supplementary material, right? Should we at least mention that name?
| assigned to them, but must not have access to unrelated submissions or | ||
| discussions. | ||
|
|
||
| The PCC should exercise some judgment in assessing expertise ratings. Some PC members consider themselves experts on most topics, while other, more modest, PC members almost never declare themselves an expert. In the end, the PCC should use his/her judgment in deciding whether the expertise on a given paper is sufficient. |
There was a problem hiding this comment.
Did we discuss removing this point or did it go elsewhere? It still seems relevant to me.
| -- whether the paper (and supporting material) contain sufficient information for others to reproduce and build on the results | ||
| - whether there is a genuine research contribution which may include a new insightful evaluation of previous work | ||
| - whether the approach is fundamentally sound | ||
| - whether the community will benefit from reading the paper |
There was a problem hiding this comment.
A bit implicit here is whether the results are of sufficiently broad interest to the community. Do we want to say something about that? I feel like we've veered a little too far in the direction of accepting any solid brick for the wall of knowledge.
andrewcmyers
left a comment
andrewcmyers
left a comment
There was a problem hiding this comment.
Overall this is a great set of changes to the guidance.
|
This latest discussion around POPL spam papers is making me wonder if we should add something about that. |
2 participants
Tentative update. Please don't merge until reviewed by the SIGPLAN EC.