Mixing
How do conferences / journals select reviewers?
Clash Royale CLAN TAG#URR8PPP
up vote
1
down vote
favorite
When you submit a paper / manuscript to a peer-reviewed venue, the organizers usually forward your work to 2-3 other "experts in your field" to assess your work.
I'm curious how this process of selection works, i.e. how is this pool of reviewers put together? and how are the best people from this pool chosen once there is a paper to review?
Is there an official list that each venue maintains and to which people can sign on if they want to be reviewers? (If so, I've never seen any "calls for reviewers" so how do they get people?)
Do organizers cold-call people/professors and ask them if they were willing to review a paper? If so, how are these people selected? [I realize that this might differ considerably, so I'm happy with anecdotal evidence, too]
journals peer-review conference paper-submission
asked 1 hour ago
FirefoxMetzger
1634
add a comment |Â
up vote
1
down vote
favorite
When you submit a paper / manuscript to a peer-reviewed venue, the organizers usually forward your work to 2-3 other "experts in your field" to assess your work.
I'm curious how this process of selection works, i.e. how is this pool of reviewers put together? and how are the best people from this pool chosen once there is a paper to review?
Is there an official list that each venue maintains and to which people can sign on if they want to be reviewers? (If so, I've never seen any "calls for reviewers" so how do they get people?)
Do organizers cold-call people/professors and ask them if they were willing to review a paper? If so, how are these people selected? [I realize that this might differ considerably, so I'm happy with anecdotal evidence, too]
journals peer-review conference paper-submission
asked 1 hour ago
FirefoxMetzger
1634
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
When you submit a paper / manuscript to a peer-reviewed venue, the organizers usually forward your work to 2-3 other "experts in your field" to assess your work.
I'm curious how this process of selection works, i.e. how is this pool of reviewers put together? and how are the best people from this pool chosen once there is a paper to review?
Is there an official list that each venue maintains and to which people can sign on if they want to be reviewers? (If so, I've never seen any "calls for reviewers" so how do they get people?)
Do organizers cold-call people/professors and ask them if they were willing to review a paper? If so, how are these people selected? [I realize that this might differ considerably, so I'm happy with anecdotal evidence, too]
journals peer-review conference paper-submission
asked 1 hour ago
FirefoxMetzger
1634
When you submit a paper / manuscript to a peer-reviewed venue, the organizers usually forward your work to 2-3 other "experts in your field" to assess your work.
I'm curious how this process of selection works, i.e. how is this pool of reviewers put together? and how are the best people from this pool chosen once there is a paper to review?
Is there an official list that each venue maintains and to which people can sign on if they want to be reviewers? (If so, I've never seen any "calls for reviewers" so how do they get people?)
Do organizers cold-call people/professors and ask them if they were willing to review a paper? If so, how are these people selected? [I realize that this might differ considerably, so I'm happy with anecdotal evidence, too]
journals peer-review conference paper-submission
journals peer-review conference paper-submission
asked 1 hour ago
FirefoxMetzger
1634
asked 1 hour ago
FirefoxMetzger
1634
asked 1 hour ago
FirefoxMetzger
1634
asked 1 hour ago
FirefoxMetzger
1634
asked 1 hour ago
FirefoxMetzger
1634
1634
add a comment |Â
add a comment |Â
3 Answers
3
active
oldest
votes
up vote
4
down vote
In the journals I help edit (all in the medical fields), there are five ways we find reviewers for our manuscripts:
- They have published with us before
- They were nominated by the authors, editors, other reviewers or peers, or members of our editorial board
- Our associate editors approached them through a scan of authors of papers on the same topic
- They have volunteered to be placed on our list of reviewers
- They work for us as consultants and are paid on a per-review basis (this is mostly relevant when we need specialist expertise such as in statistics or economics)
Once you are found, you are placed on a pool from which we can make selections on the basis of affiliation, expertise, seniority, etc.
As you can imagine, this is a rather valuable resource. We guard this list jealously and do not share this with other journals.
In my experience the response rate is typically 33%. That is, if we require five reviews, we will invite fifteen people. Thus, the pool needs to be deep in order to receive the requisite number of reviews for our purposes.
answered 56 mins ago
St. Inkbug
2,830726
2
For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
– David Ketcheson
50 mins ago
add a comment |Â
up vote
1
down vote
Answering from the perspective of computer science conferences:
There is a committee, usually dubbed the "program committee". This is a group of of people that are invited by the conference organizers and this committee is formed in advance of papers being submitted. The size of the program committee varies from a dozen for a small workshop to thousands for big machine learning conference. The list is usually published and will change from year to year.
Sometimes the program committee does all of the reviewing. Papers are assigned by program chairs based on a bidding process. However, they may also bring in sub-reviewers. Sub-reviewers are third parties that are cold-called to review a paper because of their relevant expertise. Selection of sub-reviewers is up to the program committee members. Usually, they ask people they know, or if they are lost they look through the references of the submitted paper.
answered 43 mins ago
Thomas
7,68731733
add a comment |Â
up vote
1
down vote
Examples from my experience:
- Anyone who has an account in the editorial management system for whatever reason. This could be former authors, former reviewers, former editorial board members, people who've written to us explicitly asking to be a reviewer, etc, as long as they have not indicated they are not willing to review or are not currently blacklisted (e.g. if a reviewer writes a very poor review, we might blacklist).
- Author-suggested reviewers (which we try to use sparingly since they often know the author personally)
- Reviewers suggested by other reviewers (e.g. if we invite, they decline but suggest X, we will invite X)
- From the references of the paper. If the paper is well-written then the introduction would give lots of related works, and the authors of these related works would be suitable to review the paper.
- Searching via Google Scholar, Web of Science, etc using keywords for the paper (sometimes but not necessarily the author-provided keywords).
- Personal contacts.
To choose people from the reviewer pool: choose the person with the closest-related expertise, as long as he/she is not already overburdened (e.g. already has an outstanding invitation, or has done three reviews in three months).
Cold-inviting reviewers does happen, and is a natural result from some of the methods mentioned above. It's not generally perceived as socially unacceptable however. If the reviewer doesn't want to review for the journal, they can decline indicating they're permanently not interested, we leave a note on their profile, and they're excluded from future searches.
answered 26 mins ago
Allure
19.9k1265110
add a comment |Â
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
4
down vote
In the journals I help edit (all in the medical fields), there are five ways we find reviewers for our manuscripts:
- They have published with us before
- They were nominated by the authors, editors, other reviewers or peers, or members of our editorial board
- Our associate editors approached them through a scan of authors of papers on the same topic
- They have volunteered to be placed on our list of reviewers
- They work for us as consultants and are paid on a per-review basis (this is mostly relevant when we need specialist expertise such as in statistics or economics)
Once you are found, you are placed on a pool from which we can make selections on the basis of affiliation, expertise, seniority, etc.
As you can imagine, this is a rather valuable resource. We guard this list jealously and do not share this with other journals.
In my experience the response rate is typically 33%. That is, if we require five reviews, we will invite fifteen people. Thus, the pool needs to be deep in order to receive the requisite number of reviews for our purposes.
answered 56 mins ago
St. Inkbug
2,830726
2
For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
– David Ketcheson
50 mins ago
add a comment |Â
up vote
4
down vote
In the journals I help edit (all in the medical fields), there are five ways we find reviewers for our manuscripts:
- They have published with us before
- They were nominated by the authors, editors, other reviewers or peers, or members of our editorial board
- Our associate editors approached them through a scan of authors of papers on the same topic
- They have volunteered to be placed on our list of reviewers
- They work for us as consultants and are paid on a per-review basis (this is mostly relevant when we need specialist expertise such as in statistics or economics)
Once you are found, you are placed on a pool from which we can make selections on the basis of affiliation, expertise, seniority, etc.
As you can imagine, this is a rather valuable resource. We guard this list jealously and do not share this with other journals.
In my experience the response rate is typically 33%. That is, if we require five reviews, we will invite fifteen people. Thus, the pool needs to be deep in order to receive the requisite number of reviews for our purposes.
answered 56 mins ago
St. Inkbug
2,830726
2
For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
– David Ketcheson
50 mins ago
add a comment |Â
up vote
4
down vote
up vote
4
down vote
In the journals I help edit (all in the medical fields), there are five ways we find reviewers for our manuscripts:
- They have published with us before
- They were nominated by the authors, editors, other reviewers or peers, or members of our editorial board
- Our associate editors approached them through a scan of authors of papers on the same topic
- They have volunteered to be placed on our list of reviewers
- They work for us as consultants and are paid on a per-review basis (this is mostly relevant when we need specialist expertise such as in statistics or economics)
Once you are found, you are placed on a pool from which we can make selections on the basis of affiliation, expertise, seniority, etc.
As you can imagine, this is a rather valuable resource. We guard this list jealously and do not share this with other journals.
In my experience the response rate is typically 33%. That is, if we require five reviews, we will invite fifteen people. Thus, the pool needs to be deep in order to receive the requisite number of reviews for our purposes.
answered 56 mins ago
St. Inkbug
2,830726
In the journals I help edit (all in the medical fields), there are five ways we find reviewers for our manuscripts:
- They have published with us before
- They were nominated by the authors, editors, other reviewers or peers, or members of our editorial board
- Our associate editors approached them through a scan of authors of papers on the same topic
- They have volunteered to be placed on our list of reviewers
- They work for us as consultants and are paid on a per-review basis (this is mostly relevant when we need specialist expertise such as in statistics or economics)
Once you are found, you are placed on a pool from which we can make selections on the basis of affiliation, expertise, seniority, etc.
As you can imagine, this is a rather valuable resource. We guard this list jealously and do not share this with other journals.
In my experience the response rate is typically 33%. That is, if we require five reviews, we will invite fifteen people. Thus, the pool needs to be deep in order to receive the requisite number of reviews for our purposes.
answered 56 mins ago
St. Inkbug
2,830726
answered 56 mins ago
St. Inkbug
2,830726
answered 56 mins ago
St. Inkbug
2,830726
answered 56 mins ago
St. Inkbug
2,830726
2,830726
2
For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
– David Ketcheson
50 mins ago
add a comment |Â
2
For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
– David Ketcheson
50 mins ago
2
2
For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
– David Ketcheson
50 mins ago
For me (as an editor) another main source of referees comes from looking at the authors of the most relevant and recent papers cited in a submitted manuscript. I don't have a "list" that would be "guarded jealously", however...that seems a bit strange since anyone can easily figure out who is working in a given area.
– David Ketcheson
50 mins ago
add a comment |Â
up vote
1
down vote
Answering from the perspective of computer science conferences:
There is a committee, usually dubbed the "program committee". This is a group of of people that are invited by the conference organizers and this committee is formed in advance of papers being submitted. The size of the program committee varies from a dozen for a small workshop to thousands for big machine learning conference. The list is usually published and will change from year to year.
Sometimes the program committee does all of the reviewing. Papers are assigned by program chairs based on a bidding process. However, they may also bring in sub-reviewers. Sub-reviewers are third parties that are cold-called to review a paper because of their relevant expertise. Selection of sub-reviewers is up to the program committee members. Usually, they ask people they know, or if they are lost they look through the references of the submitted paper.
answered 43 mins ago
Thomas
7,68731733
add a comment |Â
up vote
1
down vote
Answering from the perspective of computer science conferences:
There is a committee, usually dubbed the "program committee". This is a group of of people that are invited by the conference organizers and this committee is formed in advance of papers being submitted. The size of the program committee varies from a dozen for a small workshop to thousands for big machine learning conference. The list is usually published and will change from year to year.
Sometimes the program committee does all of the reviewing. Papers are assigned by program chairs based on a bidding process. However, they may also bring in sub-reviewers. Sub-reviewers are third parties that are cold-called to review a paper because of their relevant expertise. Selection of sub-reviewers is up to the program committee members. Usually, they ask people they know, or if they are lost they look through the references of the submitted paper.
answered 43 mins ago
Thomas
7,68731733
add a comment |Â
up vote
1
down vote
up vote
1
down vote
Answering from the perspective of computer science conferences:
There is a committee, usually dubbed the "program committee". This is a group of of people that are invited by the conference organizers and this committee is formed in advance of papers being submitted. The size of the program committee varies from a dozen for a small workshop to thousands for big machine learning conference. The list is usually published and will change from year to year.
Sometimes the program committee does all of the reviewing. Papers are assigned by program chairs based on a bidding process. However, they may also bring in sub-reviewers. Sub-reviewers are third parties that are cold-called to review a paper because of their relevant expertise. Selection of sub-reviewers is up to the program committee members. Usually, they ask people they know, or if they are lost they look through the references of the submitted paper.
answered 43 mins ago
Thomas
7,68731733
Answering from the perspective of computer science conferences:
There is a committee, usually dubbed the "program committee". This is a group of of people that are invited by the conference organizers and this committee is formed in advance of papers being submitted. The size of the program committee varies from a dozen for a small workshop to thousands for big machine learning conference. The list is usually published and will change from year to year.
Sometimes the program committee does all of the reviewing. Papers are assigned by program chairs based on a bidding process. However, they may also bring in sub-reviewers. Sub-reviewers are third parties that are cold-called to review a paper because of their relevant expertise. Selection of sub-reviewers is up to the program committee members. Usually, they ask people they know, or if they are lost they look through the references of the submitted paper.
answered 43 mins ago
Thomas
7,68731733
answered 43 mins ago
Thomas
7,68731733
answered 43 mins ago
Thomas
7,68731733
answered 43 mins ago
Thomas
7,68731733
7,68731733
add a comment |Â
add a comment |Â
up vote
1
down vote
Examples from my experience:
- Anyone who has an account in the editorial management system for whatever reason. This could be former authors, former reviewers, former editorial board members, people who've written to us explicitly asking to be a reviewer, etc, as long as they have not indicated they are not willing to review or are not currently blacklisted (e.g. if a reviewer writes a very poor review, we might blacklist).
- Author-suggested reviewers (which we try to use sparingly since they often know the author personally)
- Reviewers suggested by other reviewers (e.g. if we invite, they decline but suggest X, we will invite X)
- From the references of the paper. If the paper is well-written then the introduction would give lots of related works, and the authors of these related works would be suitable to review the paper.
- Searching via Google Scholar, Web of Science, etc using keywords for the paper (sometimes but not necessarily the author-provided keywords).
- Personal contacts.
To choose people from the reviewer pool: choose the person with the closest-related expertise, as long as he/she is not already overburdened (e.g. already has an outstanding invitation, or has done three reviews in three months).
Cold-inviting reviewers does happen, and is a natural result from some of the methods mentioned above. It's not generally perceived as socially unacceptable however. If the reviewer doesn't want to review for the journal, they can decline indicating they're permanently not interested, we leave a note on their profile, and they're excluded from future searches.
answered 26 mins ago
Allure
19.9k1265110
add a comment |Â
up vote
1
down vote
Examples from my experience:
- Anyone who has an account in the editorial management system for whatever reason. This could be former authors, former reviewers, former editorial board members, people who've written to us explicitly asking to be a reviewer, etc, as long as they have not indicated they are not willing to review or are not currently blacklisted (e.g. if a reviewer writes a very poor review, we might blacklist).
- Author-suggested reviewers (which we try to use sparingly since they often know the author personally)
- Reviewers suggested by other reviewers (e.g. if we invite, they decline but suggest X, we will invite X)
- From the references of the paper. If the paper is well-written then the introduction would give lots of related works, and the authors of these related works would be suitable to review the paper.
- Searching via Google Scholar, Web of Science, etc using keywords for the paper (sometimes but not necessarily the author-provided keywords).
- Personal contacts.
To choose people from the reviewer pool: choose the person with the closest-related expertise, as long as he/she is not already overburdened (e.g. already has an outstanding invitation, or has done three reviews in three months).
Cold-inviting reviewers does happen, and is a natural result from some of the methods mentioned above. It's not generally perceived as socially unacceptable however. If the reviewer doesn't want to review for the journal, they can decline indicating they're permanently not interested, we leave a note on their profile, and they're excluded from future searches.
answered 26 mins ago
Allure
19.9k1265110
add a comment |Â
up vote
1
down vote
up vote
1
down vote
Examples from my experience:
- Anyone who has an account in the editorial management system for whatever reason. This could be former authors, former reviewers, former editorial board members, people who've written to us explicitly asking to be a reviewer, etc, as long as they have not indicated they are not willing to review or are not currently blacklisted (e.g. if a reviewer writes a very poor review, we might blacklist).
- Author-suggested reviewers (which we try to use sparingly since they often know the author personally)
- Reviewers suggested by other reviewers (e.g. if we invite, they decline but suggest X, we will invite X)
- From the references of the paper. If the paper is well-written then the introduction would give lots of related works, and the authors of these related works would be suitable to review the paper.
- Searching via Google Scholar, Web of Science, etc using keywords for the paper (sometimes but not necessarily the author-provided keywords).
- Personal contacts.
To choose people from the reviewer pool: choose the person with the closest-related expertise, as long as he/she is not already overburdened (e.g. already has an outstanding invitation, or has done three reviews in three months).
Cold-inviting reviewers does happen, and is a natural result from some of the methods mentioned above. It's not generally perceived as socially unacceptable however. If the reviewer doesn't want to review for the journal, they can decline indicating they're permanently not interested, we leave a note on their profile, and they're excluded from future searches.
answered 26 mins ago
Allure
19.9k1265110
Examples from my experience:
- Anyone who has an account in the editorial management system for whatever reason. This could be former authors, former reviewers, former editorial board members, people who've written to us explicitly asking to be a reviewer, etc, as long as they have not indicated they are not willing to review or are not currently blacklisted (e.g. if a reviewer writes a very poor review, we might blacklist).
- Author-suggested reviewers (which we try to use sparingly since they often know the author personally)
- Reviewers suggested by other reviewers (e.g. if we invite, they decline but suggest X, we will invite X)
- From the references of the paper. If the paper is well-written then the introduction would give lots of related works, and the authors of these related works would be suitable to review the paper.
- Searching via Google Scholar, Web of Science, etc using keywords for the paper (sometimes but not necessarily the author-provided keywords).
- Personal contacts.
To choose people from the reviewer pool: choose the person with the closest-related expertise, as long as he/she is not already overburdened (e.g. already has an outstanding invitation, or has done three reviews in three months).
Cold-inviting reviewers does happen, and is a natural result from some of the methods mentioned above. It's not generally perceived as socially unacceptable however. If the reviewer doesn't want to review for the journal, they can decline indicating they're permanently not interested, we leave a note on their profile, and they're excluded from future searches.
answered 26 mins ago
Allure
19.9k1265110
answered 26 mins ago
Allure
19.9k1265110
answered 26 mins ago
Allure
19.9k1265110
answered 26 mins ago
Allure
19.9k1265110
19.9k1265110
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2facademia.stackexchange.com%2fquestions%2f118620%2fhow-do-conferences-journals-select-reviewers%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
