The idea is that you post your paper on some common repository like arxiv. The "reviewer entity" or RE which is self-organized group with interest in sub-field can be invited to review it. The RE accepts or rejects it. Citations graph propogates to RE as reputation score.
The problem with his proposal:
1. Authors are not anonymous which can bias prestigious REs to avoid papers from unknown authors.
2. Everything still depends on citations, most of which are usually worthless as people mostly cite to fill obligatory related sections, not because they are actually using your work.
3. No obligation for REs to review anything. Doing reviews is tiresome and busy people may just avoid it unless there is a clear obligation/honor system involved.
4. Prestigious REs will be invited to review by everyone, causing highly uneven distribution for the workload.
I believe openreview system was started based on LeCun's thoughts but needless to say we haven't found a good system that can resolve above issues. More importantly, any change in system needs to come from community leaders who are part of organizing commitees for conferences or website developers like arxiv and Google Scholars. Unfortunately, last two have been virtual stasis for long time.
It's astonishing how little investment exists for our main engines of progress that is scientific progress. Compare number of developers working on arxiv or Google Scholar vs Twitter!
The idea is that you post your paper on some common repository like arxiv. The "reviewer entity" or RE which is self-organized group with interest in sub-field can be invited to review it. The RE accepts or rejects it. Citations graph propogates to RE as reputation score.
The problem with his proposal:
1. Authors are not anonymous which can bias prestigious REs to avoid papers from unknown authors.
2. Everything still depends on citations, most of which are usually worthless as people mostly cite to fill obligatory related sections, not because they are actually using your work.
3. No obligation for REs to review anything. Doing reviews is tiresome and busy people may just avoid it unless there is a clear obligation/honor system involved.
4. Prestigious REs will be invited to review by everyone, causing highly uneven distribution for the workload.
I believe openreview system was started based on LeCun's thoughts but needless to say we haven't found a good system that can resolve above issues. More importantly, any change in system needs to come from community leaders who are part of organizing commitees for conferences or website developers like arxiv and Google Scholars. Unfortunately, last two have been virtual stasis for long time.
It's astonishing how little investment exists for our main engines of progress that is scientific progress. Compare number of developers working on arxiv or Google Scholar vs Twitter!