Going beyond impact factor to evaluate researchers with Profeza

ProfezaFor many reasons, journal impact factor and number of publications are not good metrics to assess the quality of a researcher’s work. But regardless of their increasingly bad reputations, these metrics are nearly invariably used to take decisions about recruitments of researchers, their promotions, and funding their projects. The obvious reason why nothing has changed over the years is because there are no other easy way to judge the quality of a researcher and his or her work.

We ask a lot of researchers. They must be great at scientific reasoning and have bright insights but also be able to properly communicate with their teams, with the scientific community, with the general public, and with industrial partners. They also need to be able to network and work within teams, to manage projects and people, to teach, and to write skillfully in a language that is often not their own. It is easy to see that we would need a multitude of alternative metrics to properly evaluate the various aspects of the day-to-day work of researchers. 

Profeza is a young startup that would like to provide decision makers a better overview of the work of researchers. It has launched a social journal that allows researchers to showcase the divers aspects of their work by sharing the rational of experimental design, the failed hypotheses, as well as raw data, repeat data, and supporting data that would otherwise often go unpublished. For Profeza, each scientific article is only the tip of the iceberg, standing on a immense amount of work. 

Profeza’s interface is simple and clear. First, find the publications you authored through Profeza’s search engine. Profeza’s is currently using the Pubmed database and is thus better optimized for researchers in the biomedical fields. Then in three steps you are prompted to add information to the publication:

1. Select the publication you wish to add information to.

Share contributions2. Describe your contribution to the paper and invite other authors that may not be in the author list but should get recognition for their involvement in the work.

What contribution

3. Add information. You can add text and files containing the details about the rational of design, failed hypotheses, raw data, repeat and supporting data. This is a great way to help others in your field by tell them about your failures or negative results.

Additional data

The end result is a personalized page for each article containing the additional data and information. The page gives a better picture of the work that went into the publication and provides an insight in the short term impact of the articles by displaying altmetric data. 

I think Profeza is addressing a real problem head-on. The success will of course depend on the willingness of researchers to spend time formatting and entering the information and datasets. But if institutions are willing to play along, then the incentives would be in place and a more adapted evaluation system could emerge. These are still the early days. Profeza was founded in 2014 and expects to roll out new functionalities in the near future.

Also check out this well-crafted video from Profeza which gives a nice background on journal impact factors and the problems associated with them.