Community-Academic Peer Review : Prospects for Strengthening Community-Campus Engagement and Enriching Scholarship

Scholarly peer review is hailed as an indispensable process to maintain quality and rigour in research publications. However, there is growing recognition of the limitations of peer review and concerns about the unexamined assumptions surrounding the processes that favour academic ways of knowing. In this paper, we build on these debates by exploring the possibilities for engaging communities in shaping and assessing the value of knowledge. Drawing on insights of a community-academic peer review pilot project through a pan-Canadian research partnership, we reflect on the value of incorporating community perspectives into research review processes and challenges of scaling-up these efforts. We argue that the perspectives of community-based practitioners are a necessary part of peer review—especially for Community-Based Research—to increase validity and accountability. This process gives academics and practitioners the power to collectively assess and evaluate knowledge products. Fundamentally, these efforts are about reviving higher education and critical research as part of a democratic public sphere that is open, inclusive, and relevant. We conclude by reflecting on the value of incorporating community perspectives into the peer review process. We also offer recommendations on how to recognize and incorporate community knowledge and experiences into assessment structures.

Scholarly peer review can be broadly described as the evaluation and assessment of research by qualified members of a particular academic field for the purposes of publication in academic journals, books, or conference proceedings.Originally used in the 1700s, it was not until the midtwentieth century that it became commonplace in academic work (Benos et al., 2007).Today, peer review is generally understood to be an essential and indispensable process to maintain quality and rigour in scholarly research (Benos, et al., 2007;Spier, 2002a;Ware, 2008).Further, accumulating publications in top-tier peer-reviewed journals has become a key indicator of a researcher's credibility and is essentially "the currency of career advancement" (Vosshall, 2012, p. 3590).However, some have questioned the value of scholarly peer review, arguing that it can be inconsistent, overly conservative, subjective, and biased (Ware, 2008).Further, critics suggest that unexamined assumptions surrounding the process result in a (re)centring of the university's power by favouring academic ways of knowing over and above other kinds of knowledges (Barreno, Elliott, Madueke, & Sarny, 2013;Cashman et al., 2008;Gelmon, Jordan, & Seifer, 2013).This paper builds on these debates by exploring the possibilities for engaging communities in shaping and assessing the value of knowledge.Specifically, we reflect on a community-academic peer review pilot project that sought to address some of these limitations by engaging communities impacted by research throughout a review process.
Our study is part of Community First: Impact of Community Engagement (CFICE), a seven-year action research project that aims to strengthen the ability of non-profit organizations, universities, and funding agencies to build more successful, innovative, resilient, and prosperous communities.Launched in 2012, CFICE explores ways that community-campus engagement can be designed and implemented to increase the value for non-profit community-based organizations.Through CFICE, a range of academic and community partners worked closely with Food Secure Canada (FSC) 1 to strengthen new and existing relationships around social, economic, and ecological justice in relation to food systems.Between 2016 and 2017, CFICE collaborated with FSC and the Food: Locally Embedded, Globally Engaged (FLEdGE) action research network2 to develop a series of discussion papers presenting a scan of food policies across Canada.As co-leads on the CFICE project along with members of FSC and FLEdGE, Charles Levkoe and Amanda Wilson were part of a team of community practitioners and academics that initiated the research for the discussion papers and their assessment.
Drawing on our collective experiences and reflections from the community-academic peer review pilot project, we argue that the perspectives of community-based practitioners are a necessary part of peer review-especially for Community-Based Research (CBR)-to increase validity and accountability.This process goes beyond member checking or simply sharing results with participants; it gives academics and practitioners the power to collectively assess and evaluate CBR knowledge products.Fundamentally, these efforts are about reviving higher education and critical research as part of a democratic public sphere that is open, inclusive, and relevant.
In the next section we provide a general discussion of intentions versus results of the peer review process.We then provide a more focused discussion on some limitations of using peer review for assessing CBR; and in turn, how CBR can be limited by the demands of this process.We also highlight attempts to establish alternative peer review models.We then present an overview of the community-academic peer review pilot, followed by a discussion of key learnings.In the conclusions, we reflect on the opportunities and challenges of incorporating community perspectives into the peer review processes and offer recommendations on how to recognize and incorporate community knowledge and experiences into the assessment structures of research quality.
1 Food Secure Canada is a pan-Canadian social movement organization.see https://foodsecurecanada.org 2 FLEdGE is a multi-year partnership project funded by the Social Sciences and Humanities Research Council.It is a CBR and knowledge sharing partnership committed to fostering food systems that are socially just, ecologically regenerative, economically localized and that engage citizens.See https://fledgeresearch.ca/

Intentions and Limitations of Scholarly Peer Review
Within an academic context, peer review is the primary means through which particular research, arguments, and by extension their authors, are validated (Burnham, 1990;Vosshall, 2012).Through this process editors and reviewers aim to assess the soundness, significance, and originality of the research (Benos et al., 2007).One of the key features of peer review is that the evaluation is undertaken by scholars with familiarity of, or expertise in, the specific topic area.In a typical peer review process for an academic journal, an editor pre-screens manuscripts and selects reviewers to conduct the evaluation.The reviewers then provide feedback and either recommend the manuscript for publication, reject it, or propose a series of revisions for the author to undertake.Considering the reviewers' feedback, the editor makes a final decision whether the manuscript is ready for publication.Peer review is intended to act as quality control with the intention to "ensure that the valid article is accepted, the messy article improved, and the invalid article rejected" (Gelmon et al., 2013, p. 1).This process also gives authors an opportunity to correct errors or flaws in their logic before their work reaches the public domain (Benos et al., 2007).In a study exploring the experiences and perceptions of senior authors, reviewers, and editors, the vast majority of respondents supported the peer review process and reported they felt that it improved the quality of published papers (Ware, 2008).
While peer review has been touted as indispensable (Kassirer & Campion, 1994), and reviewers described as "sentinels on the road of scientific discovery and publication" (Benos et al., 2007, p. 145), some have argued that it is sustained on the belief that it works, rather than on evidence (D'Andrea & O'Dwyer, 2017;Smith, 2006a).Assuming it is done well, Lock (1994) writes, "all that peer review can reasonably do is detect major defects of originality and scientific credibility, together with commenting on important omissions, the rigor of arguments, and defects in the writing style" (p.60).Many critics point to the subjective nature of peer review and the inevitability of bias and inconsistencies (Gannon, 2001;Kassirer & Campion, 1994;Souder, 2011).For example, the personal opinions of editors and reviewers (along with undisclosed conflicts of interest) have been shown to support specific kinds of arguments and journals (Benos et al., 2007;Smith, 2006a;2006b).While editors and reviewers can decide whether a manuscript is a good fit for their specialized discipline and audience, there is no single objective measure nor agreed-upon definition of what constitutes a "good" paper (Figueredo, 2006;Smith, 2006a).Others have questioned the normative, epistemological assumptions that are reinforced by scholarly peer review processes (Jefferson, Alderson, Wager, & Davidoff, 2002).For example, peer review has been described as a tool of scientific conservatism, lacking tolerance for alternative perspectives, and new or unconventional ideas (Atkinson, 2001;Shimp, 2004;Souder, 2011;Spier, 2002b).
Critics have also expressed frustrations with the process of scholarly peer review.Long turnaround times can significantly delay publication and the dissemination of valuable information and ideas.Claims that the peer review process lacks transparency also raise concern about reviewer accountability (D'Andrea & O'Dwyer, 2017;Derrick & Pavone, 2013;Kassirer & Campion, 1994).Further, finding willing reviewers that have no conflicts of interest and are experts in the required field in a timely manner can be extremely difficult (Elden, 2008).The number of scientific journals and published articles increases by about 3.5% each year, and the need for reviews grows exponentially (Kovanis et al., 2016).In many cases, reviewers are able to determine the identities of authors based on their knowledge of the field, thus raising questions about the anonymity of blind peer review.According to Kovanis, Porcher, Ravaud, & Trinquart (2016), while the supply of available reviewers may be sufficient to meet the rising demands, the burden is actually assumed by a small, disproportionate few (i.e.20%) that complete that vast amount of reviews (i.e.69% to 94%).The pressure to complete thorough reviews that adhere to publishing timelines is demanding, and reviewer burnout may be a factor in peer review inadequacies (Benos et al., 2007).While journal publications have become currency in the knowledge market, the incentive to provide reviews-especially robust and thorough reviews-is much weaker (Katwyk & Case, 2016).

Peer Review and Community-Based Research
Stemming from these general critiques, there are particular challenges that arise when CBR comes up against the scholarly peer review process.Derrick and Pavone (2013) claim that there is a "disjunction between the research that society needs and the research being promoted as 'excellent' by peer review committees" (p.566). 3Where the scholarly peer-review process defines the relevancy of research as it applies to the journal's specialized discipline and audience, CBR typically defines research relevance in response to a particular community's needs; that is, the discipline of study is fluid and dynamic.Furthermore, what constitutes "good research" in CBR may differ from other academic perspectives.For example, markers of high quality CBR (e.g., relationships built, addressing a community's ethical concerns, meeting community needs) are often overlooked in favour of academic debates or more objective or easily quantified measures (Gelmon et al., 2013).In addition, sharing findings that emerge from CBR does not always fit the typical structure of a scholarly research article.Researchers under pressure to publish their work in peer reviewed journals are often forced to make a range of compromises such as using disciplinary jargon, decontextualizing the findings, and sharing their research in proprietary journals owned and controlled by large publishing corporations (Gelmon et al., 2013).
The need to consistently defend CBR methods and knowledge products and duplicate findings in peer review friendly formats can disincentivize scholars from doing this type of research (Foster, 2010).This is particularly the case for untenured faculty who see CBR as "too professionally risky" (Calleson, Jordan, & Seifer, 2005; also see, Katwyk & Case, 2016).Even among academic institutions that have embedded community-engagement into their mission statements and strategic research plans, the growing expectations have not been matched by necessary institutional supports for this type of scholarship (Barreno et al., 2013).In general, academics face increased institutional pressure to focus on research and publishing, often at the expense of teaching and service to the community (Calleson et al., 2005).By extension, the pressure to preserve one's career through the publication of peer review articles (re)centres the academic institution at the expense of the community (Katwyk & Case, 2016).
Subjecting CBR knowledge products to the scholarly peer review process also awards a level of power and authority to academic reviewers who may not have prior experience with CBR.Further, enabling anonymous reviewers to evaluate and assess CBR often contradicts the values and intentions of CBR processes (Castleden, Sylvestre, Martin, & McNally, 2015;Wright, Lemmen, Block, & von Unger, 2008).The assumed expertise of academic researchers privileges the status of the university as being "more true, more real, more rational" while marginalizing other experiences and ways of knowing (Biesta, 2007, p. 471).Situating academics as experts above those directly involved in and impacted by the research reinforces inequitable power relations and runs counter to the core values of CBR, which includes mutual learning and the co-production of knowledge (Castleden et al., 2015).In this way, the scholarly peer review process fails to recognize and account for the expertise of individuals directly involved in the research.

New Trends in Peer Review
In the section above, we have pointed to a series of limitations of the peer review process in respect to its reliability as a regulatory system for quality control, as well as more specific issues that arise when peer review is applied to CBR.These critiques have also spurred a conversation on the need to reimagine the process and principles of peer review.Across academic disciplines, new models of peer review are being explored and employed.Two prominent examples include open peer review and selective community-review models.In this section, we review some examples of these trends, highlighting both opportunities and limitations.

Open Peer Review
Responding to critiques of scholarly peer review-namely a need for transparency and reviewer accountability-a number of academic journals have experimented with open peer review.Open peer review is a term used to refer to a number of different features: disclosing the author's identity to reviewers (single-blind), vice versa (unmasking), or both; documenting the pre-publication history alongside articles; and/or, inviting experts beyond those conducting the initial review to provide feedback (Ross-Hellauer, 2017).
Atmospheric Chemistry and Physics is an open access journal that uses a form of open peer review. 4Submitted manuscripts are first reviewed by the editor, then posted for eight weeks in an open discussion forum.This "interactive public peer review process" allows for designated reviewers (anonymous or identified) and other members of the scientific community (identified) to provide feedback.When a revised manuscript is submitted, a coeditor makes the final decision whether it will be accepted or rejected.Accepted papers are published with their review histories, and rejected discussion papers are also archived online.In another example, the Hybrid Pedagogy Journal uses an interactive peer review process between authors and reviewers for each of its manuscripts. 5This process allows anyone in the journal's "community" to comment, build, and revise manuscripts together.In addition, comments can be made on the manuscript after publication.These examples of open peer review demonstrate efforts to increase transparency and accountability, albeit within the confines of scholarly peer review.
Despite the success, open peer review models have faced some distinct challenges.For example, some report that producing reviews that will be public and open to scrutiny can be "demanding, delicate, and difficult" (Perakasis et al., 2017, p. 5).There are also concerns about reviewers feeling censored if their identities are known (Mandernach, Holbeck, & Cross, 2015).More specifically, knowing the identity of the reviewer and/or the author can broaden power dynamics that may bias the quality and conclusions of the review (Armstrong, 1982;Spier, 2002a).In addition, it can be demanding to keep up with the task of assessing and reassessing a manuscript.Studies report that it can take much longer to complete open review processes, and they have higher rates of declination of requests for reviews (Van Rooyen, Godlee, Evans, Black, & Smith, 1999;Walsh, Rooney, Appleby, & Wilkinson, 2000).

Models of Non-Academic Review
Some academic journals and research forums have attempted to include the perspectives of non-academics to influence the evaluation of knowledge products; that is, to extend the concept of "peer" in scholarly peer review.One example is Research Involvement and Engagement, established as a "co-produced journal" reviewed by both academics and healthcare patients. 6he open access journal is described as "an interdisciplinary, health and social care journal focus[ed] on patient and wider involvement and engagement in research, at all stages" (Research Involvement and Engagement, n.d.).One of the editors-in-chief describes the value of this kind of joint peer review: "We wanted to send a signal to the community that active collaboration [between academics and patients] is a vital part of high-quality research" (quoted in Chawla, 2014).All submitted manuscripts must also include a plain language summary to ensure it is accessible and useful to the general public.
The British Medical Journal (BMJ) has also created a role for patient reviewers with specific guidelines for the process. 7In 2014, the journal launched a patient partnership strategy, establishing a "commitment to improving the relevance and patient centredness of its research" (The BMJ, n.d.).Patient editors were added to the editorial staff and patient peer reviewers could register online and have articles electronically sent to them for review.Notably, BMJ's patient reviewers are not expected to provide an evaluation of a paper's scientific reliability or originality but are invited to provide feedback on issues within their experience or specific interest.The journal also allows for public comments to be made on articles post-publication.
Another example of community peer review is the Community-Engaged Scholarship for Health website (http://CES4Health.org)that was established as a platform for health-related CBR products other than journal articles (ex.videos, toolkits, and policy briefs)-which are usually excluded from academic peer review processes-to be collaboratively peer-reviewed and disseminated by academics and community practitioners.In this model, individuals apply to become reviewers and are trained to evaluate knowledge products using a predesigned set of review criteria.A study of the model found that it added significant value to CBR products, supported academics in promotion and tenure processes, and provided useful resources to address community health concerns (Jordan, Gelmon, Ryan, & Seifer, 2012).
In relation to food systems-themed journals, the double-blind peer review processes of the Journal of Agricultural, Food Systems and Community Development draws on a range of food systems professionals in addition to academics and researchers for peer review.Established in 2014, Canadian Food Studies/La Revue canadienne des études sur l'alimentation (CFS/RCÉA) hosts a section on its website that allows for authors to submit articles for community peer review.The website states, "In an open access journal such as CFS/RCÉA, for which the audience spans academics and practitioners, a peer review process that facilitates constructive feedback from all engaged parties may break new ground for academic publications on policy and community relevance frontiers" (CFS/RCÉA, n.d.).Despite the initial enthusiasm of this section, to date, only one article has been submitted and no feedback has been posted.
Despite some creative attempts, there is little research or reflection on the benefits and limitations of non-academic peer review and whether or not these process have generated higher quality research and/or more community engagement.To fill this gap, we describe a specific case of piloting a community-academic peer review process working with academic and community partners in Canada's food movements, followed by a reflection on the lessons learned from our approach.

Piloting the Community-Academic Peer Review Process
In the context of this ongoing discussion in the literature and experimentation in peer review practice, the community-academic peer review pilot was established in 2016 to develop a process that would evaluate a series of discussion papers jointly produced by Food Secure Canada, Community First: Impact of Community Engagement, and Food: Locally Embedded, Globally Engaged.FSC has a long history of collaborating with academics.The organization itself evolved from relationships among scholars, practitioners, and communitybased researchers who recognized the need for a national level organization to mobilize and give voice to Canada's growing food movement networks (Levkoe, 2014).The objective of the discussion papers was to report on an environmental scan of existing food policies in Canada organized around six critical themes: Sustainable Agriculture, School Food, Local and Sustainable Food Systems, New Farmers, Indigenous and Northern Food Sovereignty, and Food Security.The themes were identified through in-depth consultation with community practitioners and academics in anticipation of the Federal Government's commitment to develop a national food policy.First announced in 2015 through a mandate letter to the new Minister of Agriculture and Agri-Food, official consultations to develop a Food Policy for Canada eventually began in May 2017, with an expected release in 2019.The discussion papers aimed to mobilize knowledge and experience through collaboration among researchers, civil society and policy makers.The scan involved a review of existing policy documents, relevant scholarly and grey literature, and interviews with key food movement practitioners.Research was primarily conducted by a Master's student, in collaboration with an academic and CBR team during the summer of 2016.From this research, six themed discussion papers were developed and were accompanied by policy maps and summary tables. 8Together, these knowledge products were intended to encourage conversation on building a national food policy able to address the inter-related issues of hunger, health and sustainability; and to build capacity for FSC and the food movement it represents to be meaningfully engaged in its development.
After the discussion papers were drafted, the research team agreed they would benefit from a more thorough assessment and evaluation before being shared more broadly.A scholarly peer review process was not possible since the discussion papers were not being submitted to an academic journal, nor were they structured in a traditional scholarly format.Further, since the research was informed by the priorities and experiences of community-based food organizations, the research team had little interest in the cumbersome process of academic peer review (and many predicted it would be unhelpful).However, they wanted to ensure the discussion papers were accurate and rigorous as well as speaking directly to the experiences of both researchers and practitioners involved in food systems policy work.In response, the research team developed and piloted a community-academic peer review process to generate critical feedback from multiple different perspectives, integrating elements of both emerging peer review trends discussed earlier in this paper: open peer review and community-based reviewers.
To find community and academic peer reviewers, the research team reached out to key individuals through the FSC, CFICE, and FLEdGE networks.As described in the literature, successfully identifying and confirming peer reviewers can be a major challenge, especially when attempting to recruit non-academics.Most of those agreeing to participate in the community-academic peer review pilot noted their support for the work of FSC and the other action research networks.When reached, potential reviewers were informed not only about the discussion papers to be evaluated, but also about the broader collaboration and efforts that aimed to contribute to the development of a national food policy.This provided a justification of the need for the community-academic peer review and context to conduct the evaluation of the discussion papers.Reviewers also received an explanation of the open peer review process, which was reported as especially helpful to community practitioners unfamiliar with the process and purpose of peer review (see Appendix A).
In total, eleven individuals were contacted and eight agreed to participate in the communityacademic peer review pilot.Some reviewers were not identified strictly as academic or community participants but instead embodied a hybrid position.For example, in one case an academic reviewer had worked for many years with a community-based organization and only recently returned to complete post-graduate work.In another case a community-based reviewer from the non-profit sector held a PhD and frequently collaborated on academic research projects.Beyond the binary of community-academic, there was also considerable diversity in the backgrounds of community reviewers.Within the broad category of community practitioner, individuals had different levels of familiarity with academic research and peer review processes.This shaped how each individual approached the peer review process, influencing the kinds of comments they made as well as how they evaluated the utility and impact of the work.Of the three categories of reviews (academic, hybrid, and community), hybrid reviewers had the highest response rate (see Table 1).In addition to the list of reviewers that were invited to participate, many others were rejected as a result of pre-existing commitments or conflict of interest.Each reviewer was provided with a review template containing a series of questions to consider in their assessment and space to provide both qualitative and quantitative assessments (see Appendix B).Given that reviewers came from diverse contexts and perspectives, it was important to provide a standardized set of questions to encourage a level of consistency and comparability among the individual reviews.In responding to the questions, reviewers were prompted to evaluate the research and analysis not only in an abstract sense, but grounded in the realities and context of their own knowledge and experience-whether in research, policy, or front-line community work.This was particularly important for the discussion papers because their contributions went well beyond academic literature on each topic and they were intended to be useful to policy and program work in the broader community.Despite attempts to standardize feedback, there was significant diversity in the responses from reviewers.Most were extremely supportive and generative in their comments.Several individuals outlined substantive revisions or additional issues and questions to consider, but most included supportive comments speaking to the importance and value of the research undertaken.In particular, the community and hybrid reviewers brought suggestions on how the research would play out in a more practical sense.For example, one reviewer questioned the use of the word "capital" in the New Farmers discussion paper to refer to the financial resources required by farmers to establish a farm.They asserted that this was a value-laden term connected to debt financing, something they believed was quite harmful to farm viability.Another community-based reviewer suggested upcoming policy openings that the author might want to reference in the discussion paper on Community Food Security.One of the reviewers for the Northern and Indigenous Food Sovereignty Paper commented that it was important to keep the paper brief, knowing that its intended audience was not strictly academics.

Successes and Limitations of the Community-Academic Peer Review Pilot
Overall, the community-academic peer review pilot project contributed significantly to the final discussion papers.It provided an important platform to engage both academics and community-based practitioners in the co-creation of the knowledge products.In this section, we identify some of the primary factors that made this pilot successful and some of the challenges that arose.First, the research team realized early on that if non-academics were to be involved as peer reviewers, the process needed to be as straightforward and relevant as possible.While conducting peer reviews is seen as part of regular work for most academics, community practitioners are not generally included in these types of activities.To participate, they are required to negotiate the allocation of work hours, and in most cases take on these kinds of additional responsibilities in a volunteer capacity.The research team considered paying community peer reviewers, although there were insufficient funds and the literature suggested there may be limited benefits (see, for example, Ware, 2008).The research team also recognized that the review process needed to be authentic if community peer reviewers were to be engaged.In other words, the feedback needed to be taken seriously and applied to the further drafts of the discussion papers as a way to demonstrate the value of community perspectives.Further, inviting a community practitioner to contribute to peer review required that they understood the value of the processes of generating and evaluating new knowledge as well as the outcomes.Beyond simply a recognition of these realities, the research team made significant efforts to accommodate and support all reviewers throughout the process.
A second enabling factor was that the community-academic peer review pilot was not a stand-alone initiative.Rather, it was embedded within a broader context of communityacademic collaboration.In this case, the pilot was an integral part of Food Secure Canada, Community First: Impact of Community Engagement, and Food: Locally Embedded, Globally Engaged work, which also provided access to an existing network of potential community and academic peer reviewers.Engaging with these networks also brought a sense of legitimacy to the process and provided reassurance that the discussion papers were more than just an academic endeavour.Third, working with FSC and the broader community-academic collaboration, the pilot project benefited from adequate capacity to broker the community-academic peer review process.Specifically, the research team was able to support staff time directed at coordinating the peer review process and ensuring there was a point person throughout the course of the project to assist reviewers and to help ensure that feedback could be adequately interpreted and addressed.
Despite the overall success of the community-academic peer review pilot, there were several challenges encountered.The first major challenge was the time required to oversee and coordinate the peer review process, a point also discussed in the literature above.While individuals had the capacity to participate during the pilot, we question the replicability and long-term sustainability of this type of engaged process.As with a scholarly peer review process, sufficient time and resources need to be dedicated to identifying potential reviewers, following-up with reminders and then working with the author to incorporate the feedback.This administrative burden is perhaps even more pronounced with community-academic peer review, because the reviewers come to the process with a diverse set of experiences and circumstances that need to be supported, authenticated, and incorporated.As long turnaround times is one of the oft-cited limitations of academic peer review, the fact that our pilot reproduced this element only further emphasizes the importance of administrative support and capacity to ensure knowledge outputs are disseminated in a timely fashion.The model of the Community-Engaged Scholarship for Health, discussed above, where community reviewers are provided with training on peer review processes may be an instructive model to replicate, provided there are sufficient resources.
Another challenge, which is not unique to the community-academic peer review process, was encouraging invited peer reviewers to respond to the requests.In some cases the provision of feedback was not particularly relevant or helpful in revising the discussion papers.This was true most often with reviewers that were unfamiliar with research-oriented peer review processes.This challenge points to the value of working with hybrid reviewers with some background in both community and academic environments.Finally, in designing the community-academic peer review process, there were few existing models and little experience to help develop the pilot.Drawing on the existing literature and models, the research team was forced to improvise and adapt as the review process took shape, learning as they went.

Lessons from the Community-Academic Peer Review Pilot
Including the perspectives of community-based practitioners in peer review is an essential part of bringing increased validity and accountability to this process.As demonstrated through the community-academic peer review pilot project, the process gives academics and practitioners the power to collectively assess and evaluate CBR knowledge products.This power is especially important for those committed to movement building, as it brings increased relevance, validation, and accountability to the efforts of community-based researchers, practitioners, and academics.Lessons from the literature review show that strong relationships, essential to CBR, are typically underappreciated by the traditional academic peer review process.The pilot highlighted the importance of building and maintaining ongoing relationships of mutual benefit between community and academic partners.Agreeing to participate in this review process was not strictly a one-off request; it was contextualized within a broader ongoing relationship between FSC and a range of community and academic allies.Having FSC as a key partner in the research and subsequent peer review process provided a level of credibility and relevance to community practitioners that an academic journal seeking community input may lack.Likewise, involving academic research networks signaled that the discussion papers and the feedback would be held to a high standard.In many ways, it was the strong relationships and collaborative nature of the research that enabled the success of the community-academic peer review process.It is important to note that these relationships were built over many years with significant cross-over between academia and community participants.
Reflections on the community-academic pilot suggest that there was particular value in working with hybrid reviewers.As noted above, it was the hybrid reviewer category that had the stronger response rate in terms of securing reviewers.These individuals are perhaps also best placed to meaningfully contribute to these types of community-academic peer review processes.There is great value and insight in their ability to straddle the boundary of community and academic epistemologies, and to appreciate the needs and priorities of both community and academic voices in the peer review process.The idea of hybrid reviewers further contributes to this diversity, as they represent a blurring of lines and challenging of silos between the community and the academic.Even those reviewers who we categorized as academic might actually identify as hybrid, as many of them are deeply involved in food systems work outside of the university.However, hybrid reviewers should not replace community voices altogether.Front line food systems workers, for instance, and those with lived experience of food insecurity have particular perspectives that should also have the opportunity to evaluate and assess research knowledge products.However, given that there are concerns with reviewer burnout within academic peer review, and that much of the labour of reviewing is completed by a relatively small group, it is likely that these hybrid reviewers would receive an unsustainable number of review requests, should community-academic peer review models be widely adopted.
There is not (and should not be) a universal standard to what makes a good communityacademic peer review, as community practitioners have different research needs.Providing a platform to express these differences is an important way to recognize and value different ways of knowing.For some, a theoretically dense article has great value; and for others, anything more than a plain-language summary and set of policy recommendations has little use.Like many of the reviewers participating in the community-academic peer review pilot, the documents under review were also hybrid knowledge products: not strictly academic, but still a product of rigorous inquiry, research, and analysis.By the same token, there is a diversity of knowledge products, such as videos and other creative media, and policy briefs (for examples, see http:// CES4Health.org, as mentioned in the literature review), to disseminate CBR research beyond traditional academic articles.Although these can be well-researched and created with rigour, they are largely excluded from academic peer review processes and thus are generally seen as less valuable forms of knowledge or analysis.They may be very valuable and relevant to the community affected by the research, but inadmissible as scholarly products because they do not take the form of a conventional journal article.A peer review process that embraces flexibility and subjectivity, whether strictly academic or community-based, can be an added strength to the peer review process in general.While scholarly peer review has been shown to reinforce conservative ideas and privilege academic knowledge, community-academic peer review challenges assumptions and singular ways of knowing and presenting knowledge.CBR knowledge products can be enhanced by incorporating different and sometimes contrary perspectives and welcoming unconventional formats.Participants in the pilot helped improve the final discussion papers but also contributed to strengthening community-campus engagement and enriching scholarship within the Canadian food movement.
Reflecting on the community academic peer review pilot raises concerns regarding the exclusion of community input into the evaluation of knowledge production products, especially those that involve CBR.As highlighted in previous sections of this paper, this is often a disjuncture between the evaluation criteria for CBR and traditional academic research more broadly.Incorporating community perspectives into peer review processes is an important means through which to address these issues; however, important questions remain around the ultimate intention behind postsecondary education and research.If research is publicly funded, how do we ensure the public is the beneficiary?In Canada, most faculty and their institutions are funded, in part, by public monies distributed through government contributions to public institutions and government research grants through the Tri-Councils.It is extremely problematic that academic research is often conducted about or with community, yet the ultimate assessment and evaluation of the resulting knowledge products exclude these same groups.Finding ways to ensure that research is part of a democratic public sphere and that it is open, inclusive, and relevant should be of fundamental importance, especially for community-academic partnerships.Community-academic peer review is one way that research could be more accountable to the public.
One way to encourage these practices at an institutional level would be for academic promotion and tenure committees to recognize the value of community-academic peer review.Peer review validates research, but it also validates researchers.CBR scholars are doing work that is founded on principles of mutual contributions and the co-creation of knowledge with communities.The academic promotion and tenure system is based on rewarding individuals for their contributions in the form of peer review articles, at times creating a conflict of interests for researchers.Individuals involved in the research and the affected communities that have a refined and relevant set of real-world expertise, should be recognized as, and considered peers in this research quality assurance process.
Researchers conducting CBR and publishing their work are subject to the peer review process.If this process determines one's ability to secure funding and tenured positions, and in turn impacts their ability to sustain relationships with their community partners, then antagonistic characteristics of the process need to be revised and alternate merit assessment tools should also be introduced.By the same token, the contributions of community reviewers should also be recognized and compensated-though, not necessarily monetarily.Without providing some sort of incentive or compensation for community practitioners to engage in these processes, it is important to remain modest in one's expectations for community involvement.Furthermore, the tension of publishing for the sake of benefiting one's career versus for the sake of impact could be reduced if the peer review process included nonacademic peers.This could also ease tensions for academics as it would simultaneously balance the requirements of their academic careers by recognizing CBR as a collaborative process and moving towards more collaborative models of evaluating and reviewing research knowledge products.

Conclusion
The issues raised in this paper elucidate the need for postsecondary institutions not just to respond to public interests and societal ills, but also, more importantly, to listen and work towards collaborative solutions.Peer review is a major part of the research dissemination process, determining what gets published and what does not.That is, it mediates the conversations academics have with each other, with communities under study, and with the public.As such, there are elements that can be changed in the knowledge validation process to make it more receptive to voices and perspectives that come from outside the academy.Fundamentally, this approach demands a two-way conversation in place of a knowledge-deficit model.In other words, it means not just studying and educating community, but engaging community as full participants and co-creators.
Clearly, these issues go well beyond peer review, and are part of ensuring democracy and equity in knowledge production.Scholarly peer review is a process embedded in a Eurocentric, positivist epistemology that values certain kinds of knowledge over others.The value of community-academic peer review processes is not just about bringing community perspectives into the academic context, but about challenging relationships of power in knowledge construction and validation more broadly.This process goes beyond member checking or simply sharing results with participants; it gives academics and practitioners the power to collectively assess and evaluate each other's research.Even beyond peer review, community members should have the opportunity to be involved in the process of formulating research design from the outset to ensure questions are relevant and methodologies are sound and ethical.
However, action can also be taken within existing structures of academic peer review to bring immediate improvements alongside longer-term efforts to re-shape and re-imagine public institutions of higher education.As a first step, we encourage academic journals to involve relevant community-based researchers and practitioners in their governance structures and pool of potential reviewers.In the case of peer review processes that rely on suggested reviewers from authors, this could be accomplished by adding a prompt for authors to provide suggested reviewers from both academic and community contexts.Given that one of the existing challenges with academic peer review is attracting sufficient and suitable reviewers, this practice may help address multiple issues at once.Journals could also ask reviewers to evaluate the level, if any, of engagement with the communities under study in manuscripts under review.These changes would not radically alter the power dynamics in academic knowledge production and dissemination, but they could encourage community-academic collaboration and acknowledge the indispensable role community can play in knowledge co-creation.
Fundamentally, these efforts are about reviving higher education and critical research as part of a democratic public sphere that is open, accessible, and relevant.As indicated above, one of the core challenges of developing the community-academic peer review pilot was that there were few models and examples to draw from.Thus, we offer these learnings to others interested in experimenting with collaborative assessment and evaluation processes.We also encourage others to share their experiences in an effort to develop new and better ways of doing community-campus engagement.Ideally, this will also help produce new tools and mechanisms to further encourage and support these processes, particularly in the social sciences.
Community-academic peer review is not the only means of incorporating community perspectives into academic research, nor should it be.Indeed, a host of mechanisms should be explored to further democratize the practice of research and the processes through which particular conclusions and perspectives are deemed valid.The community-academic peer review process should not be seen as a stand-alone mechanism to bring community voices into the production, validation, and dissemination of research.Rather, it is one tool of many that is best utilized alongside other means of valuing and prioritizing the active participation and empowerment of community perspectives.

About the Authors
Charles Z. Levkoe (corresponding author) is the Canada Research Chair in Sustainable Food Systems at Lakehead University.His research focuses on social movement organizations and the growth of regional food networks in Canada.He has been active in investigations at the intersections of food movements and community-campus engagement.Email: clevkoe@ lakeheadu.caVictoria Schembri is a Master's candidate in Health Sciences at Lakehead University.Her research centres on food systems in remote, rural, and Indigenous communities in Canada's provincial North; particularly, exploring sustainable actions at the community level that build food security and food sovereignty.Amanda Wilson is an Assistant Professor in the School of Innovation at Saint Paul University in Ottawa.Her research is focused on food movements and alternative food networks, collective forms of organizing, and questions related to prefiguration and enacting a politics of possibility.This peer review process in particular is meant as a sort of 'check-in' with a community of practice (both academic and community-based), to ascertain whether the arguments and analysis of a particular author or group of authors resonates with, and is reflective of, the shared experiences and realities of that broader community.Our approach to peer review is one of collaboration and mutual support.It is an opportunity to gain additional insights, identify critical points of reflection and highlight potential areas of continued debate and discussion.

Process
This is an open review process, meaning that both the author and reviewer know the names of one another.When a reviewer's assessment is forwarded to an author, it normally includes the reviewer's name.Please let us know if you prefer to remain anonymous.
Recognizing that community organizations (as well as academics) often have limited time and resources to devote to these kinds of activities, we have developed a template with guiding questions, in an effort to streamline the process.Reviewers can also arrange to provide feedback through a phone interview.
Timeline: Should you accept the peer review invitation, we ask that you complete your assessment within one month of receiving the document.

Instructions For Reviewers
In-text Comments Reviewers can suggest edits, comments or feedback within the text of the document.This is not meant to be a copy-edit (though you are welcome to highlight any typos or grammatical errors), but rather to highlight passages that are unclear, or specific questions that arise from a particular point of analysis or piece of information.We also welcome additions and suggestions that will help strengthen the analysis.

Overall Recommendations and Feedback
Through the accompanying Reviewer Template you will be asked to respond to a series of questions to evaluate the content, style and structure of the document.