The Career Column: Publishing as a Consultant:
Challenges and Strategies
Lynn A. McFarland
George Mason University
One of the most common reasons I hear from students for choosing applied over academic jobs is that they dont want to spend the rest of their lives trying to publish. When I hear this, Im quick to point out that some of the most successful practitioners publish regularly. However, trying to conduct and publish field research while working in applied settings is not always easy. There are countless obstacles encountered when one attempts to collect field data and publish the results. So, why would a practitioner choose to pursue research opportunities? And, how do practitioners do it? To find answers to these questions, I interviewed a few I-O psychologists who have managed to publish while working a real job.
Four consultants with successful publication records agreed to speak with me about these issues:
Gary Carter (Personnel Decisions Research Institutes, Inc.PDRI), Mark Schmit (Gantz Wiley Research),
Josh Sacco (Aon Consulting), and Jeff Weekley (Kenexa). These individuals are in different stages of their careers but have all published applied research while working as practitioners. They have published in journals such as
Journal of Applied Psychology, Personnel Psychology, Academy of Management Journal, and
Human Performance, just to name a few. Clearly these individuals are in a position to offer some insights about publishing research while working as a consultant. In addition, their advice is not only relevant to practitioners; many of the issues they raise apply to anyone who attempts to publish field data.
Benefits of Conducting Research
Practitioners rarely receive formal rewards from their employers for publishing. However, presenting and publishing research can be a great way for an organization to increase its visibility. In addition, when bidding on proposals, one often must include bios and it may help to show one is involved in the field through research. Also, as Mark pointed out, his clients are often in the profession, and they are familiar with published research. Therefore, publishing is a way of demonstrating to his clients that his organization is on the cutting edge; this may make the difference between getting a clients business or not. Recognizing this, some organizations provide incentives for research activity to encourage practitioners to engage in such endeavors. For instance, consulting firms may consider research activity in the performance evaluation. This is considered professional development and can indirectly contribute to ones status in the organization. In addition, although rare, some organizations offer bonuses to those who present and publish research.
Practitioners also have personal reasons for wanting to conduct research. The knowledge that one is contributing to the profession can result in personal satisfaction. Certainly, seeing ones name in print is always welcomed and can help one achieve personal visibility, which may result in career advancement and more career choices.
Those interviewed also noted that research can benefit the client. The organization that provides data for a research project may learn some things from the information gathered that may be used to improve the workplace. This may help to strengthen and further the consultant-client relationship.
Now that its clear why practitioners may choose to engage in research, lets take a look at some of the challenges often faced by practitioners when conducting research. Then, well examine ways to overcome these challenges.
Challenges Faced by Practitioners Conducting Field Research
Initiating data collection. Most of those I spoke with indicated they really do not need to convince clients to allow them to collect data for research purposes. As Gary noted, typically, the research he seeks to conduct has applied value, so the client benefits from learning the results of such studies. However, in some instances, clients may be concerned about anonymity, particularly if the results are not favorable. In such instances, the researcher needs to make the applied value of the research clear (perhaps by reframing the research question to examine an issue of particular importance to the client) and do everything possible to ensure the clients name is protected (e.g., ensuring the organization is not mentioned in the paper).
Although most clients see the value in research and are willing to allow consultants to collect additional measures, the problem often faced is how much data the client is willing to collect. For instance, process or theoretical measures (e.g., motivation in a study examining applicant and incumbent differences in test performance; measures of cultural dimensions in a cross-cultural study), while important from a research standpoint, may not be seen as important to a client. Such measures may not only add to the length of data collection, but some of the questions asked may be considered sensitive or inappropriate. In such instances, the researcher has to determine how to best balance the clients concerns, the length of the measures, and the research questions. Changing the research question slightly may help, as this may change the nature of the measures that need to be collected and the time it will take to administer them.
Data collection. This is one of the more problematic aspects of the applied research process, for several reasons. First, scheduling data collection can be challenging. Employees already have so many demands on their time, its difficult to schedule data collection sessions that work for many people. Second, even if scheduling is possible, theres always the issue of how much time people are willing to take to participate in research. Therefore, its wise to limit the measures to the essentials. Third, attrition is a big problem. Many people may only participate in part of the study and fail to respond to some of the measures. This will cause obvious problems later when the data are analyzed and may make it difficult to publish the results.
To address these issues, the key is to limit the questions to only those that are essential. This will reduce the amount of time it takes for participants to respond to the measures and increase the likelihood theyll stay in the process. This means that researchers must give considerable thought ahead of time to the measures they need to include, in order to answer the appropriate research questions. This also requires them to stay current with the literature so they know what these questions are.
Another way to get around some of these problems is to use archival data. Then, theres no need to battle scheduling issues or to figure out how to find the time to do it. However, the use of such data does not allow for the assessment of process variables and problems of missing data still apply. Another potential problem is the data collection strategies the organization may have used. For instance, Josh pointed out that, when an organization validates a variety of tests, they may only administer 2 or 3 tests to each participant to save time. Such a data collection strategy can severely limit a researchers use of the data because it makes multivariate analyses nearly impossible.
The review process. Publishing field data can be difficult for anyone, not just practitioners. Here are some of the major issues faced by practitioners trying to get applied research through the review process.
First, a major criticism of many field studies is the lack of process measures. Consultants do not fail to recognize the importance of process measures, but its often very difficult to collect these kinds of data.
Second, a number of those I spoke with indicated they often get criticized for not having construct validity evidence for their measures. Keep in mind that consultants often create the measures their clients use. Therefore, most of these measures have not been widely researched. One way to get around this is to team up with an academic and administer the proprietary measure with well-known measures and do a construct validity study, often using college students in the lab. This takes more time, but if these measures are used in multiple research projects, the time investment would be well worth it.
Third, reviewers often suggest that the study should have used a particular methodology that is not feasible in field contexts. For instance, lets say a client wants to administer two tests and, because of time constraints the tests need to be administered on different days. For test security reasons, the client wants the tests administered in the same order to all applicants (i.e., all applicants take Test A on day 1 and Test B on day 2). Clearly, this is the best course of action from the clients perspective because this procedure ensures test security. However, a reviewer may see this procedure as limiting internal validity. After all, perhaps the order in which the tests were administered affects the study results. Since all applicants were administered Test A first, the practitioner has no way of addressing this concern. This is just one of the trade-offs that must be made when conducting field research. Therefore, practitioners must often face the dilemma of using procedures that will increase internal validity but may not be easily justified in applied contexts. As Gary is quick to point out, a practitioner must ultimately use procedures most appropriate for the client, as this is job one.
Finally, since field studies are just more messy than lab studies, there are a number of more minor issues that seem to arise with this type of research. For instance, when using archival data, its often difficult to figure out the precise data collection procedures used or the particulars about the sample. This can make reviewers very uneasy (and rightfully so). In addition, as mentioned earlier, its difficult to get everyone to participate so missing data is often a problem. Most of these issues are raised as concerns by reviewers and are generally difficult to address, particularly since the data has already been collected.
Finding the time. By far, the biggest challenge faced by those in consulting who engage in research is trying to find the time to do it. This is a challenge at every stage of the research process. As Jeff points out, there is little time to anticipate research questions or to do the necessary literature review. As a result, data that could have addressed an important research question is gathered, but because of lack of time to consider research issues, a key measure may have been excluded. Just a little more forethought could have resulted in the collection of publishable data, but this kind of forethought requires timesomething consultants are often short on.
Time is also a constraint when preparing the data for publication. As Jeff noted, writing a tech report and writing a research paper are very different. It is not possible to simply use the tech report as the basis for a research paper. Even once a paper is submitted for publication, theres little time to address reviewers concerns even if the response was favorable.
Why is there so little time for consultants to work on research? Most consulting firms support research in theory but do not allow consultants to take the time during the work day to spend on research. Therefore, consultants usually end up working on research on their own time (e.g., weekends or after work).
So how do consultants find the time to conduct and publish research? In the following section I review some of their strategies for addressing these challenges.
Meeting the Challenges
Those I spoke with offered a number of helpful suggestions for dealing with challenges that arise when trying to conduct research while working as a practitioner. Some of these suggestions apply to anyone doing field research while others are more specific to the circumstances of those working as consultants.
1. Show clients the value of applied research. If a client seems hesitant to allow you to collect data or use archival data for research purposes, explain to the client how this research will benefit the organization. Offer to write a tech report (free of charge) outlining the results and recommendations based on the studys findings. As Josh pointed out, most clients need ROI information on the services and products consultants provide. Therefore, they need to collect outcome data to justify costs. This provides consultants with an excellent opportunity to do applied research. When approaching the client about research, be sure to suggest these types of outcome measures be collected and the results shared with management to show how the expense was worthwhile.
2. Include clients as coauthors on papers that result from the research. This may be beneficial for two reasons. First, the client may want the visibility of a conference paper or publication and would be much more willing to allow you to collect data if someone from the company will also have his or her name on the paper. Second, if someone from the organization is a coauthor, this generally means you can expect more input from this person; thus making the client more likely to help you get what you need. This is also true if youre using archival data. The client should know exactly where the relevant data is and can answer your questions about the procedures used. Having a person from the clients organization as a coauthor will increase the likelihood youll be able to get this information because this individual will be personally invested in the study. This is a win-win situation because youre more likely to obtain useful data and the client gets name recognition by ensuring representation on the paper.
3. Before a project begins, think about interesting research questions. Just take some time to think about the data that will be collected and how these data might answer interesting questions. Is there another short measure that could be added that would make this research publishable? The addition of only one or two items may make all the difference, so try to give this some thought beforehand.
4. Do a cost-benefit analysis to determine what measures to use. Consultants must constantly make trade-offs when trying to collect data. As Mark suggests, time and money are two critical factors that a client will consider when deciding whether or not it is worthwhile to collect data for research purposes. Ultimately, the practitioner needs to think about which measures will be the biggest bang for his or her buck. What measures are interesting but require little time? Try to balance both research needs and practicality. Hopefully, this will result in the collection of data that can answer an interesting research question.
5. Be opportunistic. Given how little time consultants have to work on research, they need to be opportunistic. This means, instead of waiting or searching for data to answer a specific question, take a look at data you have or are about to collect and think about how those data may be used to answer interesting research questions.
6. Choose projects carefully. Not all data are publishable. Since time is limited, only choose research projects that answer interesting, high impact questions. In addition, choose projects where the data collection is clean. This means that there is little missing data, large sample size, and so forth. These characteristics will also make the data more publishable. In addition, focus on only one or two research projects a year. This way, its more likely youll get it done. The last thing you want to do is spread yourself too thin and end up getting nothing out the door.
7. Publish data from an on-going project. The benefit of doing this is that youre much more familiar with your data (as compared to when archival data is the basis of a study) and much of the data cleaning and analysis will already be done. Therefore, much of the work youre doing for the client will also contribute to getting the study data analyzed.
8. Use travel as an opportunity to work on research. Consultants often spend considerable time traveling for work. Theres little else to do on the plane when traveling to the destination or during the evenings in the hotel. Use this time to read articles or work on the paper itself. Since this is already time youre not spending with your family, you may as well use it as a chance to make progress on research.
9. Partner with academics. Most of those I spoke with suggested its a good idea to work with academics on research projects. Academics are directly rewarded for publishing and therefore are able to spend considerably more time on research activities. In addition, if you have a data set that addresses a question in a particular area, try to work with an academic who has a history of publishing research in that area. This way, you wont need to become an expert and read all the literature in that area (which you probably cant do given the limited amount of time you have to devote to such things). Another reason its a good idea to partner with someone in academics is because, since these individuals are conducting a lot of research (and are presumably submitting their work for publication), they have a very good idea of what issues reviewers may raise with the study youre working on. Hopefully, these issues can be pre-empted.
10. Have patience. As anyone who publishes will tell you, the research process can be frustrating. There may be problems during data collection, or you may get a particularly nasty review. When these problems arise, keep in mind that these snags are inevitable. Take them with good humor, and try to get past them as quickly as possible.
Conclusions
There are a number of obstacles practitioners may encounter when trying to collect and publish field research. Although at times frustrating, conducting research can yield tremendous benefits. Not only will you have the personal satisfaction of knowing youve contributed to the field, but there are also external rewards. The name recognition that comes from presenting and publishing research may lead to more business and increase your marketability. Hopefully, this article has provided you with a few tips that will make the research process go a little more smoothly.
July 2003 Table
of Contents | TIP Home
| SIOP Home