Variance: A Tool for Cultural Alignment
Lou Mischkind, Shawn Del Duco, Patrick Hyland and Joyce Chan
Many organizations conduct employee opinion surveys to boost morale and align members with the culture of the company. To this end, surveys are constructed with attention to those areas that are fundamental to a businesss success (e.g., customer service orientation, innovation, ethical standards).
Survey results are usually reported to individual units or teams as overall statistics, such as a percent of favorable responses or means for various items and indices. This allows each unit to assess its unique strengths and weaknesses. In addition, current results are often compared to prior survey results for that unit, as well as to higher level units (the company as a whole or the 2nd- or 3rd-level organization in the hierarchy). These latter comparisons utilize the concept of dispersion by showing how the unit stands relative to an overall statistic.
However, units and leaders seldom consider the concept of variance in the context of cultural alignment. How does the unit stand relative to other comparable units (between-unit variance)? How much agreement or disagreement is there among unit members (within-unit variance)? Both forms of variance are critical to any cultural alignment effort.
In this article we present a method of examining between-unit variance which has resulted in a better understanding of survey data and greater cultural alignment within organizations. We also demonstrate how within-unit variance adds value to the survey process. Finally, we present avenues of research we are currently pursuing and areas of future research that will allow practitioners to make even greater use of the variance concept in employee opinion surveys.
An effective approach for comparing unit-level survey data involves creating distributions of business units based on their responses to key survey items or dimensions. Percentiles are then calculated for each of these survey measures, allowing the practitioner to compare each units relative standing.
Examining between-unit variance is a quick and easy way to not only gauge overall survey results but also to assess the spread of responses across comparable units. A review of all of the business units allows the practitioner to identify those that performed exceedingly well or poorly and those falling in the middle. Actions are then taken to bring those low-performing units on-board with the rest of the units, thereby facilitating alignment with the overall culture of the organization. At this stage, top-performing units often become an excellent resource for those at the bottom. The value of drawing on real-life actions and solutions from units that have succeeded in the organization is immeasurable.
Between-unit variance analysis also facilitates a macro-level examination of the entire organization. Once low-performing units are identified, the demographic compositions of these units can be inspected to further isolate important distinguishing characteristics. The business function (e.g., Call Center Operations, Direct Manufacturing) and geographic location of a unit, as well as the management status, gender, and ethnicity of a units employees often impact unit-level survey results. Therefore, exploring between-unit variance can reveal flashpoints that otherwise might have been masked by only examining overall averages. In sum, between-unit analysis is an effective means for aligning low-scoring units and their members with the culture of an organization and has proven to be an effective way to yield noticeable survey improvements from year to year.
A large multinational organization administers a company-wide employee survey each year. The survey contains a variety of items and several dimensions that tap key aspects of their company culture.
Following a recent survey administration, we conducted a between-unit variance analysis of the companys results. A large portion of the organizations business units had at least one dimension in the bottom percentile. These leaders (and their superiors) were notified of their relative standing. They were provided with encouragement to improve on the less favorably rated dimensions and were supported by internal consultants and human resource specialists.
On a subsequent survey, a follow-up analysis was conducted to track change over time in employee ratings. While improvement among the low-performing units was expected, the magnitude of the changes was striking. Of the low-performing units in the previous survey, the majority scored above the bottom percentile on each of the current survey dimensions. Moreover, percent-favorable scores improved markedly across all of the dimensions in these low-performing units. In all other units, scores remained relatively unchanged.
Most importantly, focus on helping low-scoring units improve had a positive effect on the entire organization. Company-wide ratings increased significantly on survey items. Actions taken between the survey administrations signaled that management was serious about the survey effort and was willing to concentrate resources on bringing all units into alignment with the cultural values, principles, and practices of the organization.
Between-unit variance analysis also provides a sophisticated and relevant basis for developing training programs. The practices of high-performing managers and units can be isolated and contrasted with the low-performing managers and units to distinguish how they differ quantitatively and qualitatively (by examining written comments, interviewing the managers, etc.). The resulting information forms the basis of an organization-specific and culture-specific training curriculum that includes the strengths and the weaknesses identified by the between-unit variance analysis. This is a far superior alternative to off-the-shelf programs where one size fits all.
In addition to between-unit variance, within-unit variance represents an important dimension when analyzing survey results. A growing body of literature supports the utility of examining within-unit variance for organizational outcomes. For example, Schneider, Salvaggio, and Subirats (2002) found that climate strength, defined as the standard deviation of employee attitudes within bank branches, moderated the relationship between employee perceptions of service climate and customer satisfaction. In our own research, we have also found that within-unit variability plays an important role for organizational outcomes.
Within-unit variance moderated the relationship between employee perceptions and sick leave in a large governmental agency. Specifically, the relationship between ratings of management acting on employee ideas and sick leave was moderated by the extent to which units agreed on the aforementioned survey item. A low degree of within-unit variance yielded a stronger correlation, while units with high variance demonstrated a weaker correlation. All in all, these findings point to the importance of considering within-unit variability when examining survey results, particularly in regard to organizational outcomes.
Both climate strength and between-unit variance analyses are effective ways of analyzing employee opinion data. But combining the two approaches might provide the most actionable results. The moderating effect of within-unit variability described earlier suggests that all low- (or high-) scoring units are not the same. This presents direct implications for intervention strategies.
Specifically, managers can develop interventions with attention to not only how favorable or unfavorable employees are but how much agreement exists among the unit members. Managers whose units exhibit a high level of favorability and a high degree of agreement, for example, are likely to seek to maintain the status quo. On the other hand, those units with low favorability and high agreement may need to be re-invented.
Between-unit variance analyses and within-unit variance represent powerful approaches for examining survey data. By comparing unit-level ratings and variances, practitioners can make more efficient use of employee survey results and bring organizational units and participants into alignment with the overall company culture. We are presently conducting additional research to broaden the scope and further refine what we have labeled Variance Optimization, including (a) following up with clients who have implemented the between-unit variance approach to assess long-term improvements; (b) using different metrics in between-unit variance analyses; and (c) further exploring the antecedents and consequences of within-unit variance. In conclusion, all of the metrics of survey data distributionscentral tendency and between- and within-unit varianceneed to be considered in understanding survey data and optimizing the utilization of survey results.
Schneider, B., Salvaggio, A. N., & Subirats, M. (2002). Climate strength: A new direction for climate research.
Journal of Applied Psychology, 87(2), 220229.
April 2004 Table
of Contents | TIP Home
| SIOP Home