How cultural factors can affect 360° assessment results

This post was co-authored by Sophia Zhao and Kelly Hannum.

360° Assessments

Click here to learn about CCL’s Assessment Certification Workshop

360° assessment is widely used by organizations in the field of leadership development. It is also growing fast in China. The advantage of such a data collecting method is that a person can receive a holistic picture of his own behavior and performance, instead of information from a single source. To better understand how this method may, or may not, work in China, we reviewed relevant research. There are many factors that may potentially impact the accuracy of 360° assessments, but we focused specifically on how cultural factors impact 360° assessments in China.

Overall the 360° assessment method works well in China, but we also found evidence that culture has an impact on 360° ratings. The main areas to pay attention to include:

Self-rating Bias. Inflated self-ratings are more common in individualist cultures where personal achievement is emphasized. Consistent with the emphasis on modesty, people in collectivist cultures, like China, are more likely to rate themselves relatively low. The low self-rating is called the modesty effect.

Other-rating Bias. In a collectivist culture like China, relationship and harmony is emphasized; individuals are concerned about preserving others’ face. Hence, raters are more likely to avoid giving negative feedback, and their judgment is likely to focus on the positive traits instead of the negative ones. In a research paper published in the Journal of Applied Psychology, the authors found that individuals who hold collectivist values are more likely to provide more favorable ratings (creating something called a leniency effect) and also to provide ratings that do not differentiate the various dimensions of ratees’ behaviors (creating something called a halo effect).

Moreover, in a culture that emphasizes power distance, the degree of leniency and halo effects also varies depending on the rater’s hierarchical position. In a high power distance culture like China, subordinates may feel particularly uncomfortable rating their bosses and fear the consequences of providing negative feedback.  Subordinates are mostly likely to provide favorable feedback (leniency effect) on all the dimensions of behaviors (halo effect).

Self-other discrepancy. Self-other discrepancy refers to the inconsistency between self and other ratings in 360° assessment. Pioneer studies conducted in the US found that higher self-other discrepancies was related to lower leadership performance.

CCL researchers have examined this issue using data collected from thousands of leaders around the world. Overall the findings were mixed. There was no evidence that individualism/collectivism impacts self-other discrepancy. But assertiveness, the extent to which a culture emphasizes direct communication, may have an impact. In highly assertive cultures where people prefer direct communication, leaders have more feedback information on how they are perceived by their colleagues, and hence the self and other ratings tend to be consistent.

CCL researchers also examined the impact of power distance, and the results depend on the relationship between the rater and the ratee. In a high power distance culture like China, the self and peer ratings maybe consistent because leaders are most likely to seek feedback from peers. In contrast, the self and subordinate rating discrepancy may be high because subordinates may not have the opportunity to interact with the leaders and observe their behavior. There is no evidence of how power distance impacts self and boss ratings.

As mentioned previously, culture is not the only factor that influences self ratings, other ratings, and self-other rating discrepancy. In a review paper published in Leadership Quarterly, the authors summarize that the factors affecting 360° ratings include:

a) Self’s biographical characteristics, personality and individual characteristics, and job relevant experiences

b) Rater’s cognitive processes, motivation, and rater–ratee interactions and expectations

c) Contextual factors.

360° results are influenced by a variety of factors; hence, it is important to consider those when initiating a 360° assessment process and interpreting results.

The tips below are important for those using 360° assessments in China, as well as in other countries.

Communicate Purpose. Research indicates that employees’ perception of how the ratings will be used has an impact on what information they provide. Therefore,

  • Before implementing a 360° assessment, make the purpose of the assessment clear.  For example, if the results are going to only be used for development purposes, make sure everyone understands that.
  • After the assessment, help leaders receiving the feedback interpret and apply their results. Without proper interpretation, 360° assessment may be useless, and can even be harmful.

Interpret the information.

  • Consider the factors, such as the cultural context, that may influence self and other ratings, as well as the difference between them, when interpreting results.
  • When raters are from different cultures, the ratings should be interpreted with special care. Cultural factors affecting the rating should be taken into consideration.

Do not over generalize or overemphasize.

  • Be aware that there are many factors that potentially influence ratings. Take results seriously, but don’t accept everything at face value; think critically about results and what the significance of them may be.

Assessment results provide valuable data, but should be interpreted in the context of other information in order to be most helpful.  Keep an open mind and create open communication channels to understand the raters, ratees, and the context.

How do you use and interpret 360° assessment in your organization? What are the barriers to using it effectively and how do you optimize the tool?

———————-

Note:
1. Collectivism-individualism is used to describe cultural difference in how individuals see themselves in the large social group. In high collectivist societies, Individuals are integrated into strong cohesive groups; people emphasize relatedness with groups; individuals are likely to engage in group activities. Some representative collectivistic countries include: China, Japan, Turkey, and Pakistan.
In contrast, in high Individualistic societies, the self is viewed as autonomous and independent of groups, attitudes and personal needs are important determinants of behavior; and individuals are likely to engage in activities alone. Some representative collectivistic countries include: Germany, Australia, UK and USA.
2. Power distance is another dimension used to describe cultural differences in the strength of social hierarchy. High power distance cultures emphasize social hierarchy while low power distance cultures do not. In high power distance societies, power is seen as providing social order, relational harmony, and role stability. Some representative high power distance countries include: Malaysia, Guatemala, Mexico, China and Egypt. Some representative low power distance countries include: Austria, Denmark, Denmark, and Ireland.
3. Assertiveness is described as a willingness and interest in expressing positive and negative messages to others. High assertive societies value assertive, dominant, and tough behaviors; while low assertive societies view assertiveness as socially unacceptable and value modesty and tenderness. High assertiveness countries include: Albania, Germany, Greece, and Austria. Low assertive countries include: Japan, Thailand, Costa Rica and China.
avatar

About Sophia Zhao

Sophia Zhao is a research faculty member at the Center for Creative leadership. She is based in the Asia-Pacific campus in Singapore. Her main responsibility is to carry out research from and for the region. She has published research reports on such topics as senior leadership, womens leadership, and succession management. She received a B.A. from Fudan University and a Ph.D. from National University of Singapore.
This entry was posted in Coaching & Feedback, Globalization, Leading Globally and tagged , , , , , , , . Bookmark the permalink.

2 Responses to How cultural factors can affect 360° assessment results

  1. avatar Dr. Badri Bajaj says:

    Great insights about 360° feedback and tips for using the same…..

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>