Speaker’s Corner – January 2010

Making Global Assessment Work

Nancy Tippins, Ph. D.

Valtera Corporation

The current business challenges, ever-increasing globalization and expanding constituents involved in global operations has had a tremendous impact in the area of assessments.  The development of new labor markets as well as the rise in the number of expatriates, inpatriates, and parent/host/third country nationals has led to the need for global assessments:

·       Assessment programs for the same jobs that work across geographical/cultural boundaries

·       Assessment programs to evaluate the ability to work cross culturally

·       Selection systems that work operationally and are acceptable across diverse cultures

Well designed global assessments provide the following benefits:

·       Consistency in quality

·       Reduction in cost

·       Improvements in administration efficiency

·       Consistent image to applicants and employees worldwide

·       Identifying employees who are capable of working in many different locations

·       Collecting data that allow comparisons across geographical boundaries

·       Targeting human capital where needed.

However, global assessments produce some significant challenges, ranging from variability of local laws, practices and attitudes, translation considerations, and access to technology. 

To address these and other challenges, Dr. Tippins shared her Top 10 Tips:

1. Before conducting a global assessment project, verify that there are common job families and sufficient support from the leaders of the organization.  This includes support for the concept, the ability to convince others to support the process, tolerance for the time required to develop and implement the global assessment system and, most importantly the commitment to provide sufficient resources for the complete lifecycle of the assessment system.
   
   
2. Understand expectations about the assessment process and the context in which it will be used. These expectations include the goals and potential uses of the assessments (selection/promotion/certification/development) and the confidentiality, security and accessibility of the data. The administrative environment, particularly in terms of technology and personnel capabilities should also be considered. Your stakeholders may be operations and labor unions, as well as HR. Applicants will have their own expectations about the process. 
   
   
3. Take culture (national[1], geographical, corporate and departmental) and subcultures into account in the design, development, validation, implementation, and use of assessment programs.  However, nationality explains only 2-4% of the variability in people's values; employees of a company regardless of geography may be more similar than citizens in any given country. Distinguish between cultural values, customary ways of doing things and differences in experiences. Global assessments should attend to potential differences across participants and potential cultural differences across stakeholders.
   
   
4. Consider the other characteristics of the local environment:  labor markets, legal environments, labor unions, the economic conditions.  The local economy will affect your labor supply and the receptivity of various kinds of assessments.  The educational systems affect the capabilities of the labor pool. Laws regarding discrimination, privacy and even restrictions on hiring foreign workers differ from country to country. For example, the European Union Directive on Data Protection prohibits the transfer of personal information from Europe to countries in other regions without adequate protection of privacy, including notice and consent of data transfer.
   
   
5. Develop assessment tools for universal use. Base your tools on KSAO’s required to perform the job, but avoid culturally divergent or specific content, as well as content that simply does not translate well. In addition, ensure that the behavior covered in the content is uniform in its level of appropriateness across cultures.

6.   Adapt global assessments for universal use. As in the previous tip, content and language universality of all materials (including directions, questions and scoring instructions) is key.  High quality translations are essential - so be clear on what “equivalence” means, avoid jargon or culturally divergent terms, and use multiple reviews of translations by individuals who are highly skilled in both languages.  In addition, ensure that techniques, question formats and procedures are familiar to all intended populations and that assessment environments are as similar as possible across locations.

7.   Create the tools necessary for effective administration and use of scores. Use the same care and considerations mentioned above for tool development when developing aids and instructions for administration.  Pay attention to the selection and training of administrators and scorers and consider how information will be combined and integrated (locally or globally).

8.   Determine if your global assessment meets common technical criteria of validity, reliability, fairness (free from bias and potential discrimination) and job-relevancy[2]:

1. Collect evidence of equivalence of the tool across countries.  Do the original and the translated tool measure the same thing in each culture? At the same level? Are the response differences between groups a function of true differences between these groups or due to the fact that the measure does not work well with one of the groups?  Look for both conceptual and statistical equivalence.
2. Use key ethical standards such as basing evaluations and opinions on methods sufficient to substantiate conclusions, using assessments that are appropriate for the situation, conducting assessments using qualified personnel, explaining results to appropriate people clearly and maintaining test security.

Resources for more information

Ryan, Ann Marie & Tippins, Nancy T. (2009). Designing and Implementing Global Selection Systems. United Kingdom: Wiley-Blackwell.

Hofstede, G. (2001). Culture’s consequences (2^nd ed.): Comparing values, behaviors, institutions and organizations across nations. Thousand Oaks, CA: Sage.

Professional Standards for Global Assessments

·       ISO 9000 Standards for Assessment (in progress)

o      Procedures and Methods to Assess People in Work and Organizational Settings

·       International Test Commission (www.intestcom.org)

o      Guidelines for Test Use (1995)

o      Test Adaptation (1996)

·       International Task Force on Assessment Center Guidelines (www.assessmentcenters.org)

o      Guidelines and Ethical Considerations for Assessment Center Operations (2000)

·       American Psychological Association (www.apa.org)

o      Standards for Educational and Psychological Tests (1999)

o      Ethical Principles of Psychologists and Code of Conduct (2002)

·       Society for Industrial and Organizational Psychology (www.siop.org)

o      Principles for the Use and Validation of Personnel Selection Procedures (2003)

·       U.S. Equal Employment Opportunity Commission, Departments of Labor and Justice, Civil Service Commission http://www.dol.gov/dol/allcfr/title_41/Part_60-3/toc.htm

o      Uniform Guidelines on Employee Selection Standards (1978)

[1] Hofstede (DATE) provides a description of the dimensions of national cultural variability.  See reference in resources section.

[2] See Resource section for a listing of professional standards for global assessments