1. Define and explain the following phenomena, using examples when possible.

a. Item-total correlations
b. Coefficient alpha
c. Item discriminability (or item discrimination)
d. Endogeneity
e. Standard error of measurement
f. Factor loading
g. Multi-trait multimethod matrix
h. Adjusted R2
i. Metric invariance
j. Correlated errors

  1. Find a measure for the following variables and provide a rationale for your measure. Indicate how you would validate this measure or provide validation evidence (e.g., cited from previous sources) and how you would use it in a study. Note that this question is more open-ended than others. You are welcome to select what is widely considered the best measure of a given construct or discuss how you would create and/or validate a new measure. You are also welcome to select more than one measure and use them as indicators of a latent variable; realizing that a single measure may be flawed but that a composite of variables or a latent variable may compensate for flaws in individual measures. What I’m looking for here is that the chosen measure(s) are justified given the context (how you intend to use them) which might include also discussing the advantages and disadvantages.

a. Core self-evaluations
b. Firm-level performance

  1. What are some advantages and disadvantages of archival data? How might you provide evidence for the validity of a measure derived from archival data?
  2. What are the advantages and disadvantages of using survey data (self- and other-reports)? How might you provide evidence for the validity of a measure derived from survey data?
  3. Assume that you are working on a study of a relatively new construct, willpower, which is defined as the ability to overcome internal and external obstacles in pursuit of a goal. Reviewer 1 on your paper has suggested that your construct is not unique from the construct of grit, which is defined as the “ability to work strenuously toward challenges and maintain effort and interest over time despite failure, adversity, and plateaus in progress”, and thus, has asked that you provide evidence that grit is unique. Reviewer 2 has asked that since your construct is relatively new that you conduct a cross-validation study of the construct. The editor has suggested that the paper will be accepted if you can address these two points. Please discuss how you would solve the issues that the Reviewers pose.
  4. Compare and contrast the evidence necessary for content-related, construct-related, and criterion-related validity.
  5. Name at least 2 underlying assumptions of Classical Test Theory
  6. Describe the ladders of power and how it is used to transform a distribution of data. Be sure to note under which conditions it would be applied (Note: I am not asking you to list the specific name of the reexpressions).
  7. How does reliability affect the statistical power of a test?
  8. Write out the formulas for the following:
    a. Classical test theory
    b. Reliability
  9. Methodologists have long criticized the proliferation of organizational theories and declared that too many of these theories are ambiguously specified, have been subjected to weak tests, and do not focus on important questions. Explain these critiques and develop recommendations for 1) individual researchers, and 2) academic institutional gatekeepers that would foster the development of strong theory and strong theoretical tests.
  10. Assume that you surveyed 300 employees, asking them to report on the following variables:
    • job satisfaction (5 items),
    • leader-member exchange quality (LMX; 6 items)
    • organizational citizenship behavior (OCB; 8 items)
    • counterproductive work behavior (CWB; 8 items)
    • task performance (5 items)

Sample Solution