Relational Coordination
Guidelines for Theory, Measurement and Analysis
Chapter 4: Analyzing Relational Coordination (continued)
Testing for Differences Between Sites or Between Intervention/Non-Intervention
In addition to looking for differences in the strength of ties between dyads, we are typically very interested in assessing differences in the strength of ties between sites, or between intervention and non-intervention in the same site. To assess these differences, you conduct analyses of variance to find whether you have significant differences in relational coordination between your units of analysis (e.g. cross-site, or between an intervention and non-intervention). In the nine-site flight departure study, significant cross-site differences were found in relational coordination (p<0.0001), as well as significant cross-functional (p<0.0001) differences. When site-level and cross-functional differences were considered jointly, site-level differences remained significant with an F-statistic of 0.0003. The intra-site correlation for relational coordination was significantly greater than zero (p<0.001). Taken together, these results were consistent with treating relational coordination as a site-level construct.
For the nine-hospital patient care study, similar descriptive analyses were conducted with some additional detail. Using one-way analysis of variance, significant cross-site differences in relational coordination were found, F(8,327) = 5.32, p<0.001, as well as significant cross-functional differences in relational coordination,
F(5,330) = 2.89, p<0.05. When site-level and function-level differences were considered jointly, site-level differences remained significant, F(8,322) = 4.51, p<0.001, while function-level differences became insignificant, F(5,322) = 1.75, p=0.12. To further assess treating relational coordination as a site-level construct, we computed intra-class correlations ICC(1) and ICC(2). ICC(1) is the proportion of total variance that is explained by site membership with values ranging from -1 to +1 and values between 0.05 and 0.30 being most typical. This number provides an estimate of the reliability of a single respondent's assessment of the site mean. ICC(2) provides an overall estimate of the reliability of site means, with values equal to or above 0.70 being acceptable. For relational coordination, ICC(1) = 0.25 and ICC(2) = 0.81. We concluded that relational coordination performed well on both forms of intra-class correlation. Taken together, these results are consistent with treating relational coordination as a site-level construct.
Aggregating to Site Level
If you have found significant site-level differences and significant intra-site correlations in your relational coordination construct, you have the basis for building a site-level construct. To aggregate to the site level, you could simply construct a mean score for each site, equally weighting the responses of each survey respondent. However, it is recommended to use a weighted mean, in which individual responses are weighted according to the size of their function in that particular site, so that the site level measure of relational coordination reflects the functional composition of that site. Otherwise your measure of relational coordination will be influenced by the relative response rates of different functional groups.
For example, if physical therapists tend to engage in higher levels of relational coordination than nurses, and their survey response rate is higher than that of nurses, your site-level measure of relational coordination will be biased upward due to the over-representation of the functional group that is more engaged in relational coordination. If physical therapists tend to engage in higher levels of relational coordination than nurses but their survey response rate is lower than that of nurses, your site-level measure of relational coordination will be biased downward due to the under-representation of the functional group that is more engaged in relational coordination. This is especially problematic if the relative response rates of the functional groups differ between sites as they might easily do. Your site-level measures of relational coordination should reflect the functional composition of each site, and not the response rates of the functional groups in each site.
We can see from our data entry in EXHIBIT 19 that we had 20 respondents from the five functional groups that were surveyed at Site A. Summing the RC scores for all 20 respondents and dividing by 20, we get an un-weighted average RC score of 3.24 for Site A.
EXHIBIT 19: Relational Coordination Data for Site A
We can also see from EXHIBIT 19 that, of the 20 respondents, 3 were from Function 1 (15%); 5 were from Function 2 (25%); 4 were from Function 3 (20%); 2 were from Function 4 (10%); and 6 were from Function 5 (30%). Our un-weighted RC score for Site A therefore derives 10% of its value from Function 4 and 30% of its value from Function 5, for example, simply because 10% of the RC scores included in the aggregate site-level score include responses from Function 4 while 30% of those scores represent responses from Function 5.
But suppose Site A had 30 workers in the focal work process, meaning that we achieved a 67% response rate overall (20/30 = 67%). Furthermore, suppose that 4 of these workers were in Function 1 (13%); 8 were in Function 2 (27%); 6 were in Function 3 (20%), 5 were in Function 4 (17%), and 7 were in Function 5 (23%). Our aggregate RC measure should reflect the actual distribution of workers across functions in Site A who are engaged in the focal work process we are trying to understand, not the distribution of survey responses.
To do the proper weighting of RC scores for site-level aggregation, we set up a table as seen in EXHIBIT 20. To create a properly weighted RC score for Site A, we create a weighting factor for each function based on the number of workers in each function relative to the number of workers in the site involved in the focal work process. We then determine the mean RC score for each function, multiply the mean RC score for the function by the weighting factor to get an intermediate score, then sum those intermediate scores to achieve a properly weighted site-level RC score. This properly weighted score is 3.27 rather than 3.24, not a dramatic difference, but more accurate than the non-weighted score for reflecting the focal work process.
1) Weighting Factor for Function = Workers in Function/Workers in Site
2) Intermediate Score for Function = Weighting Factor for Function * mean (RC for Function)
3) Site-Level RC = sum (Intermediate Scores for all Functions)
EXHIBIT 20: Determining Weights for Site-Level Aggregation for Site A
EXHIBIT 21: Comparing Un-Weighted and Weighted Site-Level Aggregation for Site A
Analyzing Performance Effects of Relational Coordination
As explained above under "Expected Performance Effects of Relational Coordination," relational coordination is expected to improve both the quality and efficiency performance of a given work process, particularly when that work process is characterized by high levels of task interdependence, uncertainty and time constraints. Ideally, performance measures will include critical measures of both quality and efficiency for the focal work process. In the flight departure study, the impact of relational coordination was evaluated for efficiency (gate time per departure; employees per passenger) as well as quality (on-time performance; baggage handling performance; customer satisfaction). In the patient care study, the impact of relational coordination was evaluated for efficiency (length of stay), as well as quality (post-operative pain; post-operative functioning; patient satisfaction). It is a good idea to choose these performance measures based on a consensus among practitioners regarding the performance measures that are vital for success in their industry.
In order to assess the impact of relational coordination on performance, one must also understand and measure the other factors that affect those performance outcomes. Again, industry practitioners can be a vital source of information. For the flight departure study, these control measures (or covariates) included scale of operations (flights/month); size of flight (passengers/departure); length of flight (miles/departure); percent connections (passengers connecting/total passengers); and freight loading requirements (tons of freight/departure). For the patient care study, these control measures (or covariates) included site-level volume (surgeries/month); as well as patient age; comorbid conditions; type of surgery; pre-operative pain and functioning; overall health; psychological well-being; marital status; race; and gender.
For your models that predict performance, the independent variable of interest is relational coordination, measured at the site-level (unless you decided to collect a separate measure of relational coordination for each client). The control variables or covariates are also included as independent variables in the model. The dependent variables are the quality and efficiency performance measures. A separate regression model should be run to predict each measure of performance.
Multi-level regression analysis should be used to adjust coefficients and standard errors for the multi-level nature of the data. Previous analyses of the performance effects of relational coordination have nearly always used random effects models, a form of multi-level analysis. The unit of analysis is the individual client or monthly observation within the site. The random effect is the site. Random effects regressions will produce both a within-site R square, and a between-site R square. Within-site R square indicates the percent of within-site variation that is explained by the variables in the models. Between-site R square indicates the percent of between-site variation that is explained by the variables in the model. Either or both can be reported, but should be labeled and explained to readers. xxxvi
Random effects models are increasingly common, and a sentence such as the following can be used to explain their use: "For the above analyses, random effects modeling was used to adjust standard errors for the multi-level nature of the data, accounting for non-independence of the error terms (Bryk and Raudenbusch, 1992)."