Core 4 & 5 Report: Measurement and Methods Cores

• a) Specific Aims:

The overall goal and specific aims of the Measurement and Methods Cores remain unchanged.

• b)Studies and Results:

Described below are the specific achievements of Core IV and V. These include the development of a methodology for mentoring junior investigators: The Methods and Data Core Seminars and Summary Grids Charting Investigator Progress. Joint Efforts with Measurement Core include the Methods sections of the RCMAR Measurement Volume and Measurement Conference, and the new Measurement Volume summarizing this conference.

Developed a protocol for evaluating measures for cultural sensitivity

Working with the Measurement Cores (Core 4) from the other five RCMAR sites, a measurement evaluation grid was developed and used to evaluate measures in several domains. The CALME site focused on cognitive assessment measures during the RCMAR I, and the focus is now on Affective Disorder.

Developed/applied methodological techniques

Applications Software: Kleinman, a Core V investigator, wrote software for Differential Item Functioning analyses that has been published and used by several investigators. The software is available on request. Additionally, Ocepek-Welikson, another CALME Core V investigator, is proficient in Item Response Theory and Confirmatory Factor Analysis software. During the past year we have updated all Item Response Theory software to accommodate advances in the field.

Publications and Dissemination Related to Differential Item Functioning and cross-cultural comparisons (Joint Efforts with other RCMAR sites)

Continuing the work begun by CALME investigators in 1977, the CALME RCMAR commissioned and published several articles focusing on state-of-the art methodologies for cross-cultural evaluation of measures. Members of the RCMAR MDC Faculty contributed several relevant methodological articles. Additionally, an article from the CALME RCMAR was contributed to the two volume series edited by Mutran and Sudha ( UNC RCMAR) devoted to health disparities research and published in Research on Aging.

Participated in Symposia and Conferences (Joint Efforts with other RCMAR sites)

Dr. J. Teresi was a discussant (“Better research through better measurement”) at the 55th Annual Meetings of the Gerontological Society, Boston, Massachusetts, November 2002. Dr. Teresi was the invited leader of the quantitative section of a measurement conference (May, 2001) devoted to minority aging research, sponsored by the RCMAR Measurement Cores, the NIA, the AHQR, and the NCI. She contributed two concept papers used to elicit discussion and comment by other investigators. These papers were titled: "Item Response Theory in the context of development and evaluation of health measures in gerontology"; "Definitions of invariance in the context of scale evaluation".

Provided Data Analyses Consultation

Numerous meetings and reviews were provided for investigators; these are documented in the Minutes and in ongoing Evaluation Grids that document meetings and contacts with the grantees.

Conducted a Methods & Data Core Seminar

A key component of CORE V was the conduct of the Methods and Data Core Seminars and the preparation of the Summary Grids. The pilot investigators’ summary grids, charting the progress of the investigators were updated routinely. Working with CORES III and IV, the Seminars and Summary Grids comprised the cornerstone of the mechanism for providing ongoing support to investigators.

At each seminar, the grantees presented their work, followed by questions, suggestions and recommendations provided by the faculty and fellow grantees. Grantees from all cohorts were invited to attend the meetings; most remained in the group for two to three years. Mentors were also invited, and several attended selected seminars. The Core, in collaboration with the Measurement, Investigator Development and Community Liaison Core, held its first CALME II seminar meeting on October 31, 2002, the second on February 27 th, 2003 and the third on May 29, 2003.

After each meeting, detailed minutes (8 to 10 pages) were prepared and distributed by Dr. Ramirez. These minutes are sent to all faculty, grantees, mentors and members of the Executive and Review Committees. Each set of minutes summarizes the recommendations of the group and provided an action plan. The Methods and Data Core is comprised of ten faculty members representing numerous disciplines: health services research, sociology, epidemiology and medicine, biostatistics, mathematical statistics, measurement statistics, health economics, health care policy, computer science, community relations and institutional review board, grants management, industrial/social psychology, community psychology, and gerontology and geriatrics.

Assisted investigators in the development of new R01's

Numerous contacts provided by Methods and Data Core faculty on behalf of investigators are documented through an Evaluation Grid.

Assisted in scale construction and analyses of data

The psychometric properties of measures must be adequate for minority as well as for majority populations in order to increase the likelihood of accurate assessment across all groups. Item response theory ( IRT) was used to examine the psychometric properties of a cognitive screening measure used in several epidemiological surveys among Latino, African-American and White, non-Latino elderly. Estimates of precision (reliability), examined across several values of θ (the estimate of degree of cognitive impairment), were good (.70's to .90's) in the range representing most respondents. Overall reliability, although adequate, was lower among the White non-Latino and high-education groups relative to other ethnic/race and education subgroups. Differential Item Functioning (DIF) was examined using several area- and model-based tests. Tests of the magnitude of DIF showed the measure to be relatively free of DIF for the racial/ethnic group and education groups examined. However one item, related to remembering the telephone number, was more difficult for Latinos than for other ethnic/racial groups. Several other items evidenced mild DIF, and one item (difficulty remembering words or names) was a poorly discriminating item. The subjective memory items, intended to provide additional information at the earlier, "borderzone" stages of cognitive impairment, did provide more information at the mild to moderate levels of impairment, although maximum information was not provided at these borderzone ranges. This work was published in 2000; a cross validation is currently being prepared.

Teresi, J.A., Kleinman, M., Ocepek-Welikson, K., Ramirez, M., Gurland, B., Lantigua, R., Holmes, D. Applications of item response theory to the examination of the psychometric properties and differential item functioning of the CARE Dementia Diagnostic Scale among samples of Latino, African-American and White non-Latino Elderly. Research on Aging. 2000, 22, 738-773.

Publication of a volume of peer-reviewed articles related to measures for use with elderly, ethnically diverse populations

Although not an original aim of the RCMAR CALME project, the production of an edited volume of reviews was accomplished. The following review articles were published in the first RCMAR-sponsored publication, edited by Skinner (External Advisory Board of CALME), Teresi ( CORE V Director), Holmes (Executive Committee of CALME), Stahl NIA Project Officer) and Stewart (UCSF RCMAR). The issue of Mental Health and Aging, 2001 (volume 7) was devoted to methodological and review articles of measures for use among ethnically diverse populations. Contributions were from leading scholars in the area of measurement in this area. Guest associate editors were notable contributors to health disparities research. Many of the contributors and associate editors were from RCMAR sites.

Publication/editing of a special issue devoted to measurement methods

Teresi (Columbia RCMAR), Stewart ( UCSF RCMAR), Morales ( UCLA RCMAR) and Stahl (NIA) are currently editing a special volume of articles to be published in Medical Care. These articles are examples of qualitative and quantitative methods for use in examining cross-cultural invariance.

c.) Significance:

The Measurement and Methods Cores, working together, have:

1. Continued to work with the Investigator Development Core in conducting the Methods Seminars attended by the pilot investigators, core directors and faculty;
2. Produced and distributed detailed minutes of the Methods and Data Core seminars;
3. Charted the progress of the pilot investigators by constructing and updating summary grids describing each study.
4. Provided consultation to investigators regarding their data analyses;
5. Developed a protocol for evaluating measures for cultural sensitivity;
6. Promoted the development and/or application of methodological techniques, which can be used in developing new culturally-sensitive measures.
7. Assisted in scale construction and analyses of data;
8. Edited and published a volume of peer-reviewed articles related to measures for use with elderly, ethnically diverse populations and a special volume devoted to measurement issues;
9. Updated the intermediate/advanced measurement & methods readings' reference list.

• Plans:

The Measurement and Methods Cores plan to continue with the bimonthly mentoring meetings. These meetings provide a forum within which pilot researchers and other investigators will share and discuss their research. Some meetings include presentations on selected measurement and methods topics of interest to investigators involved in minority aging research (e.g., a review of domains and measures common across pilots, and a review of substantive and methodological issues involved in cross-cultural comparisons, e.g. measurement of acculturation, and discussion of design and statistical approaches e.g., intent-to-treat.). The Measurement and Methods Cores will continue to work with the other RCMAR Measurement Cores in the preparation of the special issue of Medical Care (described above) to be published in 2005.