Framework for Graduate Program Review

Graduate Education at Lehigh

We aspire for graduates of our doctoral, master’s and certificate programs to achieve leadership and personal accomplishment in their professional pursuits. They will attain both depth and breadth of knowledge needed to advance the theory and practice of their chosen fields and will be adept at applying that knowledge. They will be committed to lifelong learning and to mentoring those who succeed them. They will be creative, embrace complexity, and productively challenge accepted paradigms and theories. They will be outstanding communicators and collaborators, able to build, work with, and lead diverse teams. Our graduates will be champions of constructive and ethical action who proceed with intention and integrity. To maintain this quality and continue to offer the most compelling and valuable programs, a program review process is necessary.   

Competencies to be Developed through Graduate Education

Lehigh University, having considered the breadth of capabilities necessary for graduates to succeed as described in the vision, we identify the following five areas of competency that each graduate student should acquire (Table 1). The vision, and the competencies that enable graduates to fulfill the vision, should serve to guide each graduate program as it determines, specifically, what it should be doing with, and requiring of, its students. 
Table 1.  Lehigh University - Graduate Education Core Competencies

Programs should be empowered to build off of these core competency areas, using them as inspiration to define their own programmatic aspirations. Each program should define the competencies, and its specific goals for its students, in terms that are meaningful and relate to how they would be demonstrated in that program’s field of study, in accordance with the intent of the program.    

1 As is naturally the case when guided by a vision for future success, a program’s goals may include elements that are difficult to assess at the time of graduation. There is some natural tension between the need to identify specific learning objectives and degree requirements, and the fact that ultimate success is dependent upon the achievements of the people who carry the Lehigh label into their careers. Each program should use available evidence and literature, and its best judgment, in setting its specific learning objectives and degree requirements.

Key Components of Program Review

The goal of a program review is to enable the programs to share best practices, as well as bring awareness to the program needs and goals.  

Graduate programs will establish programmatic goals in consultation with their college(s) dean(s), including goals for student recruitment/program size, program curricular innovation, and student core competencies (guided by the University Core Competencies) that will be achieved as a result of having completed the program.  These goals should be part of an iterative program review process to identify program strengths and areas for growth.  Having all graduate programs complete such review offers four clear benefits:

  1. Continuous improvement: Reviews can guide design, development and continuous improvement of graduate programs. While programs should be constantly attentive to university and program‐specific goals, periodic review provides a critical opportunity for data collection, assessment, reflection and formulation of plans.
  2. Exchange of best practices: Shared reviews provide means for exchanging best practices among programs.
  3. Enhanced accreditation reviews: Reviews help us meet the expectations of accreditation bodies, both within colleges and across the university, in ways that reflect our values and priorities. For example, at the university level, Middle States requires that we have broadly‐ defined university‐level competencies or goals for graduate students and that programs demonstrate the inter‐relationships of university, college, program and course‐level goals. Approaching the task in the manner proposed here can satisfy Middle States expectations.
  4. Identifying needs: Review of programs across the university can help identify common needs and guide best use of university resources.  

While colleges, in consultation with their graduate programs, should establish review processes for those programs, we suggest that all such processes include local ownership and responsibility and an expectation of high quality. The most constructive review process is characterized by local ownership, with self‐study and continuous improvement in the hands of the program faculty at its center. In an effort to support the evolution of such review processes, we propose a general framework for program review and suggest roles for established bodies within existing structures. Our intent is that the colleges and the individual programs have the freedom to determine the most effective approaches.

To fulfill our expectation of high quality everywhere and a distinctive character for Lehigh graduate education, we must encourage flexibility, creativity and thoughtfulness in locally owned processes for program review. We should, however, expect every program to attend diligently to the success of its graduates and to its alignment with university goals. All program reviews should lead to continuous improvement in quality, as defined by what we value in graduate education.

The Self Study Process

The self study process should present a comprehensive assessment of the program's degree offerings.  To develop this assessment, programs should consider their stated mission, goals, and objectives, as well as their position within the history of the discipline (past, present, projected).  Comparison to a family of its peer institutions may also bring value to the program. This is an opportunity for the program to reassess itself, restating or modifying its mission and the consequent goals and objectives.  However, we aim to keep this effort to no more than 15 hours of combined effort for a program to prepare the documentation needed for university level review. 

The following suggested areas of program analysis are relevant for the self study:

Mission, Goals, and Objectives
  • State the unit’s overall mission and goals as well as the distinct mission and goals of the graduate program(s). We recommend a clear delineation between master’s programs and doctoral programs, where relevant. There should be reference to the program’s position within the university’s mission. A description of the program’s organization, and policies which guide its operations, is important to include and provide context.
  • Interpret the university-wide competencies in the context of the program(s) and likely career needs of students in the program.  Set goals for development of student competencies in each area.
Need/Demand/Admissions
  • For each degree program, please provide evidence concerning the need and/or demand for the program.  This should include 5 years of application counts and admissions/yield numbers.  This information can be gathered from CollegeNet, Salesforce, and/or through a data request from Enrollment Systems and Services.  Programs may consider peer/aspirational benchmarking in this section, as relevant.
  • Identify the specific entry requirements for the program. 
  • Consider the following questions:
    • Is your headcount/demographic distribution of students in line with program/college expectations?  What is standing in your way of meeting program/college expectations?
Interdisciplinary Activities
  • Describe the degree of rapport and exchange between the program and others, and on any means for encouraging students to take courses in other programs. 
  • Describe the intentions/expectations for interdisciplinary work. What has enabled (or hindered) interdisciplinary collaboration and interdisciplinary activities?
  • Identify joint faculty appointments and participation by program faculty in curricula outside the program.
  • Include descriptions and evaluation of any agreements (informal as well as official MOUs) with other academic institutions, industry, or related partners.
Curriculum and Milestones
  • Describe the design and requirements of the degree program.  This should be correlated to student competencies and learning outcomes.
  • Describe how your curriculum considers career development and student outcomes.
  • Describe the policies/procedures related to specific student milestones, including the following as applicable to the program:
    • Core course requirements
    • Qualifying exam procedures (including pass rates)
    • General exam procedures (including pass rates)
    • Time to degree
    • Funding support as it relates to student time to degree
Mentoring/Advising
  • How many faculty members who advise/mentor students have gone through formal mentorship training?  
  • Provide your retention data, including data on students who transition from one degree path to another.
  • Is there someone in your program who has been trained in mentoring students from a diverse range of backgrounds?
  • Do you collect information related to the student experience as students move through and graduate from programs?
  • How do(es) your program(s) approach mentoring and advising?  
    • For master’s programs, are course advisors assigned and active in this process?  
    • Are students provided periodic check-ins?
    • For PhD programs, are students regularly assessed (and the assessment recorded)?
    • Do you leverage individual development plans (IDPs)?
Student Support & Budgetary 
  • How is funding for PhD students managed? 
  • What other activities are used in the department to build student competencies?
  • How are these activities funded?
Handbook
  • If your program has a handbook (if it doesn’t, one should be developed), attach a copy as an appendix/attachment to your report.  

Outcomes Assessment

  • How does the program assess student outcomes after graduation?  Are these outcomes used in continuous improvement strategies for the program? 
Role of the Program: Periodic Program Review

  1. Program review should be conducted at regular and appropriate intervals, on a schedule determined by GRC, taking into account factors such as allocation of administrative support and alignment with accrediting agency cycles. For graduate programs accredited by external agencies, program reviews should seek synergy across review processes and avoid unnecessary duplication.
  2. The program review should include diligent and frank self‐assessment of the extent to which a graduate program is meeting its goals, attending as much as possible to evidence of competency leading to success after graduation.
  3. The review should support formulation of action plans for continuous improvement, including how to address identified shortcomings. Self‐assessment, augmented by any input from accreditation bodies or external advisors, should determine the need for revision of a program’s goals and/or the means through which its goals are met.
  4. In formulating action plans, the program should consider how they align with and complement college and university strategic plans. 
  5. The review should serve as a regular opportunity for the program to articulate needs to enable program success.
Role of Colleges in Program Design and Review

  1. The college assures that each of its programs completes a diligent and substantive periodic review according to its schedule and provides support for conducting those reviews. This may include establishing appropriate review processes for use within the college as well as necessary administrative support.
  2. In concert with the GRC, colleges establish and maintain the calendar of reviews for their graduate programs.
  3. Internally, colleges make use of the results of program reviews to enhance graduate programs and address college strategic needs and priorities. 
  4. Ensure that the program(s) know where data can be found (OID/Tableau), and provide assistance as needed to secure data required.
  5. Colleges make available to the GRC reports on the outcomes of graduate program reviews.
     
Role of Graduate Associate Deans (GADs)

  1. GADs monitor the schedule of periodic reviews for graduate programs in their colleges. GADs notify the GRC when changes to that schedule are necessary, providing an explanation for any change.
  2. GADs work with deans and graduate programs to determine who should prepare summaries of program reviews and make review presentations to the GRC. In some cases, this presenter may be the GAD.
  3. GADs serve as agents for sharing problems, solutions and best practices, within colleges (in accord with college practices) and across colleges.
  4. GADs assist colleges, departments and graduate programs in understanding university (GRC) expectations.
  5. GADs help the university (GRC) understand college, department and graduate program priorities and needs.
Role of the GRC

Per R&P, GRC has responsibility to “[r]eview the appropriateness and quality of graduate programs and research activities in relation to the educational objectives of the university.” Because these recommendations are framed specifically in terms of fulfillment of university‐wide goals by each graduate program, GRC has a central role. Because the work of each program, with support from its college, needs to set its own objectives and determine how best to meet them, the natural role for GRC is in setting expectations and facilitating exchange of information. GRC can also play a critical role in discerning university‐wide needs.

  1. The GRC sets expectations for periodic reviews of all Lehigh graduate programs, reviews that are diligent, creative and constructive.
  2.  In consultation with the colleges, the GRC derives a calendar for all periodic program reviews.
  3. The GRC reviews summaries of the periodic program reviews.
  4. The GRC facilitates sharing best practices among graduate programs, both within and across departments and colleges.
  5. Based on the findings of multiple program reviews, the GRC provides guidance to the colleges, programs and university administration, as appropriate, on potential ways to enhance graduate education at Lehigh. This may include articulation of common graduate program needs, prioritization among multiple priorities, and identification of desirable policies and practices, and communication of these to appropriate administrative units.  Faculty Senate and the Provost will receive reports on best practices and recommendations on an annual basis.

Attachment 1. Proposed graduate student competencies, with non‐field‐specific examples.