Applies to: WSU Elson S. Floyd College of Medicine Graduate Medical Education (GME) Sponsoring Institution and all accredited post-graduate medical training programs (i.e. residency or fellowship) and non-accredited clinical fellowship programs sponsored by the College of Medicine.
Date: January 17, 2023
1.0 Policy Statement
It is the WSU College of Medicine GME policy that all Accreditation Council for Graduate Medical Education (ACGME) programs sponsored by the College of Medicine appoint a Program Evaluation Committee (PEC) which performs a formal, systematic evaluation of the program and documents this in the Annual Program Evaluation (APE) as part of the program’s continuous improvement process in compliance with Institutional and ACGME requirements.
2.0 Definitions
Accreditation Council for Graduate Medical Education (ACGME)
Accredits Sponsoring Institutions and residency and fellowship programs, confers recognition on additional program formats or components, and dedicates resources to initiatives addressing areas of importance in graduate medical education.
Clinical Competency Committee (CCC)
A required body comprising three or more members of the active teaching faculty that is advisory to the program director and reviews the progress of all residents or fellows in the program.
Designated Institutional Official (DIO)
The individual in a sponsoring institution who has the authority and responsibility for oversight and responsibility of all ACGME accredited programs.
Graduate Medical Education Committee (GMEC)
An institutional committee of the College of Medicine charged with the responsibility of monitoring and advising on all aspects of institutional, residency, and fellowship education as required by the ACGME.
Program Evaluation Committee (PEC)
A required body that provides a systematic collection and analysis of information related to the design, implementation, and outcomes of a graduate medical education program for the purpose of monitoring and improving the quality and effectiveness of the program.
Program Director (PD)
The individual designated with authority and accountability for the operation of a residency/fellowship program.
Trainee
A physician in training at an ACGME accredited graduate medical education program, the term includes Interns, Residents, and Fellows or other trainee enrolled in an educational program whose education falls under the purview of the College of Medicine Office of Graduate Medical Education.
3.0 Responsibilities
GMEC and DIO.
4.0 Procedures
The program director must appoint the Program Evaluation Committee (PEC) to conduct and document the Annual Program Evaluation (APE) as part of the program’s continuous improvement process. The Program Director must submit a written description of the PEC to the College of Medicine Office of GME, utilizing the College of Medicine GME PEC Template (Attachment A).
At a minimum, the PEC must meet annually and include at least two program faculty members, at least one of whom is a core faculty member, and at least one Trainee (unless there are currently no Trainees in the program).
The Program Evaluation Committee responsibilities are to:
- Act as an advisor to the program director, through program oversight;
- Review the program’s self-determined goals and progress toward meeting them;
- Guide ongoing program improvement, including development of new goals, based upon outcomes;
- Review the current operating environment to identify strengths, challenges, opportunities, and threats as related to the program’s mission and aims;
- Plan, develop, implement, and evaluate educational activities of the program;
- Review and make recommendations for revision of competency-based curriculum goals and objectives;
- Address areas of non-compliance with ACGME standards; and,
- Create the annual program evaluation review document to include the action plan Distribute annual review to faculty and residents o Submit the review to the DIO via the GME Office.
The PEC must evaluate the program’s mission and aims, strengths, areas for improvement, and threats. The following elements should be considered in the committee’s assessment of the program:
- Curriculum;
- Outcomes from prior Annual Program Evaluation(s);
- Quality and safety of patient care;
- Patient safety and quality;
- Recommendations from the Clinical Competency Committee;
- Aggregate resident and faculty:
- Well-being;
- Recruitment and retention;
- Workforce diversity;
- Engagement in quality improvement and patient safety;
- Scholarly activity;
- ACGME Resident and Faculty Surveys; and,
- Written evaluations of the program.
- Aggregate resident:
- Achievement of the Milestones;
- In-training examinations (where applicable);
- Board pass and certification rates; and,
- Graduate performance.
- Aggregate faculty:
- Evaluation; and,
- Professional development.
The Annual Program Evaluation (APE) will document on behalf of the program, formal, systematic evaluation of the program and render a written report which is submitted and reviewed by the GMEC during the Annual Program Director Update.
Guidelines for Program Consideration:
In order to ensure the APE report is as complete and accurate as possible, the PEC must monitor and track each of the following areas at a minimum:
- Current resident performance.
- Faculty development activities.
- Graduate performance including performance of program graduates on the certification.
- Examination.
- Program quality.
- Annual written evaluations of the program by residents and faculty.
- Progress in achieving goals set forth in previous year’s action plan.
PEC will review the templates created (Attachment A) as well as the ACGME “A Quick Guide to the SWOT Analysis” (Attachment B) before or during the PEC meeting.
The program director is ultimately responsible for the work of the PEC. The program director must assure that the annual action plan is reviewed and approved by the program’s teaching faculty. The approval must be documented in meeting minutes. The program’s annual action plan and report on the program’s progress on initiatives from the previous year’s action plan is sent to the GME Office annually for review by the GMEC.
5.0 Related Policies
N/A.
6.0 Revision History
GMEC Approval: March 17, 2020
Revision/Review Date(s): January 17, 2023
Responsible Office: WSU College of Medicine Sponsoring Institution
Policy Contact: Designated Institutional Official
Supersedes: N/A
Attachment A: Program Evaluation Committee
[Insert Name of ACGME Training Program]
The goal of this Program Evaluation Committee (PEC) is to oversee curriculum development and program evaluations for the [insert name of the ACGME training program].
The PEC of [insert name of the ACGME training program] will meet [insert meeting schedule–monthly, quarterly, semi-annually, annually]. The PEC will have at least three members, two program faculty and one trainee from the program. Faculty members may include physicians and non-physicians from the [insert name of the ACGME training program]. The PEC is composed of the following members:
- Chair: [List name of faculty member and title of position]
- [List name of core faculty member and title of position]
- [List name of faculty member and title of position]
- [List name of resident or clinical fellow]
- Additional Members as appropriate
The Committee’s Responsibilities Are to:
- Plan, develop, implement, and evaluate educational activities of the program;
- Review and make recommendations for revision of competency-based curriculum goals and objectives;
- Address areas of non-compliance with ACGME standards;
- Review the program annually using evaluations of faculty, residents or clinical fellows, and others;
- Document on behalf of the program, formal, systematic evaluation of the curriculum at least annually and render a written Annual Program Evaluation (APE), which must be submitted to the GMEC annually in the Annual Program Director Update;
- Monitor and track each of the following:
- Resident performance; o Faculty development;
- Graduate performance including performance on certifying examination;
- Program quality; and
- Progress in achieving goals set forth in previous year’s action plan.
- Review recommendations from the Clinical Competency Committee.
The PEC will be provided with confidential resident/clinical fellow and faculty evaluation data by the GME Office, resident management system, and the program’s administrative staff in order to conduct their business.
The program director is ultimately responsible for the work of the PEC. The program director must assure that the annual action plan is reviewed and approved by the program’s teaching faculty. The approval must be documented in meeting minutes. The program’s annual program evaluation (APE) and action plan to include the program’s progress on initiatives from the previous year’s action plan is sent to the GME Office annually for review by the GMEC.
Effective: [insert date]
Attachment B: A Quick Guide to the SWOT Analysis Department of Field Activities
What is a SWOT Analysis?
The SWOT (Strengths, Weaknesses/Areas for improvement, Opportunities, Threats) analysis often is used in strategic planning. The analysis focuses on the four elements included in the acronym, allowing organizations to identify the forces influencing a strategy, action, or initiative.
Knowing these positive and negative elements can help organizations or their units, including residency and fellowship programs, to more effectively identify strengths and improvement needs, and assess their environment.
The results of the SWOT analysis are typically recorded in a two-by-two table that shows the four dimensions side-by-side for comparison, as shown in one of the sample SWOT analyses that follows. Alternatively, the four cells can be presented below each other, as is done in the other example.
Internal Factors
The first two letters in the acronym, S (strengths) and W (weaknesses), refer to internal factors. These are elements under the control of the program, including faculty members’ qualifications and experience, current trainees, and the resources available to the program. Examples of areas typically considered include:
- Financial resources (institutional support, department support, added resources from grants, other sources).
- Physical resources (location, facilities, equipment, technology).
- Human resources (faculty, community voluntary faculty, coordinators, other program staff).
- Other resources (reputation, graduate network).
- Current processes (curriculum, rotations and experiences, simulation, didactic program).
External Factors
External forces influence and affect every organization and individual. Whether these factors are connected directly or indirectly to an opportunity or threat, it is important to take note of and document each one. External factors typically reference things the program does not fully control, such as:
- Specialty selection and workforce trends (desirability of the specialty, shifts in patient needs, resident/fellow interest and demand, career opportunities in the specialty).
- Institutional and local priorities (opportunities for expansion, need for program contraction, relationships with other programs, opportunities for collaboration with other entities and organizations).
- Economic trends (local, regional, and national financial trends).
- Funding (institutional support, state, and other possible sources).
- Local and regional competition.
- Political, economic, and social environment.
Using the Data from the SWOT Analysis
Once the SWOT analysis is completed, the program can decide on high value areas for improvement, or strategies to maintain and sustain current areas with good performance. Ideally, strategies should focus on leveraging strengths; addressing critical “weaknesses” (areas for improvement); taking advantage of desirable opportunities; and mitigating threats.
Often, strategies emerge by pairing information in the four cells. All four cells can be assessed in pairs of two. Often, the most important pairing is that of internal weaknesses and external threats, as this may identify the most serious issues facing the program.
For the Self-Study, programs should complete a general SWOT analysis that considers their strengths, weaknesses/areas for improvement, and the factors in their environment. It is also possible to conduct a SWOT analysis of a planned major change in a residency of fellowship program, such as expanding the number of trainees, or changing a major participating site.
Summarize the information on the program’s environmental context that was gathered and discussed during the Self-Study. (The italicized text in is used to show SWOT analysis dimensions used in selecting action items after the Self-Study. This information is NOT included in the Self-Study Summary submitted to the ACGME).
In the Summary of Achievements, which is submitted prior to the 10-Year Accreditation Site Visit, programs will report on Program Strengths, as well as improvements in areas identified as Program Weaknesses/Areas for Improvement made in the period between the Self-Study and the 10-Year Site Visit.
Program Strengths
- Proactive leadership by the program director, department chair, and program coordinator.
- A training model that provides one-on-one mentoring between faculty members and residents allowing early operative experience.
- The program is “resident focused,” and the limited number of fellows do not interfere with resident education.
- Robust simulation training using high-fidelity models, a boot camp for arthroscopy, use of animal models for microvascular surgical techniques, and work with industry partners to expand resident access to experience with procedures.
- A diverse set of training sites in close proximity to the primary institution.
Program/Areas for Improvement
- The individualized attention to residents, with one-on-one mentoring by faculty, could be negatively affected by the expansion of the health care system to multiple locations, addition of many new, young faculty members, and the accompanying increase in clinical volume.
- It is a challenge for residents to accomplish high-quality research during their five-year program with only a 2.5-month block of time for research and a limited number of faculty mentors.
- While the department has ample research infrastructure, residents may benefit from more formal guidance and mentoring in the process of selecting and initiating their projects, and ongoing monitoring to ensure progress.
- As the institution has increased the sophistication of its quality improvement (QI) activities, the high level of performance of institutional QI suggests a need for basic Kaizen training on the part of residents participating in these activities (contrasted with learning by doing).
Program Opportunities
- As the Department continues to grow, this creates additional opportunities for new teachers, added variety, and clinical and educational innovation. More midlevel providers are needed for care continuity for patients, and to support residents to ensure the increased patient care service needs do not trump the educational benefits of growth.
- The Department should continue to build on early success with simulation training. Past efforts from a sawbones have grown to a robust cadaveric program. There are many future opportunities to build assessment of competency into the simulation program, particularly with the intern class. We will not do this at the expense of protected low stress learning time. Simulation allows for interaction with other residencies.
- The Department should take advantage of the ability to pull data from the EPIC system, and leverage EPIC as a resource to obtain data on resident and faculty practice outcomes and as tools for investigative research studies.
Threats Facing the Program
Based on the information gathered and discussions during the Self-Study, what are real or
potential significant threats facing this program?
- Continued growth of the department may create a situation where residents no longer spend significant time on any rotation with a single faculty mentor, which disrupts the mentorship goal of the program and the mentorship currently provided.
- Research funding nationwide continues to be difficult. Smaller fund requests can support needs for specialized support for statistics, research planning, travel, and basic equipment.
- Further hospital growth may disrupt the resident call schedule, and distribute resident call over too many services. Service needs should not be allowed to disrupt the educational mission, and inpatient services may require midlevel provider support to manage the service burden.
- As value-based care is adopted, the Department must ensure that residents are intimately involved in the clinical care delivery and decision-making process. Residents should be exposed to best practices, and a variety of practice to allow for independent learning.
Strengths
Used in deciding on action items after the Self-Study, but should not be included in the Self-Study Summary that is submitted to the ACGME.
- Small program size with no fellowship programs, which provides opportunity for significant amount of hands-on experience and progressive responsibility.
- Positive relationship between house staff and residents promotes empowerment of residents with an emphasis on residents’ active participation in their own education, and in quality improvement activities and advocacy projects.
- Diverse patient population and pathology, including excellent exposure to pediatric trauma cases.
- Resources of the medical school, with opportunities for residents to mentor medical students.
Areas for Improvement
Used in deciding on action items after the Self-Study, but should not be included in the Self-Study Summary that is submitted to the ACGME.
- Provide support and channel residents’ interests toward research opportunities.
- Improve balance of faculty vs. residentdriven didactics; the curriculum could be broadened to incorporate more topics related to practice management, job interviews and negotiating employment contracts, and general “business of medicine” topics.
- Continue to address areas and sources of conflict between residents and neonatal nurses/nurse practitioners in the NICU.
Opportunities
- Fully realize the advantages for learning and team-based care and education resulting from the organization of the Women and Children’s Health service line.
- Use the resources of the medical school to enhance opportunities for resident research and participation in scholarly activity.
- Consider enhanced resident involvement and added coordination of community outreach activities through a new coalition.
Threats
- Threats to patient base and referrals, and to resident recruitment in a competitive marketplace with three other large pediatric programs.
- Competition for GME resources within the medical school and hospital and nationally, and the potential vulnerability of HRSA support for the two primary care training positions.
- Lack of faculty resources in pediatric subspecialties.
- Growth in number of medical students, which has not been accompanied by significant growth in the pediatric faculty, results in growing concern about the ability to maintain the current quality of the resident experience in the inpatient unit and in clinic.
- An ongoing threat to the balance of education and service, and to the availability of faculty for clinical teaching.
Sources
Morrison M. SWOT Analysis, 2006. Accessed April 6, 2017. Osita C, Onyebuchi I, Justina N. “Organization’s stability and productivity: the role of SWOT analysis” International Journal of Innovative and Applied Research Jan 2014, 2(9):23–32. Accessed April 6, 2017.