Family Medicine

General Evaluation Tools (list of evaluation forms only)

Assessment is a joint responsibility of the faculty and the residents. They equally play an important role in the assessment process. Residents are continuously assessed during the duration of their residency training program.

The assessment can be done on a daily basis, mid-block, at the end of the block, and at the end of the training program. The supervisor/consultant, responsible for the resident during that rotation, makes the evaluation at the end of the rotation through New Innovations System. Supervisors are required to give continuous verbal feedback during the rotation. The resident bears the responsibility for ensuring that the evaluations are completed in a timely fashion. Residents who are posted in other departments/specialties are expected to perform and receive evaluations for the following activities, in addition to the In-Training Evaluation Report (ITER)

Presentations (CMEs, journal clubs, grand rounds, etc.)

Procedural skills as listed in the procedures logbook

Mini-Clinical Evaluation Exercise (mini-CEX)

Residents are also expected to evaluate the rotation and the trainer/s by the end of each block.

I. Assessment of Residents

A. Formative Assessment: The following formative assessment forms are completed by the Trainers/Rotation Supervisors to assess the performance of the Residents during their training.

1.      In-training Evaluation Report (ITER) – end-of-block Evaluation

2.      Assessment of Procedural Skills

3.      Mini-Clinical Evaluation Exercise (Mini-CEX)

4.      Presentation Evaluation

5.      Case-Based Discussion (CbD)

6.      Journal Club Evaluation

7.      Multisource Feedback (MSF)

8.      Research Block Evaluation

9.      Portfolios and Logbooks

1. In-Training Evaluation report (ITER)

The ITER is an assessment instrument to document direct observation. The OMSB ITER is designed to assess the competencies outlined in the OMSB Quality Assurance Standards. It aims to highlight the strengths, identify the weaknesses, and aids in developing a plan of action for improvement. Residents at the beginning of each block should be familiar with the specific objectives of the rotation. The Trainers should give midblock feedback to the Residents. Residents are assessed at the end of each block for successful meeting of the rotation’s objectives.

In addition, ITER is also used every three blocks in the Continuity Clinic.

2. Assessment of Procedural Skills

Procedural skills involve the mental and motor activities required to execute a manual task. This tool assesses safe technical performance, appropriate knowledge, and decision making. Furthermore, it addresses pre- and post-procedural skills including consent taking, communication skills, and complications of procedure and their appropriate management.

3. Mini-Clinical Evaluation Exercise (Mini-CEX)

Mini-CEX is a structured assessment of an observed clinical encounter or a “snapshot” of a resident-patient interaction. This tool assesses a clinical encounter with a patient to provide an indication of competence in skills essential for clinical care such as history taking, examination, and clinical reasoning. It is designed to assess the resident’s medical knowledge, patient care competencies, and professionalism. The assessor may focus on one or two competencies per encounter.

The Resident receives immediate feedback to aid learning. It can be used at any time and in any setting when there is a resident and patient interaction and an assessor is available.

4. Presentation Evaluation

The purpose of evaluating the Resident’s presentation is to recognize strengths and identify areas of needed improvement. This evaluation may be used to assess the resident’s oral presentation skills, systematic way of presentation, and medical knowledge.

5. Case-Based Discussion (CbD)

CbD is a structured discussion of clinical case managed by the Resident. CbD is aimed to assess the Resident’s clinical approach and reasoning, analytical, deductive, and decision making skills, and the application of medical knowledge. This provides the resident opportunity to present and discuss his/her case with the trainer enabling the discussion of the ethical and legal framework of practice. The resident receives systematic and structured feedback.

The Resident with his supervisor will select a case in which he/she has been directly involved and agree on a time of discussion. A minimum of 30 minutes will be allotted for the CbD.

6. Journal Club Evaluation

A journal club is defined as an educational meeting in which a group of individuals discuss current articles providing a forum for a collective effort to keep up with the literature. Its main purpose is to facilitate the review of a specific research study and to discuss implications of the study for clinical practice.

The Journal Club Evaluation Form has been developed to assess the residents’ ability to understand the research process and improve his/her ability to critically appraise literature. It helps in building the residents’ medical knowledge as well as interpersonal and communication skills. The residents are expected to be assessed using this form whenever they are presenting during journal club meetings.

7. Multisource Feedback (MSF)

MSF is often called 360-degree assessment. MSF uses specific instruments designed to gather data about particular behaviors or professional constructs (e.g. professionalism and communication skills of the Resident). There should be at least 12 assessors in addition to self-assessment. The assessors can be resident peers, supervising physicians, allied health professionals, patients and family members, etc. Feedback is provided in aggregate form for each source. MSF can be used to provide formative and summative assessments, and identify learners in difficulty.

The OMSB requires the Resident to be evaluated via the MSF at least once a year. The Program Director/Assistant Program Director will choose the assessors. Each assessor will complete the form and this will be submitted to the Program Director/Assistant Program Director or the Program Administrator. The Resident will not see the individual responses and the Program Administrator will summarize the results and the Program Director will discuss the aggregate result and feedback with the Resident.

8. Research Block Evaluation

During the Research Block, residents should be evaluated using the Research Block Evaluation Form, which assesses the residents’ performance in relation to their research project. This form must be completed at the end of each Research block.

9. Portfolios and Logbooks

A portfolio is a dynamic collection of work that exhibits the residents’ efforts, progress and achievements in multiple areas over time. The portfolio encourages the Resident to reflect on the learning process. Logbook, on the other hand, is used to track the educationally relevant activities, such as the number of procedures performed, and it documents that a learning activity has taken place. The portfolios and logbooks assist in formative and summative assessment of the performance of the Residents.

The portfolio may include: i) the logbook, ii) a summary of the research literature reviewed when selecting a treatment option, iii) a quality improvement project plan and report of results, iv) ethical dilemmas faced and how they were handled, vi) self-reflection, etc. The logbook and portfolio are owned by the Resident and are reviewed by the Program Director/Associate Program Director during the face-to-face six-month and annual feedback sessions.

B. Summative Assessment: The following are the summative assessment tools that are utilized by the residency programs to determine the progression of the residents in the program.

Six-Month/Annual Evaluation

Final In-Training Assessment Report (FITAR)

1. Six-Month/Annual Evaluation

Residents undergo a biannual evaluation using the Six-Month and Annual Evaluation forms. This is a summative assessment of their progress during the year. The Program Director/Associate Program Director meets with the Residents individually to conduct the face-to-face feedback session. Residents are counseled regarding their strengths and weaknesses at the end of six blocks and at the end of the academic year. Remedial action plan will be discussed with the resident if applicable.

The annual evaluation will confirm the progress of the resident through the Training Program.

2. Final In-Training Assessment Report (FITAR)

The FITAR is a composite record of a resident’s training progress and performance during the entire duration of his/her residency training. This record indicates whether or not the resident has acquired the minimum required competencies, as per the objectives of the training program, and is competent to practice as an independent specialist.

The FITAR should be completed by the Program Director/Associate Program Director three (3) months before a resident completes or exits the training program.

II. Tools for Evaluating Various Aspects of the Training Program

1. Trainer Evaluation

Evaluation is important for Trainers too. The Trainer Evaluation Form is designed to assess the Trainers role as teachers, not only on the Medical Knowledge role, but also as good role models to the residents. This will highlight their strengths and aid in improving the deficiencies.

Residents are required to assess their trainers at the end of each block using the Trainer Evaluation Form. At the end of the academic year, individual Trainer Evaluation forms are collated and summarized by the Program Administrators. The summary is then submitted to the Chairman/PD for review and distribution to the concerned Trainer. The Trainers are given individual feedback by the Chairman/PD/Associate PD.

2. Rotation Evaluation

The aim of the Rotation Evaluation form is to assess the effectiveness of the rotation. The learning environment, the volume and variety of cases, departmental and interdepartmental meetings, and amount of teaching are some of the criteria that are assessed via this form. This helps the Program Evaluation Committee to address the weaknesses in the rotation and at the same time strengthen the rotation for the benefit of residents’ learning.

All residents are required to evaluate their rotation at the end of each block.

3. Program Evaluation

The Program Evaluation Form has been designed to assess the overall performance of the Program. Both Residents and Trainers evaluate the program. The following criteria are assessed: administrative structure, curriculum, resident performance, program performance, and overall satisfaction of residents and trainers. This evaluation will be anonymous and an aggregate report prepared by the Program Administrator, will be submitted to the Chairman for review and dissemination to the Program Education Committee members. The Program Evaluation Committee will take into consideration the feedback and recommend necessary changes to the Program Education Committee to act upon. The feedback will be considered in further developing and improving the training program.

Resident Performance Assessment Tools

 

EVALUATION	FREQUENCY
ITER	1/block rotation  1/3 blocks CDR
Mini-CEX	6/year
Resident Professionalism Evaluation Form	2/year
Case-based Discussion/Long Case Evaluation	4/year
Presentation Evaluation	4/year
Procedural Skills	a. Core Procedures (evaluation required)  b. Basic Procedures (only in logger)
Linking Learning to Practice	1/block
Journal Club	PGY1 - 6/year  PGY2 - 6/year  PGY3 - 6/year  PGY4 - 8/year
Research Supervisor Assessment Form	2/year
MSF	1/year
Extended and Weekend Shifts Evaluation	5/FM rotation block
Milestone Reflective Practice	2/year
Critical Appraisal Evaluation	2/year
Resident as Teacher Evaluation	2/year (PGY4 residents only)
Resident as Leader Evaluation	Junior level - 2/year  Senior level - 4/year

 

End-of-Year Examination

The examination consists of 100 one-best-answer multiple-choice questions. A few items will have associated images.

PASSING AND RE-SIT:

- The final year of residency training (where resident is exempted from both the written and clinical parts of EOY examination)
- The year the resident passed both components (Written and Clinical) of OMSB Part I Examination (where resident is exempted from EOY examination)

OMSB Part I Examination

This examination consists of the written component (Multiple Choice Questions). This is usually held in July of the PGY2 year, and a pre-requisite examination for the OMSB Part II Examination.

PASSING AND RE-SIT:

OMSB Part II Examination

This is a Specialty Certification Exam. This consists of two components (Written paper and Simulated Surgery), taken after completing all the requirements of the Family Medicine Residency Program.

The Simulated Surgery (Clinic) is not primarily a test of knowledge but that of consulting and communication skills. It is intended to be just like an everyday surgery in any general practice. It is set in a consulting room with patients who will come in for 10 minute appointments. The residents will demonstrate their consulting skills to an assessor/examiner. The cases are typical problems found in general practice. There are 14 stations: seven (7) Arabic speaking stations and seven (7) English speaking stations.

ELIGIBILITY:

The Family Medicine Resident is eligible to take the Specialty Certification Examination, if he/she has completed all the requirements of the Family Medicine Residency Program with valid ACLS and PALS certification. Residents with until three (3) blocks of extension will only be allowed to take the exam.

PASSING AND RE-SIT:

CONTENT OF THE EXAMINATION:

The discipline of general practice has few fixed boundaries, being defined as much by what patients elect to present to us as by our own views on the GP’s job description. General practice is also constantly evolving, reflecting advances in clinical practice, shifts in social expectation and changes in the political, administrative, and fiscal framework.

This defines the curriculum for the specialty certificate examination (OMSB Part II). It sets out to test all those areas of professional knowledge, skill, and values which reflect the consensus view of what comprises a good practice today within the context of the primary health care service and setting in which the candidate is working.

In devising the modules which make up the examination, the Panel of Examiners is guided by the following blueprint which describes in general terms the domains of competence required of a contemporary general practitioner:

Factual knowledge

Evolving knowledge: uncertainty, “hot topics”, qualitative research

The evidence base of practice: knowledge of literature, quantitative research

Critical appraisal skills: interpretation of literature, principles of statistics

Application of knowledge: justification, prioritising, audit

Problem-solving: general applications

Problem-solving: case-specific, clinical management

Personal care: matching principles to individual patients

Written communication

Verbal communication: the consultation process

The practice context: 'team' issues, practice management, business skills

Regulatory framework of practice

The wider context: medico-political, legal and societal issues

Ethnic and trans-cultural issues

Values and attitudes: ethics, integrity, consistency, caritas

Self-awareness: insight, reflective learning, 'the doctor as person'

Commitment to maintaining standards: personal and professional growth, continuing medical education.

Within each module a variety of contexts will be examined in order to test an appropriate range and depth. Candidates may find it helpful to consider the various roles the doctor may adopt in the course of ordinary practice, for example:

·  Residents who failed this examination will need to take the resit examination.

·  Residents who failed the resit examination will not progress to the next year level.

·  Required in all academic years except:

·  The resident will only be allowed to take the clinical part once he/she passed the written part of the examination.

·  A resident will be allowed three (3) attempts to pass the OMSB Part I Examination, as long as the training period does not exceed more than six years. If the resident fails the third attempt, his/her training will be terminated.

·  The resident who failed the written part of the examination will be allowed to progress if he/she passed the End-of-Year Examination.

·  A PGY3 resident will not progress to PGY4 unless she/he passes the OMSB Part I examination.

·  As per the OMSB policy, the resident will ONLY be allowed to take the Simulated Surgery Exam once he/she passed the Written Exam.

·  The resident needs to pass both written and clinical components in order to be deemed to have achieved overall success in the OMSB Part II examination.

Academic Day

The academic day per level (PGY1 to PGY3) is assigned twice per block from 2:00 to 4:00 pm and once per week for PGY4. The academic day schedule includes seminars, case presentations, formal lectures, journal clubs, time for self-directed learning, and research. The core topics are covered throughout the residency. The topic should be presented by the residents or guest speaker from other specialty according to the presentation topics. The resident must discuss the objectives of the assigned topic with the moderator early enough before the day of the presentation (at least two weeks) and have adequate time for preparation. The presentation must be interactive, up to date, evidence-based, and clinically oriented as much as possible.

The main goals of these teaching sessions are to provide the updated evidence based on theoretical knowledge, to develop lifelong learning skills with critical thinking and to learn how to use knowledge in their clinical practice. In addition, to enhance self-confidence for residents to speak in groups and to discuss and critically appraise opinions.

Academic activities are mandatory and essential part of the residency training. Residents are required to attend at least 75% of the academic activities in an academic year to be eligible for promotion from one level of training to a higher level. Each Resident is responsible for ensuring attendance at workshops. Residents who fail to fulfil this requirement will not be awarded a Completion of Training Certificate. Repeated absence at workshops will also prevent the Resident from eligibility for conferences and training abroad. Attendance is strictly monitored during academic activities and duly documented and monitored using the New Innovations System.

Among the academic activities required to be attended by Residents include, but not limited to:

Simulation-Based Learning

Simulation is a technique for practice and learning that can be applied to many different disciplines and trainees. It is a technique (not a technology) to replace and amplify real experiences with guided ones, often “immersive” in nature, that evoke or replicate substantial aspects of the real world in a fully interactive fashion. Simulation-based learning can be the way to develop health professionals’ knowledge, skills, and attitudes, whilst protecting patients from unnecessary risks. Simulation-based medical education can be a platform which provides a valuable tool in learning to mitigate ethical tensions and resolve practical dilemmas. Simulation-based training techniques, tools, and strategies can be applied in designing structured learning experiences, as well as be used as a measurement tool linked to targeted teamwork competencies and learning objectives. (Simulation-based learning: Just like the real thing. Fatimah Lateef. J Emerg Trauma Shock. 2010 Oct-Dec; 3(4): 348–352.)

Basic Procedural Skills Workshop for PGY1 Residents (Casting, Nasogastric tube insertion, Catheterization)

Basic Procedural Skills Workshop for PGY2 Residents (Pap smear, Suturing, IUCD insertion)

Basic Intra-articular Joint Injection Workshop for PGY3 and PGY4 Residents

Basic Obstetrics Procedural Skills Workshop for Family Medicine Male Residents

·  Evidence-Based Medicine Workshop (Basic) – PGY1

·  Application to Evidence-Based Medicine: Randomized Controlled Trials – PGY1

·  Development of Health Education Materials in Primary Health Care – PGY1

·  Diabetes Workshop – PGY2

·  Introduction to Simulated Surgery (Clinic) Examination – PGY2

·  Journal Club

·  Technical Meetings

PROGRAM E-MAIL ADDRESS: familymed@omsb.org

Program Medical Executive:  Myra Cleofe Sinibalo

Office number:     (968) 24181067

Program Medical Executive: Jaime Cayaban

Office number:  (968) 24181648