Structured Assessment of Trainee Performance?

[pkokorowski]

#188

Just wondering if anyone is using a standard method to evaluate trainee performance for specific robotic procedures? How did you choose components to include? Would you be willing to share with the group?

[nsiddiqui]

#190

Currently I tend to divide the cases- pyeloplasty: dissection/ dismemberment, suturing and thereafter gauge fellow’s involvement. With more cases, I would consider measuring the number of moves/ steps/ sutures as a method to assess skills acquisition.
I am hoping that Tom will elaborate further on this question.

[sdave]

#191

Evaluating resident performance needs further evaluation. I usually give them the suturing component of the residual renal pelvis after completion of the ureteropelvic anastomosis. I also mark out the specific points on the screen I would like them to place stitches through on either side and look at how efficient and precise they are at placement.

[nsiddiqui]

#194

Weighing in here with the caveat that our fellowship uses CSATS for Objective Skills Assessment. In general, we divide the cases into critical steps that confer outcomes. For example: Pyeloplasty – Exposure of UPJ, Fixation of Pelvis (hitch stitch), division and spatulation of pelvis/ureter, posterior anastomosis, anterior anastomosis, (and if resident involved – closure of mesenteric window). For Reimplant – ureteral mobilization, dissection from under uterine artery (female), bladder hitch if desired, detrusor tunnel (detrusorotomy), anchoring and advancement of ureter, closure of tunnel (detrusorrhaphy)

These steps are analyzed by CSATS and also by me in the OR. From the progress the fellows make through each of these steps, I determine to which steps they graduate next. For pyeloplasty, usually the exposure and posterior anastomosis is the last thing the fellows learn to do because I believe these two steps set the tone for success.

For reimplant, the detrusor flap creation and the advancement stitch are the last steps the fellows learn independence on.

If you wanted to get objective scoring data without CSATS, just download Alvin Goh’s paper on GEARS (Global Evalautive Assessment of Robotic Skills) J Urol paper. This will give you a common language with which you can score your trainees.
Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ. Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. The Journal of urology. 2012 Jan 31;187(1):247-52.

[nsiddiqui]

#195

Tom,
On average, how many cases do you think trainees will need to become independent to perform a pyeloplasty?
AND

Since you have longitudinal data on your trainees, do you find that simulation exercises expedite their independence? Especially in the last surgical steps you delegate to them?
I think that there is a huge value in watching recorded videos and self evaluate in addition to peer assessment.

[nsiddiqui]

#197

Walid,

I would say around 10 pyeloplasties robotically. This number continues to come down as more incoming fellows are very adept at robotic skills. the fellows we see now can execute robotically on anything you ask them to do. They are merely learning the judgment piece by the time they get to us.

I don’t use reality based sim as much as I’d like for pyeloplasties, but the video review is very helpful. The platform that the fellows use to have their videos and specifically the steps they perform is ideally suited for review and feedback.

Tom

[Author]

Posts