This is the summary of the LV segmentation challenge results that has been presented at the STACOM 2011 workshop. More details of this result can be seen in [Suinesiaputra et. al., STACOM 2011].

Participants

We received initial 38 requests for participation from 29 different research institutions. Only 5 participants submitted their results and 3 of them published their papers at the STACOM 2011 workshop. Here are the characteristics five participants, or the auto raters:

Rater Description Type
SCRDeformable registration method2D feature recognition
INRVoxelwise classification technique3D models
AOOptical flow algorithm2D lines/pixels
DSContour tracking algorithm2D lines/pixels
EMACM + optical flow method3D models

There were 2 expert raters: AU & NU raters. Their characteristics are shown below.

  AU Rater NU Rater
Type3D models2D contours
InteractionSupervised FEM fittingManual delineation
SoftwareAMRG CIMQMass
TimeframeFull cardiac cycleED & ES
ViewAny slice orientationShort-axis view
Clinical valuesGlobal LV functionsNot available
SourceUniversity of Auckland, NZNorthwestern Univ., USA

With this expert rater characteristics, we defined the following validations:

  1. Validations on clinical values and performances for the participant submissions of a subset (or full set) of the validation set were made by using the AU rater as the reference.
  2. Validations on clinical values for the collation study were made by using the AU rater.
  3. The ground truth images estimated by using the STAPLE method took all raters (2 experts and 5 auto) as the input from the overlapped 18 cases and from ED & ES of the short-axis view only.

Clinical validations

Rater EDV diffs (ml) ESV diffs (ml) Mass diffs (gr)
SCR13.03 (± 18.13)18.98 (± 16.20)-9.56 (± 22.58)
INR-79.88 (± 39.62)-61.96 (± 38.83)74.75 (± 53.74)
AO8.69 (± 99.39)13.36 (± 64.16)51.49 (± 95.65)
DS3.94 (± 23.14)25.65 (± 17.71)1.08 (± 28.40)
EM-77.75 (± 50.60)-45.46 (± 36.44)-51.92 (± 39.92)

Rater performance

Region Operating Characteristics (ROC) curves