View in #cornerstone3d on Slack
@Rod_Fitzsimmons_Frey: Hi, everyone. I’ve been struggling with a bug where segmentations are drawing onto my viewports but in some cases the data seems detached from the SegmentationRepresentation data. It’s dicom dependent, as in it happens for some dicoms all the time, and other dicoms none of the time. My saga is detailed in a thread from a couple days ago. With a new discovery I’m hoping for some guidance from the experts here on where to look in the CS code, or how to mitigate in my own code.
I’ve found that there are two tags:
(5200,9229) Shared Functional Groups Sequence SQ: <Sequence, length 1>,
(5200,9230) Per-Frame Functional Groups Sequenc SQ: <Sequence, length 37>
where if they are NOT present in the dicom, the bug manifests, and if they ARE present the bug is absent. I have verified by removing the tags from “good” dicoms and seeing the bug appear. I have also confirmed the tags are present in all known “good” dicoms and absent in all known “bad” dicoms. (I can’t figure out a way to add the tag to “bad” dicoms though.)
I’d very much appreciate any guidance. This is a potentially company-ending bug for my little startup.
@Alireza_Sedghi: DICOM SEG is a multiframe as far as i know and a multiframe should have these two
https://dicom.innolitics.com/ciods/segmentation/segmentation-multi-frame-functional-groups
Multi-frame Functional Groups Module – DICOM Standard Browser
@Rod_Fitzsimmons_Frey: Probably true, but I need to work around it somehow. These particular dicoms are off a Phillips cart based US machine from our local hospital
@Alireza_Sedghi: hmmm, if you anonymize and share we might be able to see
@Rod_Fitzsimmons_Frey: Thank you… I’ll verify that I haven’t modified the dicom inappropriately first. My software anonymizes dicoms and I might be messing it up in the process.
(that just occurred to me this second)
Here is an example from the interwebs that I am not able to create a functioning segmentation on. I downloaded this from https://rubomedical.com/dicom_files/index.html
Sample DICOM files
It is missing the (5200, 9229) tag
Frame Increment Pointer Atrribute is related to that tag (5200,9230) Per-Frame Functional Groups Sequence.
• In my working dicoms: Frame Increment Point points to (5200,9230) Per-Frame Functional Groups, which is a <Sequence, length 37>
• In my non-working dicoms: Frame Increment Point points to (0018,1063) Frame Time, which is a single value representing milliseconds between frames.
From your experience could this be relevant in how Cornerstone sets up its representation data?
Frame Increment Pointer Attribute – DICOM Standard Browser
These tags seem to be exclusively accessed during the image loading process, so it may be relevant that I’m using wadouri in the dicomImageLoader package.
@Alireza_Sedghi: do you have the referenced series too?
the original us too?
@Rod_Fitzsimmons_Frey: No, I just downloaded that file. My ignorance is probably showing again
@Alireza_Sedghi: so this is just XA right?
where is the SEG
@Rod_Fitzsimmons_Frey: There is no seg yet. My software creates segmentations, but I can’t with this particular dicom.
My issue is when I create a new Cornerstone segmentation + representation it doesn’t get hooked up correctly with this (and these other) dicoms
With the result that when I draw on the viewport with e.g. a circularscissors, the pixeldata of the segmentationRepresentation image is unmodified.
I don’t deal with SEG dicoms at all, since I am also allowing researchers to segment videos, pngs, etc. I grab the cs segmentation data and store it as pngs on the server.
Sorry for being unclear, it’s a lot of information to convey and I always end up posting a wall of text that nobody can parse. I try to be more succinct but then leave out details.
@Alireza_Sedghi: might be a multiframe handling issue
it worked fine in OHIF
check this file
https://github.com/OHIF/Viewers/blob/01fa5d83a65647a2ebf9c8e805fbfdb1c751d4b1/platform/core/src/utils/combineFrameInstance.ts
https://github.com/OHIF/Viewers/blob/01fa5d83a65647a2ebf9c8e805fbfdb1c751d4b1/platform/core/src/utils/combineFrameInstance.ts|combineFrameInstance.ts
we don’t have sophoisticated multiframe handling in cs3d
@Rod_Fitzsimmons_Frey: Thanks very much for that pointer, I’ll check it out
Is this function analogous to combineFrameInstanceDataset in dicomImageLoader imageLoader/wadouri?
@Alireza_Sedghi: I think so, it is a better version of those
@Rod_Fitzsimmons_Frey: Thank you for your help, this is a fantastic trailhead.
I haven’t had any luck with this so far, but we have discovered that for the “bad” dicoms - those lacking tag (5200, 9229) — all the segmentation data is stored on the last frame of the segmentation image’s scalarData, e.g. for a 640x480x10 slice image, all the data is in the last 640x480 bytes. That is true regardless of what frame is displayed when the data is painted. Not sure what that means, but posting it here in case it triggers any ideas.
@Jason_Hostetter: yes it sounds like a multiframe issue - sounds like cornerstone is unable to determine which specific frame to attach the generated seg data to. The “good” dicom images are also multiframe? Like you have an ultrasound cine or XA cine that works as expected?
@Alireza_Sedghi: does it work in OHIF?
try drag and drop and see if it works
https://viewer-dev.ohif.org/localbasic
@Rod_Fitzsimmons_Frey: That’s a great resource, thanks for the link. The ones I got from my client aren’t loading, they’re missing a Modality tag (!!!). That’s a different issue I guess. The one I got from rubomedical.com load fine… is there a way in the viewer to add a labelmap? I can’t find it.
It’s reasonable that Cornerstone expects to see (5200, 9230) since it’s required for multi-frame images. I guess that dicoms be dicoms though, and it’s clearly not always there. It’s also missing on old dicoms apparently. AFAICT Cornerstone is using this primarily to determine image position? I am not sure, but I think that, when (5200,9230) is not present, Cornerstone3d could use constant geometries from SpacingBetweenSlices, SliceThickness, SliceLocation to determine Image Position for each frame. Not sure if it’s better to do that in the loader itself where (5200,9230) is accessed, or in the client of that metadata.
Spacing Between Slices Attribute – DICOM Standard Browser
Slice Thickness Attribute – DICOM Standard Browser
Slice Location Attribute – DICOM Standard Browser
@Jason_Hostetter: For non-regular multiframe, e.g. ultrasound, that doesn’t work bc there is no standard slicethickness, slicelocation, etc. You may also check if the required sequence is getting erroneously removed by a misconfigured anonymization pipeline
@Rod_Fitzsimmons_Frey: It probably is getting removed erroneously, but I won’t be able to correct that.
From the dicom standard (C.7.6.16.1.1)
Even if there are no Functional Groups in the Per-Frame Functional Groups Sequence (5200,9230) an empty Item is encoded for every frame, which an IOD is permitted to specify for a Type 1 Sequence, as described in PS3.5, unless all Items for all frames are empty, in which case the Per-Frame Functional Groups Sequence (5200,9230) may be entirely omitted.
Does this imply that we could provide a default empty item for each frame, should (5200,9230) be missing?
@Alireza_Sedghi: if you think the issue is in cs3d, please create a github issue for us to track, and provide anonymized SEG + referenced data
@Rod_Fitzsimmons_Frey: OK. Is there a way for me to test adding a labelmap in the OHIF link you pasted?
@Alireza_Sedghi: try this
/local (not localbasic) then open it in segmentation mdoe
@Rod_Fitzsimmons_Frey: Thank you. OHIF works perfectly.
@Alireza_Sedghi: then look inside OHIFMetadataProvider
and that combineFrameInstance