mrueschman

It looks like these midsleep times were likely calculated from questions on the Baseline Lab Adult-Child Questionnaire (page 7, question 43). Those questions asked for average times (over the past month) that the participant went to bed and woke up on both weekdays and weekends.

Are you asking about the Cleveland Family Study (CFS) dataset? I was looking through the CFS variables and did not see a school start time variable.

As for family health history, some of this could be ascertained from the medical history variables and a variable like familyi, which is an identifier for family units.

Let me know if I can help with any other questions!

The series of ndes*ph variables may be tricky because they (can) include desaturations that occurred in wake time.

As to your assumptions:

1. You could use these to calculate ODI (at 3%, 4%, etc.) if the desaturation events were scrubbed from wake time.
2. Typically, the desaturation events will be output to the XML annotation files even when they are independent of apneas/hypopneas/arousals.

That said, SHHS is a bit tricky and presents difficulties with the above. We have lost data from the SaO2 channel over time due to the studies being converted from Compumedics Replay (original MS DOS scoring program used 20 years ago) to Profusion at some point. Only desaturations that were associated with other events were retained, for whatever reason. In newer datasets (e.g. CHAT) the XML files will contain desaturation events that occurred independent of other events.

Let me know if you'd like more clarification -- I can call upon some of my colleagues who know more about SHHS' history and the utility of these variables. Thanks for inquiring!

@fsann I updated shhs1-204871-profusion.xml to remove that invalid character, as suggested by the resident expert here. Were you doing some sort of batch processing across all the files when you encountered this one that wouldn't parse? Hopefully there aren't any others that sneaked through our processing steps!

Thanks again for bringing it to our attention.

Thanks for raising this issue! I can confirm what you are seeing in that particular file. It looks like this for me:

I know we have seen this before in our XML exports from the Profusion software. I will talk it over with my colleague who has worked extensively with the EDF/XML files we have posted. If the XML files otherwise appears intact/correct, I believe our crude "fix" will simply be to remove the invalid character and re-save the file. You could probably go ahead and do that in the meantime.

I'll post an update when we clear things up!

Winda,

Dr. Quan does have the files, but he does not have permission to disseminate those data. We are arranging an NSRR Stakeholders meeting for this month and this may be one of the topics at hand. The option open to both of us at this point in time is to reach back out to the parent cohorts to (re)seek access to these specialized data points and to obtain permission for analyzing such data. I think the purpose of our internal discussion will be to see whether it is worthwhile to try and invest the time and effort to obtain additional data (e.g. physical activity) to somehow bring under the more "open" umbrella here at sleepdata.org. If we make any progress I will provide updates.

Stephany,

As an update -- we have regenerated most of the "NSRR XML" files for our datasets (only MrOS is still pending). We fixed the two issues you brought to our attention. Remo's suggested changes/fixes from above were both incorporated into the translation tool.

Winda,

It looks as if these are B/C variables that were available in select cohorts, but were not compiled and published with the main SHHS data sources and A variables.

From S. Quan:

In any event, these data are SHHS cohort specific and need to be requested from individual cohorts if they are available. Not every cohort has the data.

Your question spurred an additional question about whether we (the NSRR group) should attempt to compile some of these additional data domains from the SHHS cohorts in order to share these out to our users. That would obviously take awhile, but I will try to revisit this thread if anything ever come of it!

Thanks,

Mike

Winda,

I see the Table you are referencing. Nothing comes to mind for me regarding those domains, and I can't find anything of much use in the SHHS data dictionary. I went ahead and emailed a couple of the paper authors to see if they could provide any guidance.

I'll get back in touch when I hear back.

Mike

Thanks for inquiring about this on the forum! It appears you have discovered a bug with the NSRR-translated XML files. The bug exists in the EDF Editor and Translator tool, which appears to not be counting the first epoch (30 seconds) of the recording in the duration of the first staging element (typically Wake).

To answer your questions:

1. Yes, the staging information appears different between the files, which is incorrect. Your best bet is to use staging information from the Profusion XML files until we implement a fix in the tool and regenerate the NSRR-translated XMLs.

2. As for your second question about total lengths of sleep staging, I believe the NSRR-translated XMLs drop the last staging group in the file. Hence, if you had two hours of wake (Profusion XML staging = 0) at the end of the recording, this wouldn't be output to the NSRR-translated XML file. I gather this was done for "cropping" purposes to eliminate lots of not-that-useful data from the end of the file and make the hypnogram display better in the accompanying EDF Viewer. That said, I am going to follow-up with the developers of the tool to confirm and ask whether it might be more appropriate to have the total lengths of sleep staging match 1:1 between the two XML files (i.e. output the last staging group).

Thanks again for bringing this to our attention!