We use cookies and other tools to enhance your experience on our website and to analyze our web traffic.
For more information about these cookies and the data collected, please refer to our Privacy Policy.

total sleep time dismatch issue

7 posts
bio
Was this reply useful? Learn more...
 
[-]
minsooyeo +0 points · about 4 years ago

HI, i am on sleep study with CFS & MESA data

on the study i has found critical issue can affect other future works

this issue should be identified promptly.

you know, total sleep time (TST) is can be calculated from sleep stage like equation 1 and the sleep stage can be extracted from annotation xml file

equation1) TST = in bed time - (wake after sleep onset + sleep latency)

Common sense, a TST from the sleep stage and TST from variable csv file are must be same, however several subject are difference

To check the exact value of errors, i ploted figures can express the two values.

i has attached the plots and writed down details, i hope you to deal with this issue promptly


plot figures link https://drive.google.com/open?id=1XQcjGReenZwWpAcf-xMacDm-QSRZDGtv


MESA (1899 subjects), unit: min

(subject number, from variable TST, from sleep stage TST)

(1738, 226, 256)

(2852 , 393, 502.5)

(6476 , 403, 411)


CFS (694 subjects), unit: min

(subject number, from variable TST, from sleep stage TST)

(800937, 19, 28)

(800170, 248, 265)

(801121, 269, 276.5)

(802726, 276, 283.5)

(802368, 326, 340)

(802678, 335, 362.5)

(802001, 359, 363.5)

(801376, 380, 383.5)

(800402, 379, 432.5)


many thanks,

442 posts
bio
Was this reply useful? Learn more...
 
[-]
mrueschman +0 points · about 4 years ago

Thanks for bringing this to our attention. I will explore some of these discrepancies. Can you please answer a couple questions?

  1. What variable from the dataset did you use for "variable TST"?
  2. Did you obtain access to these data on a different account? I do not see an approved data request under this account.
1 post
Was this reply useful? Learn more...
 
[-]
brianhoonsukbyun +0 points · about 4 years ago

Hello mrueschman,

I'm Brian, a colleage of minsooyeo (Minsoo Yeo). I have gotten the access permission of those data . You can check my account about that. We're working in the same team. Actually he is sitting beside me. :-) If there is anything he has to do to use the data, please let us know.

I always appreciate your kind supports.

7 posts
bio
Was this reply useful? Learn more...
 
[-]
minsooyeo +0 points · about 4 years ago

Thanks for your kindly reply,

I send you information you requested


[1]. about "variable TST"

CFS: "SLPPRDP" variable

MESA: "slpprdp5" variable


[2]. account issue

Yes i has obtained access to data on a different account, one of the our team member got access for data. If you need to a information about the account, i will inform you that


Sincerely,

442 posts
bio
Was this reply useful? Learn more...
 
[-]
mrueschman +0 points · about 4 years ago

Thanks to you both. Please have Minsoo submit a data request for MESA , CFS, and whatever other datasets your team is working with. He can use the same sort of language from brianhoonsukbyun's request.

Your findings are not entirely unanticipated. We have encountered such discrepancies ourselves. Right now the NSRR team is undertaking a large-scale effort to review all our datasets for issues exactly like this. We will make corrections whenever possible or otherwise note that the issue exists (and possibly why).

I hope to have a chance to look at a handful of these specifically and report back some initial findings next week.

442 posts
bio
Was this reply useful? Learn more...
 
[-]
mrueschman +0 points · about 4 years ago

Hello again - thanks for your patience.

For MESA - would you please re-download and check 2852 again? This study looks OK to us. This study was re-exported/fixed around the time of Brian's original data request, so perhaps you have an old copy.

For the other two MESA studies (1738, 6476) we traced the discrepancy issue back to issues of data loss/corruption with the original scoring files, which caused these unexpected mismatches. We were unable to fix the scoring data at the source to match the data you see in the summary result file (CSV). I will make a note of this issue with the scored data export here: https://sleepdata.org/datasets/mesa/pages/polysomnography-introduction.md

I will look into the CFS issues next.

7 posts
bio
Was this reply useful? Learn more...
 
[-]
minsooyeo +0 points · about 4 years ago

Thanks for your kindly reply

I has totaly understanded your answers and written down two data requirement forms (cfs, mesa) you asked me

I will re-download data and review one more time

Sincerely,

7 posts
bio
Was this reply useful? Learn more...
 
[-]
minsooyeo +0 points · about 4 years ago

I have confirmed that there is no problem with 2852 subject of mesa data ("slpprdp5" variable --> 393, the TST from new xml files sleep stage --> 393)

thank you