We use cookies and other tools to enhance your experience on our website and to analyze our web traffic.
For more information about these cookies and the data collected, please refer to our Privacy Policy.

STAGES annotation vs edf

6 posts
Was this reply useful? Learn more...
 
[-]
ShashaankVattikuti +0 points · 11 months ago

Hello,

First: I saw a similar issue posted and closed in the past. However my issue is not the same. https://sleepdata.org/forum/annotations-in-stages-dataset/1#comment-1003

For at least the MAYO data, I am finding that the annotation files start times do not match the EDF start time. For example, MAYO0002 starts at 22:24:10 (first event) but the EDF start time is 20:18:25. There's a two hour difference. I am not sure how to align the staging in the annotation file to the EDF. Please advise. MAYO0004 is like this as well as others.

446 posts
bio
Was this reply useful? Learn more...
 
[-]
mrueschman +0 points · 11 months ago

Thanks for raising the issue. I see what you mean in those (and other) MAYO files. I'll inquire with the STAGES data contributor.

6 posts
Was this reply useful? Learn more...
 
[-]
ShashaankVattikuti +0 points · 11 months ago

Thank you. I made a minor clarifying edit to my issue. To be clear the issue is the time mismatch between the files. I also observe that the EDFs have some erroneous date, but I assume that was done on purpose. Aligning the start times between the annotation and EDFs, though, is the important issue.

446 posts
bio
Was this reply useful? Learn more...
 
[-]
mrueschman +0 points · 11 months ago

Thanks for clarifying. You are correct - the original collection dates were all scrubbed from the EDFs as part of our normal de-identification process. We set all the calendar dates to 01/01/1985 - the first allowable date per the EDF specification.

My question to the STAGES team focused on the time mismatch. I opened 1-2 files from all the other sites and did not see similar mismatches. Looking back through old emails, I did see a couple mentions of the MAYO files being particularly difficult to process/harmonize, so perhaps some sort of inadvertent time shift occurred.

446 posts
bio
Was this reply useful? Learn more...
 
[-]
mrueschman +0 points · 11 months ago

Today I removed the MAYO PSG data from the site at the behest of the STAGES team; please do so on your end as well. The data contributor recognizes these issues in the EDFs/annotations and hopes to reprocess/fix the MAYO files at a later date.

Thanks again for raising the issue here.

6 posts
Was this reply useful? Learn more...
 
[-]
ShashaankVattikuti +0 points · 11 months ago

Can I use the MAYO data knowing there is this annotation issue? Or, is this an official request to not use the data until it is fixed? I am fine either way.

446 posts
bio
Was this reply useful? Learn more...
 
[-]
mrueschman +0 points · 11 months ago

I would request you not use the data until it is fixed (out of an abundance of caution).

6 posts
Was this reply useful? Learn more...
 
[-]
ShashaankVattikuti +0 points · 11 months ago

Acknowledged. We will not use the MAYO data until it is fixed. Thank you for looking into this.