bug: chmoding files: all/all/all ???

[georgemarselis-nvi]

2024-12-09 21:15:18 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/201218_M06578_0041_000000000-JF7TM_demultiplex/Reports/html/000000000-JF7TM/all/all
2024-12-09 21:15:18 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/201218_M06578_0041_000000000-JF7TM_demultiplex/Reports/html/000000000-JF7TM/all/all/all
2024-12-09 21:15:18 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/201218_M06578_0041_000000000-JF7TM_demultiplex/Reports/html/000000000-JF7TM/all/all/unknown
2024-12-09 21:15:18 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/201218_M06578_0041_000000000-JF7TM_demultiplex/Reports/html/000000000-JF7TM/default/Undetermined
2024-12-09 21:15:18 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/201218_M06578_0041_000000000-JF7TM_demultiplex/Reports/html/000000000-JF7TM/default/all
2024-12-09 21:15:18 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/201218_M06578_0041_000000000-JF7TM_demultiplex/Reports/html/000000000-JF7TM/default/Undetermined/all
2024-12-09 21:15:18 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/201218_M06578_0041_000000000-JF7TM_demultiplex/Reports/html/000000000-JF7TM/default/Undetermined/unknown
2024-12-09 21:15:18 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/201218_M06578_0041_000000000-JF7TM_demultiplex/Reports/html/000000000-JF7TM/default/all/all
2024-12-09 21:15:18 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/201218_M06578_0041_000000000-JF7TM_demultiplex/Reports/html/000000000-JF7TM/default/all/unknown

[georgemarselis-nvi]

@magnulei @CathrineAB is this a bug?

what is this

2024-12-11 18:02:58 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/241202_M06578_0219_000000000-LT29R_demultiplex/Reports/html/000000000-LT29R/all/all
2024-12-11 18:02:58 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/241202_M06578_0219_000000000-LT29R_demultiplex/Reports/html/000000000-LT29R/all/all/all
2024-12-11 18:02:58 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/241202_M06578_0219_000000000-LT29R_demultiplex/Reports/html/000000000-LT29R/all/all/unknown
2024-12-11 18:02:58 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/241202_M06578_0219_000000000-LT29R_demultiplex/Reports/html/000000000-LT29R/default/Undetermined
2024-12-11 18:02:58 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/241202_M06578_0219_000000000-LT29R_demultiplex/Reports/html/000000000-LT29R/default/all
2024-12-11 18:02:58 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/241202_M06578_0219_000000000-LT29R_demultiplex/Reports/html/000000000-LT29R/default/Undetermined/all
2024-12-11 18:02:58 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/241202_M06578_0219_000000000-LT29R_demultiplex/Reports/html/000000000-LT29R/default/Undetermined/unknown
2024-12-11 18:02:58 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/241202_M06578_0219_000000000-LT29R_demultiplex/Reports/html/000000000-LT29R/default/all/all
2024-12-11 18:02:58 molly.marsel.is demultiplex_script.py DEBUG chmod 644 /data/demultiplex/241202_M06578_0219_000000000-LT29R_demultiplex/Reports/html/000000000-LT29R/default/all/unknown

all/all/all

any clue?

[CathrineAB]

I need more context to answer. What is this, when did it happen? what did you try to do?

[magnulei]

Im guessing you want to know if the directory structure is correct?
I believe it is. Looking at it quickly it seems like this is the logic:

• First level represent the project account level (all = all projects)
• Second level represent the sample level (all = all samples)
• Third level represent demultiplexed or unknown (all = all barcodes)

so all/all/all is the report for all barcodes from all samples from all projects.

edit:
this is confirmed from the html report. Header says / [all projects] / [all samples] / [all barcodes]

[georgemarselis-nvi]

Thanks, Magnus. This can become "documentation".

So, don't the directory structure and the SampleSheet got to match? cuz

000000000-LT29R/default/all/unknown

I don't get what 'default' signifies here for example

000000000-LT29R/all/all/unknown
000000000-LT29R/default/Undetermined

or what is default and what is undetermined.

I need more context to answer. What is this, when did it happen? what did you try to do?

@CathrineAB just looking at the directory structure, it kept sayin all/all/all and i thought i was mishandling code, writing the wrong directory structure. Turns out, it is stuff I do not know, so, I would like to take this opportunity to document the directory structure in the demultiplex folder.

[CathrineAB]

The undetermined are at least the reads not belonging to an index pair; so the PhiX reads (from control DNA) will go to this bin. There is no line in the SampleSheet for this as there is no index pair determining it.

[georgemarselis-nvi]

The undetermined are at least the reads not belonging to an index pair; so the PhiX reads (from control DNA) will go to this bin. There is no line in the SampleSheet for this as there is no index pair determining it.

I take that since you say 'there is no line', i'd guess we don't care to keep track or a record of these.

[CathrineAB]

We need to keep the reads. Some people use them for error calculations. Maybe you can hear with Arvind if it is possible to include it as a line in the SampleSheet. I cannot see a way to do this since every other possible barcode than the ones already used for the other lines in the samplesheet should be included for this "sample".

[magnulei]

About PhiX

As Cathrine mentioned above, PhiX doesnt have any barcode, so it cant be demultiplexed.
FYI, in case you didnt know, PhiX is there mainly to give us quality metrics from the machine (reads are annotated to the Phix genome on the fly on the instrument, and we get error % and some other stuff). Some like to extract these reads to make these calculations themself.

About Undetermined
Keep in mind that "Undetermined" isnt just PhiX. Its also all crappy reads that couldnt get a barcode match.
If there is an issue in demultiplexing, the answer could be hidden in the undetermined files. Wrong indexes in the sample sheet will be easy to spot by looking at "Top Unknown Barcodes" in the laneBarcode.html for / [default project] / Undetermined / [unknown barcode]

Some thoughts on unknown/undetermined/default project
Looks like the 'default' project is a "bucket" for anything that didnt demultiplex.
And 'all' is a "bucket" for all samples (ie samples in the sample sheet)

Also,
unknown is a unknown barcode
undetermined is a unknown sample
(which is sort of the same thing)

000000000-LT29R/all/all/unknown --> all projects, all samples, all unknown barcodes (will always be empty since per definition there are no underemined reads in this bucket)

000000000-LT29R/default/Undetermined --> default project (ie undetermined reads), undetermined samples

(note: this is my guesswork from looking at the structure and html files)

[georgemarselis-nvi]

@magnulei : When you say PhiX, i assume you mean this: https://www.illumina.com/products/by-type/sequencing-kits/cluster-gen-sequencing-reagents/phix-control-v3.html

(note: this is my guesswork from looking at the structure and html files)

No worries, i'm just looking to fill in gaps in documentation and my knowledge

for week 51, i'll do a write up for this.