Currently, YEAT does not have a document summarizing the results of a workflow. It would be great if we could generate an HTML report containing and highlighting various statistics and details for each sample.
YEAT has grown and matured significantly over the years, and one of its biggest strengths is the ability to run a combinatorial set of samples and assembly algorithms. However, one downside of generating all of these results is that, in order to view them, users need to navigate through multiple directories to find the relevant output and analyze it. This can become increasingly daunting as the number of samples and assembly configurations grows. To address this, an HTML page that consolidates the results and provides hyperlinks to specific output files would be extremely helpful for analysts.
One table we could include might look something like this:
| Sample |
Assembly |
Arguments |
Status |
# Contigs |
Files |
| sample1 |
SPAdes |
default |
✓ Complete |
10 |
View |
| sample1 |
SPAdes |
--isolate |
✓ Complete |
5 |
View |
| sample1 |
SPAdes |
--corona |
✗ Failed |
— |
Logs |
| sample2 |
... |
... |
... |
... |
... |
After spending some time thinking about this report, I quickly realized that we might not be able to use MultiQC to our advantage, unlike in our other workflows. The biggest challenge with YEAT is that we have a limited number of "unique" identifiers for sample names. Specifically, YEAT can run multiple assembly algorithms and configurations on the same sample. Determining how to distinguish between the different combinations of samples, assemblies, and arguments is therefore an interesting problem.
Notice in the table above that the first column contains sample1. There are three entries with the same sample name. This is problematic for MultiQC because the first column in MultiQC tables must be unique. If it is not, a subsequent instance of sample1 would override the previous one. To work around this, we would need to create a unique identifier by combining the sample, assembly, and arguments into a single value, potentially using underscores.
One way we could potentially solve this is by using a combination of sample and assembly labels and maintaining separate tables for each type of information.
In YEAT, each sample and assembly instruction has a unique label. We could potentially use these labels to create the following table structure:
Samples table:
| Sample |
Read 1 |
Read 2 |
| sample1 |
r1.fq |
r2.fq |
| sample2 |
r1.1.fq |
r2.2.fq |
Assemblies table:
| Assembly |
Arguments |
| default_spades |
|
| spades_isolate |
--isolate |
Results table:
| Sample-Assembly |
# Contigs |
... |
| sample1-default_spades |
10 |
... |
| sample2-default_spades |
5 |
... |
By using multiple tables, we could split the information into logical sections like this. However, I wonder if this is too janky of a solution and whether it would make the report more difficult to understand or use.
At this point, I wonder if generating our own HTML template would be the best solution to this problem rather than shoehorning MultiQC into the workflow simply because it provides an easy way to generate a nice UI. A custom HTML report would give us more flexibility in how we represent the relationships between samples, assembly configurations, arguments, and results, while also allowing us to provide direct links to the relevant output files.
In the end, the primary goal of the report would be to provide a centralized location where users can quickly understand the outcome of a YEAT workflow.
Ideally, the report should allow users to:
- Quickly see all samples and assembly configurations that were run.
- Compare results across different assembly algorithms and configurations.
- Identify successful, failed, and incomplete runs.
- View key assembly statistics at a glance.
- Navigate directly to relevant output files and logs.
- Search and filter results by sample, assembly, or configuration.
The report should also scale reasonably well as the number of samples and assembly configurations increases. This will become an interesting problem to solve because, as the number of possible combinations increases, so does the amount of information that needs to be rendered on the HTML page.
Currently, YEAT does not have a document summarizing the results of a workflow. It would be great if we could generate an HTML report containing and highlighting various statistics and details for each sample.
YEAT has grown and matured significantly over the years, and one of its biggest strengths is the ability to run a combinatorial set of samples and assembly algorithms. However, one downside of generating all of these results is that, in order to view them, users need to navigate through multiple directories to find the relevant output and analyze it. This can become increasingly daunting as the number of samples and assembly configurations grows. To address this, an HTML page that consolidates the results and provides hyperlinks to specific output files would be extremely helpful for analysts.
One table we could include might look something like this:
After spending some time thinking about this report, I quickly realized that we might not be able to use MultiQC to our advantage, unlike in our other workflows. The biggest challenge with YEAT is that we have a limited number of "unique" identifiers for sample names. Specifically, YEAT can run multiple assembly algorithms and configurations on the same sample. Determining how to distinguish between the different combinations of samples, assemblies, and arguments is therefore an interesting problem.
Notice in the table above that the first column contains sample1. There are three entries with the same sample name. This is problematic for MultiQC because the first column in MultiQC tables must be unique. If it is not, a subsequent instance of sample1 would override the previous one. To work around this, we would need to create a unique identifier by combining the sample, assembly, and arguments into a single value, potentially using underscores.
One way we could potentially solve this is by using a combination of sample and assembly labels and maintaining separate tables for each type of information.
In YEAT, each sample and assembly instruction has a unique label. We could potentially use these labels to create the following table structure:
Samples table:
Assemblies table:
Results table:
By using multiple tables, we could split the information into logical sections like this. However, I wonder if this is too janky of a solution and whether it would make the report more difficult to understand or use.
At this point, I wonder if generating our own HTML template would be the best solution to this problem rather than shoehorning MultiQC into the workflow simply because it provides an easy way to generate a nice UI. A custom HTML report would give us more flexibility in how we represent the relationships between samples, assembly configurations, arguments, and results, while also allowing us to provide direct links to the relevant output files.
In the end, the primary goal of the report would be to provide a centralized location where users can quickly understand the outcome of a YEAT workflow.
Ideally, the report should allow users to:
The report should also scale reasonably well as the number of samples and assembly configurations increases. This will become an interesting problem to solve because, as the number of possible combinations increases, so does the amount of information that needs to be rendered on the HTML page.