nasa · cyouh95 · Dec 1, 2024 · Jan 5, 2025 · Jan 5, 2025 · Jan 10, 2025
diff --git a/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software.md b/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software.md
diff --git a/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/DT_LICENSE.pdf b/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/DT_LICENSE.pdf
diff --git a/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/Quarto_LICENSE.pdf b/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/Quarto_LICENSE.pdf
diff --git a/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/R-utils_LICENSE.pdf b/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/R-utils_LICENSE.pdf
diff --git a/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/biomaRt_LICENSE.pdf b/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/biomaRt_LICENSE.pdf
diff --git a/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/dplyr_LICENSE.pdf b/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/dplyr_LICENSE.pdf
diff --git a/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/glue_LICENSE.pdf b/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/glue_LICENSE.pdf
diff --git a/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/purrr_LICENSE.pdf b/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/purrr_LICENSE.pdf
diff --git a/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/stringr_LICENSE.pdf b/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/stringr_LICENSE.pdf
diff --git a/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/tibble_LICENSE.pdf b/3rd_Party_Licenses/Microarray_Affymetrix_3rd_Party_Software_Licenses/tibble_LICENSE.pdf
diff --git a/Microarray/Affymetrix/Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md b/Microarray/Affymetrix/Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md
diff --git a/Microarray/Affymetrix/README.md b/Microarray/Affymetrix/README.md
@@ -1,7 +1,7 @@
 # GeneLab bioinformatics processing pipeline for Affymetrix microarray data
 
 
-> **The document [`GL-DPPD-7114.md`](Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114.md) holds an overview and example commands for how GeneLab processes Affymetrix microarray datasets. See the [Repository Links](#repository-links) descriptions below for more information. Processed data output files and processing code is provided for each GLDS dataset along with the processed data in the [Open Science Data Repository (OSDR)](https://osdr.nasa.gov/bio/repo/).**
+> **The document [`GL-DPPD-7114-A.md`](Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md) holds an overview and example commands for how GeneLab processes Affymetrix microarray datasets. See the [Repository Links](#repository-links) descriptions below for more information. Processed data output files and processing code is provided for each GLDS dataset along with the processed data in the [Open Science Data Repository (OSDR)](https://osdr.nasa.gov/bio/repo/).**
 
 --- 
 

diff --git a/Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix/CHANGELOG.md b/Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix/CHANGELOG.md
@@ -5,24 +5,47 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [1.0.5](https://github.com/nasa/GeneLab_Data_Processing/tree/NF_MAAffymetrix_1.0.5/Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix) - 2024-08-30
+## [1.0.5](https://github.com/nasa/GeneLab_Data_Processing/tree/NF_MAAffymetrix_1.0.5/Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix) - 2026-05-XX
 
 ### Added
 
-- Add support for bacteria annotations using manufacturer annotations ([#113](https://github.com/nasa/GeneLab_Data_Processing/issues/113))
+- Support for custom annotations, see [specification](examples/annotations/README.md) ([#113](https://github.com/nasa/GeneLab_Data_Processing/issues/113))
 - Add option to skip differential expression analysis (`--skipDE`) ([#104](https://github.com/nasa/GeneLab_Data_Processing/issues/104))
+- Add nextflow schema support for parameter validation and help text generation
+- Add conda support for easier local development and debugging
 
 ### Changed
 
-- Small bug fixes in `Affymetrix.qmd`
-  - Check if `getBM()` returned results before concatenating it to dataframe to avoid error in `bind_rows()` ([#96](https://github.com/nasa/GeneLab_Data_Processing/issues/96))
+- Replace `RUNSHEET_FROM_GLDS` and `RUNSHEET_FROM_ISA` processes and their associated workflow logic with a new staging analysis subworkflow supporting both accession-based and input-file-based execution modes
+- Rework publish directory behavior as part of the staging analysis subworkflow where `outdir` is now the base directory for `GLDS-NNN/` output directory if `--accession` is provided, or the base directory for `results/` output directory if `--runsheet` is provided
+- Bump gl-microarray image from version 1.0.0 to 1.1.0 to match R package updates in the [GL-DPPD-7114-A pipeline document](../../Pipeline_GL-DPPD-7114_Versions/GL-DPPD-7114-A.md)
+- Rename `annotation_config_path` as `array_annot_path` and `config.csv` as `design_info.csv` throughout the workflow and documentation to better reflect the purpose of the file and its contents
+- Convert `generate_protocol.sh` to a Python script for automated handling of reference/annotation parameters
+- Add `create_date` to design_info.csv and parse it in the protocol
+- Move protocol creation from post-processing to main nextflow script to make passing needed values easier and more robust
+- Update software table generation to exclude `purrr` from table if custom annotations are not used
+- Rename module files from UPPERCASE.nf to lowercase.nf following Nextflow community convention
+- Flatten directory-based modules (PROCESS_NAME/ with scripts under resources/usr/bin/) to single lowercase process_name.nf files directly under modules/
+- Move process scripts from modules/PROCESS_NAME/resources/usr/bin/ to the top-level bin/ directory
+- Update processed data protocol to auto-populate workflow version from `nextflow.config` and add Caenorhabditis elegans, Saccharomyces cerevisiae, Escherichia coli, and Pseudomonas aeruginosa to supported organisms ([#98](https://github.com/nasa/GeneLab_Data_Processing/issues/98))
+- Fixes in `Affymetrix.qmd`
+  - Replace live biomaRt::getBM() queries in the QMD with direct downloads of Ensembl's FTP mart-dump tables; drop chunking/retry/Sys.sleep tied to those queries
   - When renaming column names, specify which columns to rename to avoid unintentional renaming ([#97](https://github.com/nasa/GeneLab_Data_Processing/issues/97))
   - When renaming factor names, prevent cases where a factor is partially renamed because it contains a substring that is another factor ([#100](https://github.com/nasa/GeneLab_Data_Processing/issues/100))
   - Update MA plot to support HTAFeatureSet ([#105](https://github.com/nasa/GeneLab_Data_Processing/issues/105))
   - Remove extra `.1` suffix in AFFY HTA 2 0 Probe IDs in the raw data to allow for merging to BioMart data ([#106](https://github.com/nasa/GeneLab_Data_Processing/issues/106))
   - Decrease legend size when sample names are long to prevent it from covering plot ([#107](https://github.com/nasa/GeneLab_Data_Processing/issues/107))
-- Update processed data protocol to auto-populate workflow version from `nextflow.config` and add Caenorhabditis elegans, Saccharomyces cerevisiae, Escherichia coli, and Pseudomonas aeruginosa to supported organisms ([#98](https://github.com/nasa/GeneLab_Data_Processing/issues/98))
-- Update software table generation to exclude `R.utils` from table if data files are not compressed ([#99](https://github.com/nasa/GeneLab_Data_Processing/issues/99))
+  - Simplify group sample retrieval during differential expression group-wise statistics computation to use a more concise `filter/pull/sort` chain instead of `group_by/summarize/filter/pull`, addressing the deprecation warning in dplyr >= 1.1.0 where returning more than 1 row per `summarise()` group is deprecated
+- Changes to post-processing workflow
+  - Resolve output directory `GLDS-NNN/` or `results/` to match main workflow behavior
+  - Replace dp_tools dependency in assay table update and md5sum table generation with standalone scripts
+  - Rename `UPDATE_ISA_TABLES` and `update_curation_table.py` to `UPDATE_ASSAY_TABLE` and `update_assay_table.py` to better reflect their purpose
+  - Add new PURGE_PROCESSING_INFO Nextflow module to strip full paths in nextflow_processing_info_GLmicroarray.txt before publishing
+  - Add parameter validation and summary log from nf-schema
+
+### Removed
+
+- Packages `R.utils`, and `biomaRt` are no longer used in the processing code, and have been removed from software table generation
 
 ## [1.0.4](https://github.com/nasa/GeneLab_Data_Processing/tree/NF_MAAffymetrix_1.0.4/Microarray/Affymetrix/Workflow_Documentation/NF_MAAffymetrix) - 2024-05-17