Case study · Research software
GMMGenomics Metadata Multiplexing
A small R Shiny app that turns a lab's plate layout, cell sorter files and primer list into one sample sheet, so nobody has to stitch them together by hand.
Research Software Engineer at WEHI, February to July 2024
The problem
Some single-cell RNA sequencing at WEHI uses CEL-Seq2, a protocol where cells are sorted one at a time into the wells of a plate. The sorting is done by a FACS (fluorescence-activated cell sorting) machine, which writes FCS files that record the well each cell went into. Every well also gets a primer with its own short barcode, so the reads can be traced back to their cell after sequencing.
Before a plate goes to sequencing, someone has to line all of that up in a sample sheet. That meant merging a colour-coded plate layout, the FCS files, a template sheet from the lab and a primer index sheet by hand. Hand-made sheets break in quiet ways. A barcode gets pasted twice, a well is skipped, or a sample name is typed two different ways, and the mistake travels with the data into sequencing.
What GMM does
GMM puts the whole merge on one page. You choose a single folder, and the app recognises each file by its name: the plate layout spreadsheet, the FCS files, the template sheet and, if there is one, the primer index sheet. It then runs four steps and gives back one file to download as CSV, TSV or Excel.
1
Read the plate layout
The lab marks samples on a plate map with cell colours. GMM reads the colours and writes out the plate, well and sample name for every well.
2
Combine the FCS files
The index-sort FCS files are stacked into one table, with the plate, sample and well position of every sorted cell.
3
Merge on plate and well
The sample sheet, the lab's template and the FCS table are joined on plate number, well position and sample name.
4
Add primer indexes
If a primer index sheet is in the folder, each well gets its barcode added. This step is optional.
How it was built
- Where
- WEHI, Parkville
- When
- Feb to Jul 2024
- Front end
- R Shiny
- Merge logic
- Python via reticulate
- R
- Shiny
- DT
- reticulate
- renv
- Python
- pandas
- openpyxl
- fcsparser
- Bash
- Git
GMM was carried forward by several cohorts of the student team at WEHI's Research Computing Platform, starting in 2022. Earlier cohorts worked on the problem before us. In late 2023 Marek Cmero, a Senior Research Officer at WEHI, wrote the FACS merge logic in Python in his celseq-sample-sheet-generator repository, and the project chose to build on it. My cohort, Semester 1 2024, started the current version of the app.
The app is R Shiny on the front and Python underneath. Shiny handles the folder upload, the progress bar, the result table and the downloads. Marek's merge functions run in Python through reticulate, inside a virtual environment that the app sets up when it starts. It was built to run on Milton, WEHI's HPC, through its internal R Shiny service, so researchers could open it in a browser without installing anything.
Between March and May 2024 I wrote the Shiny side of the app: the interface, the server handlers for upload, processing, display and download, the setup scripts, and a small Python wrapper that chains Marek's functions into a single call. I also added test inputs with expected outputs, so we could check the merged sheet against known results. After Jude moved the Shiny code into Marek's repository in May, I added a column clean-up step there, which was merged into his main branch.
Jude Thaddeau Data, from the same cohort, wrote most of the documentation: the READMEs, the wiki, the architecture and workflow diagrams, and the attribution to Marek. Jude also moved the Shiny code into Marek's repository.
Try it: plate to sample sheet
This is a separate concept demo that I wrote from scratch in JavaScript for this page. The samples and barcodes are synthetic, nothing leaves your browser, and it shares no code with GMM.
Pick a plate size and a sample, then click a well or move with the arrow keys and press Enter to place it. Every well already holds a made-up six-letter barcode, which you can edit. The checks look for two easy mistakes: a barcode used twice and a well left empty.
Synthetic data. Not GMM and not a real plate.
Arrow keys move between wells. Enter or Space places the chosen sample. Delete clears a well.
- Filled
- 88/96
- Empty
- 8
- Problems
- 1
96-well plate. 88 of 96 wells filled, 8 empty. Barcode problems: 1.
Selected well
A1Sample 1
Six letters from A, C, G and T.
Checks
- Barcode TAAGTC is used in A1 and D9.
- 8 wells are empty and will be left out of the sheet.
Sample sheet preview
Fix the barcode problems above to download.
| plate | well | row | column | sample | barcode |
|---|---|---|---|---|---|
| DEMO-PLATE-01 | A1 | A | 1 | sample_1 | TAAGTC |
| DEMO-PLATE-01 | A2 | A | 2 | sample_1 | AACGTC |
| DEMO-PLATE-01 | A3 | A | 3 | sample_1 | GCTGGA |
| DEMO-PLATE-01 | A4 | A | 4 | sample_2 | GAGTTG |
| DEMO-PLATE-01 | A5 | A | 5 | sample_2 | GCTTTA |
| DEMO-PLATE-01 | A6 | A | 6 | sample_2 | TTATGA |
| DEMO-PLATE-01 | A7 | A | 7 | sample_3 | GCCGTC |
| DEMO-PLATE-01 | A8 | A | 8 | sample_3 | CTTCAG |
Showing the first 8 of 88 rows.
Credits
GMM was a team effort across several cohorts. Everyone below appears in the project's git history or on its wiki Contributors page.
- Marek CmeroSenior Research Officer at WEHI and project supervisor. He developed the FACS merge logic that GMM uses, which came from his celseq-sample-sheet-generator repository.
- Rowland MosbergenProject supervisor at WEHI.
- Jude Thaddeau DataMy teammate in the Semester 1 2024 cohort. Documentation, wiki and diagrams, and the move of the Shiny code into Marek's repository.
- Andy Le NguyenSemester 2 2023 cohort. Early work on parsing FCS files and merging primer index files.
- Nandi RuanSummer 2023 to 2024 cohort. Worked on the early Shiny prototypes.
- Gloria Zilan HuangSummer 2023 to 2024 cohort. Worked on the early Shiny prototypes.
What I learned
Most of the work was in the edges. Lab files arrive with slightly different names, an optional sheet is sometimes missing, and a plate map keeps its meaning in cell colours. Getting the app to accept an empty primer index file and several FCS files at once is where I learned the most.
I also learned to build on someone else's logic with care. Marek's functions did the hard part, so my job was to wrap them well, test them against known outputs and credit him clearly.
Both repositories are public on GitHub.