2D gel quality control with SpotCheck: application note

SpotCheck is the gel quality control workflow in SameSpots. It measures how reproducibly a lab runs 2D gels: you build a gold standard from gels of a reference sample run to the quality you want to keep, then compare each new gel of the same sample against it. In the studies on this page a test gel passed when at least 80% of its spots had normalized volumes within 3 standard deviations (3 SD) of the gold standard. In the application note, SpotCheck failed both gels a novice ran with a deliberate protocol change and passed the gel run to the supervisor’s protocol. In the 2-DE reproducibility study presented at HUPO 2010, 18 of the 19 participating labs produced gels that passed, and the 19th passed after optimizing its gel running in response to feedback.

Both documents are free to download: the SameSpots application note, Maintaining 2DE Quality Within-Lab (PDF, 1.2 MB) and the HUPO 2010 poster, Healing the Achilles Heel of Proteomics: 2-DE Reproducibility Study (PDF, 0.6 MB).

Key findings

• SpotCheck caught both deliberate protocol changes. A novice’s gel run with a different first-dimension IEF step scored 77.4% of spots within 3 SD of the gold standard, and a gel run with a higher-salt buffer scored 71.4%. Both failed the 80% pass mark.
• The same novice passed by following the protocol. Using the supervisor’s first- and second-dimension steps, the novice’s gel scored 80.8% and passed.
• Within-lab gels were highly consistent. Four test gels compared with a gold standard of seven gels run in the same lab over several days scored 99.7%, 99.9%, 94.7% and 93.6%.
• Distorted gels fail. Against a between-lab gold standard of ten gels, three test gels scored 99.8%, 99.8% and 98.5%; a fourth gel with clear distortions scored 69.6% and failed.
• Reference materials and feedback work. 18 of the 19 labs that submitted gels produced images that passed. The remaining lab’s first gels failed; it optimized its gel running after feedback, and its later gels passed.

Why 2D gel reproducibility needs measuring

Reproducibility was called the Achilles heel of proteomics at HUPO 2007, whatever the method, top-down or bottom-up (ref 2). For 2D gel electrophoresis the variation comes from many steps: sample preparation, the first-dimension isoelectric focusing, the second-dimension SDS-PAGE, staining and imaging. A small change at any of them, such as a different IEF program or a buffer with more salt, can change spot patterns and volumes in ways that are hard to judge by eye.

That matters most when samples are precious. A lab that cannot show its gels are running consistently risks spending limited clinical or experimental material on gels that cannot be compared. A measured quality check, run on an inexpensive reference sample before the real experiment, turns “these gels look about right” into a pass or fail result that can be recorded and repeated.

How SpotCheck measures gel quality

SpotCheck is a workflow within SameSpots that applies its image alignment and spot quantification to a reference sample. The poster describes three steps (ref 2):

1. Run a set of gels to the standard you want to keep, using a single reference sample, and analyze them in SameSpots.
2. Convert that analysis into a SpotCheck gold standard, and set the similarity required of later gels. In both studies here, 80% of spots had to have normalized volumes within 3 SD of the gold standard’s average normalized volumes. The criterion can be made more or less stringent.
3. Compare a test gel of the same sample, run to the same protocol. SpotCheck crops and aligns the test image to the gold standard so all data share the same coordinates, measures the normalized spot volumes and gives a pass or fail result.

The result comes with a map of the gel that colors each spot by how far it lies from the gold standard: within 3 SD, between 3 and 4 SD, or more than 4 SD. Noisy gel edges can be excluded from the comparison. The map shows where on the gel the variation is, which helps you optimize the gel running protocol (ref 1).

Application note: maintaining 2D gel quality within a lab

The application note asks a practical question: can SpotCheck set a lab’s 2D gel quality and teach a new user to reach it before they run a large experiment with limited sample (ref 1)?

Method. The reference sample was a HeLa cell lysate prepared by CilBiotech, suspended in lysis buffer and run to a basic, generic protocol: first-dimension IEF on 24 cm IPG strips, pH 4 to 7, then second-dimension 12% SDS gels, 20 x 25 cm and 1.5 mm thick, followed by staining and imaging. The lab supervisor ran four gels to the quality the lab required, and SameSpots turned them into a SpotCheck gold standard. A novice then ran gels to the same protocol, with and without known changes, and each gel was compared with the gold standard.

Results.

• Different first-dimension IEF step: 77.4% of spots within 3 SD. Failed.
• A buffer with a higher salt concentration: 71.4% of spots within 3 SD. Failed.
• Identical first- and second-dimension steps to the supervisor: 80.8% of spots within 3 SD. Passed.

SpotCheck results for the three novice gels, each compared with a gold standard built from four gels run by the lab supervisor. Data from the application note (ref 1).

Conclusion. SpotCheck gave the lab an objective way to measure and maintain its 2D gel quality, and the map of the regions with the most variation helped optimize the gel running protocol (ref 1).

The gels were run by Claudia Agarinis and Dr Hans Voshol of the Novartis Institutes for BioMedical Research, Basel, Switzerland, and the reference sample was supplied by Alain Miller of CilBiotech, Belgium (ref 1).

Poster: 2-DE reproducibility study across 19 labs

Healing the Achilles Heel of Proteomics: 2-DE Reproducibility Study, HUPO 2010. Select the image to download the poster (PDF).

The poster reports one outcome of the Fixing Proteomics campaign, a noncommercial, vendor-independent initiative set up to tackle the experimental problems holding proteomics back, with reproducibility as the central theme. Its authors were P. Lavery, A. Borthwick and W. Dracup of the SameSpots development team, H. Voshol of the Novartis Institute for Biomedical Research and K. Bala of Bio-Rad Laboratories (ref 2).

Method. The same HeLa cell reference sample, in a standard 2-DE lysis buffer, was sent to 20 labs with a reference protocol and a request for at least three gels each. Each submitted image was cropped and aligned to a gold standard, and its normalized spot volumes were compared with the gold standard’s average spot volumes. A gel passed when 80% of its spots had normalized volumes within 3 SD of the gold standard. Two gold standards were used: a within-lab gold standard of seven gels run from the same sample in one lab over several days, and a between-lab gold standard of ten gels representing the accepted limits of gel running variation across labs (ref 2).

Results.

• Within-lab: all four test gels passed, at 99.7%, 99.9%, 94.7% and 93.6% of spots within 3 SD.
• Between-lab: three test gels passed, at 99.8%, 99.8% and 98.5%. A fourth gel with clear distortions scored 69.6% and failed.
• Across all participants: 18 of the 19 labs that submitted gels produced images that passed. Lab 15’s first gels failed; after feedback from the study participants and the person who ran the gold standard gels, the lab optimized its gel running and its later gels passed.

Percentage of spots within 3 SD of the gold standard for four within-lab and four between-lab test gels. Data from the HUPO 2010 poster (ref 2).

Conclusions. The authors concluded that a complex top-down method such as 2-DE benefits greatly from reference materials and a standardized approach, that participants reached a high degree of both within-lab and between-lab reproducibility, and that feedback on gel quality, with an interface that shows where the most variable spots are, lets users gain confidence before they use precious samples (ref 2).

Using SpotCheck in your own lab

The two studies suggest a simple routine for any lab that runs 2D gels:

• Choose a reference sample you can run repeatedly, such as a commercial cell lysate or a large pooled sample from your own system, and aliquot it.
• Build the gold standard from your best gels. The studies here used four, seven and ten gels. Have your most experienced person run them to the documented protocol.
• Set and record the pass mark. Both studies used 80% of spots within 3 SD; tighten or relax it to suit your work, then keep it fixed.
• Test before you spend precious samples: when someone new joins the lab, when a reagent lot, strip supplier or instrument changes, and when you change any step of the protocol.
• Use the map to troubleshoot. If a gel fails, look at where the out-of-range spots are, fix that step and run the reference sample again.
• Image consistently. SpotCheck compares spot volumes, so capture every gel at the same bit depth and resolution, without saturation. Our guide to imaging 2D gels covers the settings.

For the full analysis workflow that follows a passed gel, see how to analyze 2D gels with SameSpots, and for results from labs using SameSpots, see the SameSpots case studies.

Check your own 2D gels with SameSpots

Run your reference gels through SameSpots: align and detect every spot automatically, see which spots fall outside your standard, and go straight on to the full analysis once your gels pass.

Frequently asked questions

What is SpotCheck in SameSpots?

SpotCheck is a quality control workflow in SameSpots that compares a new 2D gel of a reference sample with a gold standard built from gels run to the quality you want to keep. It reports a pass or fail result and a map of which spots differ most.

How do you measure 2D gel reproducibility?

Run the same reference sample on several gels, align the images and compare normalized spot volumes. In the studies on this page, a gel was considered reproducible when at least 80% of its spots had normalized volumes within 3 standard deviations of the gold standard’s average.

What does “80% of spots within 3 SD” mean?

For each matched spot, SpotCheck checks whether the test gel’s normalized volume lies within 3 standard deviations of that spot’s average volume across the gold standard gels. If at least 80% of spots do, the gel passes. Both the percentage and the number of standard deviations can be changed.

How many gels do I need for a gold standard?

The gold standards in these studies used four, seven and ten gels of the same reference sample. More gels capture more of your lab’s normal variation; all should be run by an experienced person to your documented protocol.

What reference sample was used?

A HeLa cell lysate prepared by CilBiotech, run on 24 cm IPG strips, pH 4 to 7, and 12% SDS gels of 20 x 25 cm.

Can SpotCheck help train new staff?

Yes. In the application note a novice’s gels failed when the protocol was changed and passed when the supervisor’s protocol was followed, which gives a trainee an objective target to reach before running real samples.

References

1. TotalLab. SameSpots application note: Maintaining 2DE Quality Within-Lab. Download the PDF.
2. Lavery P, Borthwick A, Dracup W, Voshol H, Bala K. Healing the Achilles Heel of Proteomics: 2-DE Reproducibility Study. Poster, HUPO 9th Annual World Congress, Sydney, September 2010. Download the PDF.