Assessment Tool for Food Safety Product/Service Vendors

• How customizable is the software/system/process?
• What kind of training is needed? How are training needs supported?
• Is this product specific to the US or can it be used/installed/is it permitted in other countries?
• How does the system work?
• What is the fee, and the fee structure (initial and ongoing fees)?
• Are there regulatory agencies that have studied or are in support of this system?
• Is regulatory approval needed, and has it been obtained? Is the technology considered an antimicrobial substance or a device and properly registered under applicable requirements?
• Does the vendor have any systems in actual production that can be viewed?
• Are there OSHA concerns related to the use of this technology?
• Is there special equipment or personal protective equipment required for use of this technology?
• Does the system require calibration? If so, can calibration be done in house or is it required to be performed by a technician with special equipment? What is the frequency of calibration? Are there any costs associated with calibration activities?
•  Is customer support available 24/7?

It is important to consider all aspects of equipment use, particularly preventive maintenance and sanitation, when considering new vendors. Most questions revolve around how to clean equipment and determining the level support given by vendors. Understanding your facility capabilities and layout is necessary. Here are some basic questions:

• Are instructions/guidance available from the original equipment manufacturer regarding appropriate cleaning and sanitizing?
• How accessible are all areas to cleaning? How easy is it to fully break down the equipment?
• Is there training available on software, cleaning and operation of equipment?
• What are the validation studies on the process (wash equipment, modified atmosphere, etc.)?
• Does the vendor have certifications (OSHA, ASME, AWS, etc.)?
• If part of the equipment, what are the software capabilities?
• What is the pack/process/wash line capacity and how are they determined? For wash lines, what is required wash water vs product ratio in wash tanks?

Intervention related (including antimicrobials) Very few interventions are truly a “kill step” and produce companies should be cautious of claims that sound too good to be true. Efficacy in a laboratory setting rarely translates to the results achieved when the intervention is commercialized.

• What concentration/dose is needed? How was that established? What contact time/ parameters are needed?
• Was a peer-reviewed validation study conducted?
• What are the details of a validation study (matrix, conditions, point/location/time of sampling, starting inoculum, type of organism, etc.)?
• Were physical and chemical factors considered as part of any validation and how would they apply to your process?
  • Temperature: The activity of most disinfectants and sanitizers increases as the temperature increases, but there are some exceptions. Too great of an increase in temperature can cause the disinfectant or sanitizer to degrade or even gas off.
  • pH: An increase in pH improves the antimicrobial activity of some sanitizers and disinfectants (glutaraldehyde and quaternary ammonium compounds), but decreases the activity of others (phenols, hypochlorites, and iodine).
  • Water hardness: This reduces the kill rate in certain sanitizers and disinfectants.
• What effect does organic or inorganic matter have on the efficacy?
  • Organic matter can interfere with the antimicrobial’s efficacy by interacting with the active ingredient. It can also reduce the level of activity or protect the microorganisms from attack by acting as a physical barrier.
• How is the dose reliably administered, and how can that be monitored?
• What is the variation within the data (e.g., by season, location, product type, etc.)?
• What is the impact on product quality?
• Will there be any impact on equipment or other incompatibilities with your current system?
• How/where in the process is the intervention applied?
• How can this intervention fit within your current set up/process?
• Once the said system is in place, can/will the vendor help with a validated study in your facility/situation?
• Has the intervention been applied to a system similar to your process and validated? Are details and/or references available?
• Are there any waste disposal issues? Special treatments needed?
• Will the application require any changes to the label? Will organic or other status be affected?
• Is the antimicrobial technology a substance or device and properly registered under pesticide registration requirements?
  • Substance: The use of the term substances and mixtures of substances in the definition of pesticide provides the key to separating pesticides from devices. If the product contains a substance that is intended to prevent, destroy, repel, or mitigate a pest then the product is a pesticide and, in general, will require EPA registration. Antimicrobial substances should have an EPA registration with the active ingredient for the product, organisms controlled, and concentrations required for sanitization and disinfection.
  • Device: If the product consists of an object or article that incorporates a substance or mixture of substances intended to prevent, destroy, repel, or mitigate any pest, the entire product is considered to be a pesticide and is subject to registration under FIFRA section 3. Devices usually generate the antimicrobial onsite. Examples would be ultraviolet light systems, ozone generators, and ultrasonic devices for which claims are made to kill, inactivate, entrap, or suppress the growth of fungi, bacteria, or viruses in various sites.

Regarding laboratory services, the most important questions revolve around the laboratory’s capabilities and certifications. Keep in mind your facility and employee needs and abilities when determining which lab to use. Some basic questions are as follows:

• What is the sample turnaround time?
• How are results communicated? What level of detail accompanies the report?
• Who has access to test results? What is the nature of notifications (and escalation, in the event of a positive)?
• Has the lab been accredited? By whom, when and for what tests? Can a copy of the auditor’s report be reviewed?
• What are the State and Federal certifications needed to perform the required test?
• Is there 24/7 customer support? Is there access to a dedicated customer services and support team?
• Does the customer have access to software to analyze data?
• What is the lab’s industry reputation? Can they provide referrals?

For most methods, there is a delicate balance between time to result and accuracy. On the surface, the claims usually focus on detection rates and on shorter turn-around times. When it comes to detection understanding how many cells are needed for their method to detect is important. Enrichment time is also often understated or even ignored. While there are PCR methods that give results in minutes, they virtually all need many hours of enrichment. Here are some questions to consider:

• Is the test considered “presumptive” or confirmatory? If presumptive, what would be needed to confirm the result?
• How many analytes/ organisms are detected at the same time?
• On what product matrices was the method validated?
• Are there available third-party validations (AOAC, published studies, etc.)
• What is the evidence that the test kit functions and performs as advertised?
• How fresh was the product used in the validation? This is important because older, temperature-abused product pulled from a grocery outlet has very different microflora than something fresh at harvest.
• How many cells are needed for detection? What product volume/ mass is needed?
• How did the vendor get the number of cells to the instrument for detection (very important if they talk about very low numbers; were samples concentrated, what was the original sample volume, how long was the enrichment, etc.)
• In laboratory trials where there is a claim about the ability to detect, ask how the sample(s) were set up? Did the vendor, for example, apply a known quantity directly to the instrument or sample collection point? How would this compare with collecting actual samples in a processing or production environment?
• Is enrichment required? • How long and what are the conditions and preparation needed for the enrichment? Will it resuscitate injured/ damaged cells?
• What level of amplification is necessary for the detection technology to work?
• Can the test tell if the organism is alive or dead?
• What are the false positive and false negative rates? How do they depend on other conditions (e.g., length of enrichment time).
• How specific is the test and what are the organisms for which there is cross-reactivity (e.g., a test for E. coli, vs STEC, vs O157:H7).
• Does the test yield a viable culture from which additional analyses can be done?
• If the fee is based on a price per test- what is the basis for that? Does this assume full capacity of a system (e.g., “batching” of samples)? What if only one sample was run at a time?
• Does the vendor provide support with training of personnel?
• Is there customer support access 24/7?
• If the method it to be performed on-site, what kind of positive controls are needed and how can they be safely managed? What kind of technical expertise, personal protective equipment, and lab facilities are needed?