Food Safety

Assessment Tool for Food Safety Product/Service Vendors

In January 2020, the United Fresh Food Safety & Technology Council formed a working group of twelve food safety professionals from across the industry to identify key areas that companies could address to assess the potential of new and novel technologies, services, and products. This group organized and augmented ideas developed during a council exercise. The purpose of this document is to facilitate reviews of the products, services and technologies offered to the produce industry. Providers of such products and services may benefit from understanding the issues that may be raised when initiating conversations with potential users.

The document is divided into the following sections:

  • General questions
  • Software
  • Equipment
  • Intervention related
  • Laboratory services
  • Testing 
  • Summary appendix that provides additional context and considerations

General Questions

  • How customizable is the software/system/process?
  • What kind of training is needed? How are training needs supported?
  • Is this product specific to the US or can it be used/installed/is it permitted in other countries?
  • How does the system work?
  • What is the fee, and the fee structure (initial and ongoing fees)?
  • Are there regulatory agencies that have studied or are in support of this system?
  • Is regulatory approval needed, and has it been obtained? Is the technology considered an antimicrobial substance or a device and properly registered under applicable requirements?
  • Does the vendor have any systems in actual production that can be viewed?
  • Are there OSHA concerns related to the use of this technology?
  • Is there special equipment or personal protective equipment required for use of this technology?
  • Does the system require calibration? If so, can calibration be done in house or is it required to be performed by a technician with special equipment? What is the frequency of calibration? Are there any costs associated with calibration activities?
  •  Is customer support available 24/7?


Regarding software the things to consider involve how “off-the-shelf” the platform is. Key questions revolve around what the software really does and how well it works in your environment. Understanding your process workflow, who is capturing what data where, and how it is currently being used are often misunderstood and this can be costly in terms of money and time. Some basic considerations:

Many business reports and magnifying glass.

  • Software providers should be challenged to analyze your operation’s specific workflow and how the software will capture it correctly, accounting for who is doing what now, and where and what the output of the data capture will look like.
  • Will the use of this software require more people taking measurements or capturing data in a different way? Who is responsible for collecting and inputting data?
  • If you are unable to conform current data capture to the way the software is currently designed, will it require programming changes? Is there a charge for this?
  • If you need to add/delete information along the way will it require advanced programming and mean data loss? Are the data protected/not changeable?
  • How often do software upgrades occur? Ask for actual data reports on this.
  • What does a software upgrade look like? Do all users have to stop using the program/app? Does it require a system reset? What is the potential downtime for this? What are the typical things that go wrong? Is there a cost involved?
  • If the vendor mentions a number of their current customers- ask them whether the customers have given their approval to use their name. How many of the customers mentioned are still using the software?
  • What is the support program? Response time? Training? Turn-around time to resolve issues with the software? Customer service support 24/7?
  • For other data platforms for which you may want to combine data, how easy is it to get a data dump of the raw data?
  • Will the vendor provide any examples of data linkage or data output between your system and others?
  • How are data stored (automatically, uploaded to the cloud)?
  • If everything is pushed to the cloud what happens when access to the cloud is limited, e.g., dead spots or temporary outages?
  • What specific features (e.g., traceability, documenting corrective actions) are included?
  • How do the features correspond to audit requirements of different schemes/ regulations? How easy is it to demonstrate compliance?
  • What languages are available? How can language barriers be overcome?
  • For software that collects lab data, are you limited to using one lab or can data be transmitted/downloaded from other labs?
  • What is the interoperability of this system with other software that you, customers, and/or vendors use?
  • What are the mechanisms for oversight/ review of data?
  • What training is available for users during launching of software? What level of computer competency is needed by users?
  • Is there an ability to customize the software for specific applications?
  • How many users are covered by the initial cost? Are there different levels of access and permissions (e.g., admin, read-only etc.)?


It is important to consider all aspects of equipment use, particularly preventive maintenance and sanitation, when considering new vendors. Most questions revolve around how to clean equipment and determining the level support given by vendors. Understanding your facility capabilities and layout is necessary. Here are some basic questions:

  • Are instructions/guidance available from the original equipment manufacturer regarding appropriate cleaning and sanitizing?
  • How accessible are all areas to cleaning? How easy is it to fully break down the equipment?
  • Is there training available on software, cleaning and operation of equipment?
  • What are the validation studies on the process (wash equipment, modified atmosphere, etc.)?
  • Does the vendor have certifications (OSHA, ASME, AWS, etc.)?
  • If part of the equipment, what are the software capabilities?
  • What is the pack/process/wash line capacity and how are they determined? For wash lines, what is required wash water vs product ratio in wash tanks?

Intervention Related

Male hand hold yellow liquid in test tube.

Intervention related (including antimicrobials) Very few interventions are truly a “kill step” and produce companies should be cautious of claims that sound too good to be true. Efficacy in a laboratory setting rarely translates to the results achieved when the intervention is commercialized.

  • What concentration/dose is needed? How was that established? What contact time/ parameters are needed?
  • Was a peer-reviewed validation study conducted?
  • What are the details of a validation study (matrix, conditions, point/location/time of sampling, starting inoculum, type of organism, etc.)?
  • Were physical and chemical factors considered as part of any validation and how would they apply to your process?
    • Temperature: The activity of most disinfectants and sanitizers increases as the temperature increases, but there are some exceptions. Too great of an increase in temperature can cause the disinfectant or sanitizer to degrade or even gas off.
    • pH: An increase in pH improves the antimicrobial activity of some sanitizers and disinfectants (glutaraldehyde and quaternary ammonium compounds), but decreases the activity of others (phenols, hypochlorites, and iodine).
    • Water hardness: This reduces the kill rate in certain sanitizers and disinfectants.
  • What effect does organic or inorganic matter have on the efficacy?
    • Organic matter can interfere with the antimicrobial’s efficacy by interacting with the active ingredient. It can also reduce the level of activity or protect the microorganisms from attack by acting as a physical barrier.
  • How is the dose reliably administered, and how can that be monitored?
  • What is the variation within the data (e.g., by season, location, product type, etc.)?
  • What is the impact on product quality?
  • Will there be any impact on equipment or other incompatibilities with your current system?
  • How/where in the process is the intervention applied?
  • How can this intervention fit within your current set up/process?
  • Once the said system is in place, can/will the vendor help with a validated study in your facility/situation?
  • Has the intervention been applied to a system similar to your process and validated? Are details and/or references available?
  • Are there any waste disposal issues? Special treatments needed?
  • Will the application require any changes to the label? Will organic or other status be affected?
  • Is the antimicrobial technology a substance or device and properly registered under pesticide registration requirements?
    • Substance: The use of the term substances and mixtures of substances in the definition of pesticide provides the key to separating pesticides from devices. If the product contains a substance that is intended to prevent, destroy, repel, or mitigate a pest then the product is a pesticide and, in general, will require EPA registration. Antimicrobial substances should have an EPA registration with the active ingredient for the product, organisms controlled, and concentrations required for sanitization and disinfection.
    • Device: If the product consists of an object or article that incorporates a substance or mixture of substances intended to prevent, destroy, repel, or mitigate any pest, the entire product is considered to be a pesticide and is subject to registration under FIFRA section 3. Devices usually generate the antimicrobial onsite. Examples would be ultraviolet light systems, ozone generators, and ultrasonic devices for which claims are made to kill, inactivate, entrap, or suppress the growth of fungi, bacteria, or viruses in various sites.

Laboratory Services

Regarding laboratory services, the most important questions revolve around the laboratory’s capabilities and certifications. Keep in mind your facility and employee needs and abilities when determining which lab to use. Some basic questions are as follows:

  • What is the sample turnaround time?
  • How are results communicated? What level of detail accompanies the report?
  • Who has access to test results? What is the nature of notifications (and escalation, in the event of a positive)?
  • Has the lab been accredited? By whom, when and for what tests? Can a copy of the auditor’s report be reviewed?
  • What are the State and Federal certifications needed to perform the required test?
  • Is there 24/7 customer support? Is there access to a dedicated customer services and support team?
  • Does the customer have access to software to analyze data?
  • What is the lab’s industry reputation? Can they provide referrals?


For most methods, there is a delicate balance between time to result and accuracy. On the surface, the claims usually focus on detection rates and on shorter turn-around times. When it comes to detection understanding how many cells are needed for their method to detect is important. Enrichment time is also often understated or even ignored. While there are PCR methods that give results in minutes, they virtually all need many hours of enrichment. Here are some questions to consider:

  • Is the test considered “presumptive” or confirmatory? If presumptive, what would be needed to confirm the result?
  • How many analytes/ organisms are detected at the same time?
  • On what product matrices was the method validated?
  • Are there available third-party validations (AOAC, published studies, etc.)
  • What is the evidence that the test kit functions and performs as advertised?
  • How fresh was the product used in the validation? This is important because older, temperature-abused product pulled from a grocery outlet has very different microflora than something fresh at harvest.
  • How many cells are needed for detection? What product volume/ mass is needed?
  • How did the vendor get the number of cells to the instrument for detection (very important if they talk about very low numbers; were samples concentrated, what was the original sample volume, how long was the enrichment, etc.)
  • In laboratory trials where there is a claim about the ability to detect, ask how the sample(s) were set up? Did the vendor, for example, apply a known quantity directly to the instrument or sample collection point? How would this compare with collecting actual samples in a processing or production environment?
  • Is enrichment required? • How long and what are the conditions and preparation needed for the enrichment? Will it resuscitate injured/ damaged cells?
  • What level of amplification is necessary for the detection technology to work?
  • Can the test tell if the organism is alive or dead?
  • What are the false positive and false negative rates? How do they depend on other conditions (e.g., length of enrichment time).
  • How specific is the test and what are the organisms for which there is cross-reactivity (e.g., a test for E. coli, vs STEC, vs O157:H7).
  • Does the test yield a viable culture from which additional analyses can be done?
  • If the fee is based on a price per test- what is the basis for that? Does this assume full capacity of a system (e.g., “batching” of samples)? What if only one sample was run at a time?
  • Does the vendor provide support with training of personnel?
  • Is there customer support access 24/7?
  • If the method it to be performed on-site, what kind of positive controls are needed and how can they be safely managed? What kind of technical expertise, personal protective equipment, and lab facilities are needed?

Summary Appendix

Developing an ecosystem of supplier networks across a range or areas is critical to ensure compliance with regulatory rules, speed to market, quality of product, and food safety. Establishing key parameters with appropriate guidelines will be critical to ensure that the suppliers are aligned with corporate governance in food safety and finance.

Area: Software
Guideline for consideration:

  1. Required level of competency to run
  2. Training availability
  3. Customizability after start-up, and on-going after users get familiar
  4. Users covered by initial cost
  5. Customer service requirements, resolution time requirements
  6. Can it be expanded outside the US, available languages
  7. Compatibility with SAP or other in-house software system
  8. Cost structure; one time start-up fees, maintenance fees
  9. Compatible with regulatory agencies
  10. Beta testing programs availability, output and format requirements
  11. Cost effectiveness – less people, faster output
  12. Breadth of customizing allowed and cost
  13. Data loss contingency
  14. Cadence of upgrades, cost
  15. Ability to view on mobile devices, remote sensing capabilities
  16. Area coverage in the U.S.
  17. Compliance compatibility
  18. Features for traceability, documenting corrective actions
  19. If data collection from one lab, compatibility with other labs

Area: Equipment
Guideline for consideration:

  1. Research and science support documentation
  2. Accessibility for cleaning/sanitation
  3. Training for operating, cleaning, and maintenance
  4. Validation studies
  5. Vendor certifications (OSHA, ASME, AWS, etc.)
  6. Software requirements
  7. Cost and expenses (operating)
  8. Pack/process/wash capacity, method of determination
  9. Ratio of wash water/product in wash tanks for wash lines

Area: Interventions (including antimicrobials)
Guideline for consideration:

  1. Liquid/solid or hardware, and registration under appropriate requirements:
  2. Describe the validation process. What are the standards used to ensure that ingredient or hardware or both is achieving the desired result of prevention, reduction, or elimination of microbial hazards?
  3. What are other factors to consider that affect the efficacy of the ingredient, hardware, or both.
  4. Organic or inorganic matter effect on efficacy.
  5. Confirm that substance will be effective on the surface applicable on which it will be used. Ensure no effect on the function and appearance of the surfaces.
  6. Product matrices used for validation

Area: Laboratory Services
Guideline for consideration:

  1. Results timing and turnaround
  2. State and Federal certifications on required tests
  3. Customer service contracts and support
  4. Dashboard capabilities, support team
  5. Software for data analysis, and support
  6. References from customers, institutes, universities

Area: Testing Kits
Guideline for consideration:

  1. Validation studies completed or in-process, and applicability to product/process
  2. Third party validation partners
  3. Supporting data
  4. Vendor customer service and training (type)
  5. PPE requirements
  6. Calibration and accuracy requirements
  7. Protocol to get number of cells to instrument for detection 
  8. Enrichment requirements; conditions, preparation
  9. Level of amplification necessary for detection technology to accurately measure
  10. Cost and price per test as it relates specifically to that product line
  11. Sample preparation requirement and alignment with validation studies
  12. Accuracy and precision; percent false negatives/positives
  13. Cross reactivity with different organisms
  14. Definition and type of indicators
  15. Microbe index
  16. Single/multiple target pathogens
  17. Arrays multiplex
  18. Operator skill level
  19. Time for results

Related Resources