Augmented Reality Clear sight for better laboratory results
Accuracy is paramount in laboratory settings and ensures that lab results are valid. Augmented reality (AR) can help to prevent errors. In this interview, Lynda Metref and Fabian Kappeler talk about an AR-assisted pipetting system and describe how this concept could be used in the laboratory.
Ms. Metref, Mr. Kappeler, at the MEDICA 2019 trade fair, you introduced an AR-assisted (AR) pipetting system as part of the MEDICA LABMED FORUM. What prompted the idea for this product?
Lynda Metref: We wondered how pipetting could be done in the future? How can we use technology to support pipetting accuracy? And since pipetting is still frequently done manually, what are the challenges for lab technicians?
We introduced a system with a camera that tracks objects on the laboratory bench. With the help of augmented reality (AR), we can superimpose process details to assist the user. This enables the tracking of events during the lab work. Image recognition helps identify the reagent that was picked up with the pipette and helps assess whether the volumes are accurate.
Right now, this integration of AR into pipetting procedures is still a concept we would like to explore with an industry partner. We pursue a user-centered approach and believe this is an important subject that could have future potential for new product development.
How does the system support laboratory work tasks?
Metref: Pipetting is a skill that requires long-term concentration. Technicians must accurately carry out the process steps, protocol the steps at the same time, and master the practical skills of handling reagents and lab equipment. It is demanding work. They also have to deal with distractions, all of which is stressful. Automated process documentation could relieve the burden on users and significantly improve the quality of work, especially as it pertains to error reduction.
Fabian Kappeler: Manual pipetting is used for small sample volumes and/or occasional procedures or a process that has to be modified. It takes concentration, and if you make a mistake, you may not notice it until much later with potentially enormous consequences. In research, this might mean that results ultimately do not match and the work you did for several weeks was all for nothing, or that precious, unique samples are now lost.
Our idea is to keep the benefits of manual pipetting, i.e., the efficiency and flexibility when pipetting small volumes, but still have the quality control and documentation of a typical automated pipetting system. These systems are usually programmed for high-throughput screening and large-volume samples that are always pipetted in the same way. However, automation does not efficiently support smaller batches.
Have you identified specific tasks where an AR system is especially beneficial?
Kappeler: We believe augmented reality is an exciting aspect in general. For example, it enables you to give users job instructions by superimposing the procedures for particular assays.
The entire process could be stored electronically as a protocol. The sequence of identified steps would then be automatically compared with the stored lab protocol. This allows the system to detect in real-time whether or not all steps have been performed accurately. This might even involve the system’s refusal to dispense from the pipette if it detects that the lab technician is about to make a mistake.
What do you think is next for “Laboratory 4.0”?
Kappeler: Digitization, networked instruments, and smart products are trends that permeate our world. The laboratory and its functions are still somewhat on the conservative end of this spectrum. Having said that, the aforementioned trends will eventually also make their way into the laboratory realm. Smart devices will increasingly support the various functions of laboratories, making life in the lab easier and results more accurate and reliable.
The interview was conducted by Timo Roth and translated from German by Elena O'Meara.