Abstract: In this research, we propose and demonstrate the concept of 'Self-Maintainability' which would be a key approach to building and designing software for versatile and intelligent laboratory automation systems. In recent years, significant strides have been made in automating biological experiments. Automation offers several benefits, including enhanced reproducibility and reduced workload for researchers. However, challenges arise in practice, such as the need for human oversight of consumables and addressing issues like misplaced samples or consumables. Effective management of consumables and the capability to adapt to disruptions, such as misplacements or changes in user requests, are crucial, irrespective of the experiment's specifics. Traditionally, an operation team handles these challenges. While human intervention suffices in repetitive, stable environments, it falls short in dynamic, large-scale operations. To address this, we introduce the concept of "self-maintainability" (SeM) – the system's ability to maintain itself to the state where it can function effectively in environments subject to material consumption, equipment wear, and potential operational disruptions caused by external disturbances. To realize the concept of SeM, we have developed the Earthcom, an autonomous experimental control system embodying SeM. Earthcom utilizes OT2Eye, an object detection model trained on labware, to process images from a webcam. The detection results are communicated to a large-scale language model (LLM), specifically ChatGPT, via a JSON embedded in the user's chat message. ChatGPT then interacts with a custom API server on the Opentrons OT-2 Raspberry Pi, utilizing Function Calling, enabling liquid handling. This system interprets natural language instructions, generates operational code for Opentrons OT-2, seeks human approval, and then carries out the experiment. It autonomously completes tasks not explicitly directed by users, like tip attachment, and integrates tip rack positioning using image recognition. Additionally, the system can first run the program in a simulated environment to ensure error-free operation before actual device implementation, with the language model automatically rectifying any detected errors. The system can accurately answer queries about the location of plates or tip racks and adjust its responses if the user repositions these items. Crucially, it identifies and suggests corrections for user errors, such as incorrect deck numbering for plate placement. Earthcom adeptly handles user errors and seamlessly integrates responses to issues related to consumable management and operational disturbances—areas traditionally overseen by human operators. By utilizing sensors and interactive communication with users, Earthcom actively collects vital information about constraints, including the status of consumables and equipment as well as user requests. This functionality elevates Earthcom beyond the realm of a standard automated system that merely executes user commands. Instead, it functions as an autonomous laboratory controller.
