A physics driven neural networks-based simulation system (PhyNNeSS) and its application to local and remote surgery simulation.

摘要

In this thesis we propose a novel physics-driven neural networks-based simulation system (PhyNNeSS) that is capable of simulating the response of nonlinear deformable objects in real time. While an update rate of 30 Hz in considered adequate for real time graphics, a much higher update rate of about 1 kHz is necessary for haptics. Physics-based modeling of deformable objects, especially when large nonlinear deformations and complex nonlinear material properties are involved, at these very high rates is the most challenging task in the development of real time simulation systems.;In PhyNNeSS, an off-line pre-computation step is used in which a database is generated by applying carefully prescribed displacements to each node of the finite element models of the deformable objects. The data in then condensed into a set of coefficients describing neurons of a radial basis function network (RBFN). The trained neural networks can then be used in real time computations. We show, through error analysis, that the scheme is scalable, with the accuracy being controlled by the number of neurons used in the simulation. More neurons may be chosen for higher fidelity but slower simulation while fewer neurons may be chosen for coarser, but more rapid simulation. We present realistic simulation examples from interactive surgical simulation with real time force feedback.;PhyNNeSS is then applied to the solution of nonlinear coupled electro-thermal problems arising in monopolar electrosurgery. An instrumented rig was setup to perform monopolar electrocautery with the controlled motion using a robot and resulting temperature distribution on the surface was recorded. Comparison of experimental results, finite element solution and PhyNNeSS establishes the effectiveness of PhyNNeSS in solving multi-physics problems.;The major advantage of PhyNNeSS is its scalability which is essential for tele-training and tele-mentoring applications when the trainees are geographically distributed but computational resources are centralized. Hence, we have developed a multi-user collaborative surgical simulation environment based on PhyNNeSS. The major problem in such simulators that limits their use is that they tend to become unstable in the presence of time delays among the participants. A hybrid network architecture is proposed and tested in which the server can update the clients with just enough information required to independently simulate localized interaction without the necessity to interact with the server at every time step. Results for three classes of interaction, global-scale (RPI-Tokyo), continental scale (RPI-UWashington) and local area network were tested for interactive surgical collaboration experiments, each for three different accuracy levels.