Discover Interprocess Communication (IPC)

Click to jump down this page to a topic:

Purpose
Background
IPC Components
- Discover BTCL IPC Commands
- Biosym/MSI IPC Client Application Program Interface
IPC Procedure

Purpose

The ipc command and related commands (see Background below) provide the means to exchange data and BTCL code between the Discover program and user programs running on the same or different hosts.

Background

Through the IPC package, the Discover program has the ability to establish bidirectional communication with external FORTRAN/C/C++ programs written by Discover users. The purpose of this is to provide scientific application developers an efficient means of exchanging information such as atom coordinates, energies, energy gradients, and other information between the Discover program and their own programs. This is accomplished without linking code into the Discover program: communication is implemented via named pipes for the local host and via TCP/IP for remote connections. Some of the benefits and types of IPC applications which can be developed include:

Algorithms that modify the behavior of the standard Discover program, e.g., modified minimizers, dynamics.
Programs that add a special energy, gradient, stress, and/or Hessian to the normal Discover energy evaluation.
Programs that use the Discover program as an ``engine'' for computing useful molecular mechanics properties.

In addition to communicating with user programs, the Discover program can also be programmed to communicate with additional instances of itself. This allows for a simple implementation of network parallelism.

IPC Components

The IPC capabilities of the Discover program have two components: the Discover BTCL interface and the client application program interface (API).

Discover BTCL IPC Commands

ipc procedure: Primary access to Discover IPC is through the ipc procedure. This procedure is used in BTCL (run_name.inp) scripts to initiate IPC connections to other programs.
object varName read/write operations: The read and write operations of the object command allow common BTCL objects to be read or written via the IPC connection.
discoverIpc command: This command implements the basic IPC functions for the Discover program. In general, you do not need to use this command directly; all common IPC functionality is encapsulated in the ipc procedure. However, for some users, the discoverIpc command allows more flexible configuration of the Discover program's IPC capabilities.
discoverIO command: As with the discoverIpc command, discoverIO is generally not used directly by the script writer. During a Discover run's energy evaluation, the discoverIO command is used to coordinate fast exchanges of atomic coordinates and energy-related values between the Discover program and an external program.

Biosym/MSI IPC Client Application Program Interface

The client API consists of a small number of C and FORTRAN functions which client programs can use to initiate an IPC connection to the Discover program and then exchange data. The complete source code for the client-side IPC is provided without restrictions on its use and with examples.

ipc Procedure

The ipc procedure is implemented in Tcl and encapsulates the Discover program's IPC capabilities into a single procedure.

Note: The distinction between a command and a procedure in Tcl is in how the Tcl verb is implemented. Verbs implemented in C-code are usually called commands, while verbs implemented as a body of Tcl are called procedures. Syntactically, a command and a procedure are identical, and so we will drop the distinction and just refer to the ``ipc command''.

Syntax

The ipc command is similar in structure to the object command:

ipc varName operation ?-option ...? args ?args ...?

varName is the name of a Tcl variable that is used as a reference to the IPC object in question. The varName Tcl variable set by the ipc varName init ... command; subsequent IPC operations use this variable to determine which IPC connection to operate on.

For many ipc commands, varName may be a Tcl variable that refers to an IPC object or a Tcl file handle. Where the command behavior differs based on whether the handle is an IPC object or Tcl file handle, the descriptions of each operation will indicate.

operation is one of the following:

Common IPC Operations

init: Initializes an IPC connection to another program.
info: Determine characteristics of an IPC connection.
shutdown: Close an IPC connection.

Advanced IPC Operations

accept: Accept a connection request on a listening socket.
connect: Open a connection to a TCP/IP host/port.
eval: Get data from an IPC connection and evaluate it as a BTCL command string.
listen: Start a listening socket so that other programs including other instances of the Discover program can connect to this running Discover.
poll: Determine if input is available on a connection or a connection is pending on a listening socket.
send: Send a raw string over an IPC connection.
server: Put the Discover program into ``server'' mode.

Common IPC Operations

init

ipc varName init ?-options ...? ?args ...?

Used to establish a link between the Discover program and another program. The external process may be running on the same host or a remote host. The external process may be another instance of Discover. This mechanism can be used to implement network parallelism.

Generally, the external process uses the BiosymIpc client library to establish a connection back to the Discover program. As soon as the external process calls the BiosymIpc_Init C function or the BiosymIpc_Initf FORTRAN function, the connection is established and the Discover program moves on to the next BTCL command.

The args specify the external process to run. These arguments are passed to the Tcl exec command. The input and output of the external process can be redirected to specific files.

Upon establishing the connection, a handle is created in the caller's variable named varName. This variable must be used in all subsequent operations on the newly created IPC channel. When varName is unset, either explicitly or by going out-of-scope, the IPC connection is shut down.

options

-autoEnergy atomSpecification

A common use for an external program is to add an energy component to the Discover energy evaluation routine. The -autoEnergy option takes care of the details involved in establishing this external energy contribution using the BTCL restraint command. During an energy evaluation, the coordinates of the atoms specified in atomSpecification are transferred to the external program. The Discover program then expects to receive an energy contribution, gradients, and possibly other information. See the discoverIO command for more details.

-connect host:port | host port

Directs the currently running Discover program to establish an IPC connection to the specified host and port, rather than invoke a new executable. The Discover program generally uses this when it is invoked by another instance of Discover to act as a server. See Advanced IPC operations for details on other ways to use the -connect option to the init operation.

-energyName string

Optionally used in conjunction with -autoEnergy to provide a name for the energy contribution given by the external process. This name will appear in the Energy database under the major heading ``External''.

-port port

Optionally used in conjunction with the -connect option to specify a port number to attempt to connect to.

-remote hostName

This option specifies that the external process is to be run on a remote host. You must have permission to perform an rsh to the specified remote host.

info

ipc varName info ?-option ...?

Returns information about the IPC connection specified. If no options are given, all information is returned. Use info with no options to see what options are available. The data for each IPC connection are stored in a global associative array called IPC. The keys into this array consist of a unique ``handle'' name for each connection, concatenated with a period and a member name as follows:

IPC(handle.inFH)           Discover input Tcl input file handle
IPC(handle.outFH)          Discover input Tcl output file handle
IPC(handle.remoteVersion)  BiosymIpc version in use by external
IPC(handle.localVersion)   BiosymIpc version in use by Discover
IPC(handle.restraint)      optional energy restraint object
IPC(handle.command)        external program invocation
IPC(handle.atomList)       optional atom specification

shutdown

ipc varName shutdown

Closes the specified IPC connection. The same effect can be achieved with unset varName.

Advanced IPC Operations

Many of the following operations are used internally by the ipc command. Although documented here, most users will not find it necessary to use these operations unless they are developing very special-purpose Discover communication tasks.

accept

ipc varName accept ?hostname?

Accept a connection request on a previously opened listening socket specified by varName. varName may contain a normal Tcl file handle or an IPC object handle. If an IPC object is specified, the inFH (input file handle) member of the IPC object is used to refer to the listen socket.

If the optional hostname is present, the peer attempting to connect must have originated from the specified host.

The result is an ordered list of two Tcl file handles. The first handle is to be used for reading on the connection; the second handle is to be used for writing on the connection.

connect

ipc varName connect hostname port

Attempt to make a connection to the specified host and TCP/IP port. The port can be a service name or a numerical port specification. The result is an ordered list of two Tcl file handles. The first handle is to be used for reading on the connection; the second handle is to be used for writing on the connection.

eval

ipc varName eval ?-nowait?

Read input from an IPC handle or Tcl file handle and evaluate the incoming data as a BTCL command. Normally, the eval operation waits until input is available on the specified connection. If the -nowait option is specified, the command returns an empty string if no input is available on the specified connection.

The result that is returned is the result of the evaluation of the input data.

The incoming data is handled differently depending on whether varName contains an IPC handle or a simple Tcl file handle:

IPC handle: The actual Tcl file handle used for input is obtained from the outFH member of the IPC object referred to. (Use ipc varName info inFH to obtain the input Tcl file handle for an IPC object.) The file handle is read and interpreted as an IPC frame which must be of type STRING. The strings in the incoming IPC frame are concatenated and then evaluated in the BTCL interpreter (via Tcl's eval command).
Tcl file handle: A single line of input is read. If the result is not a complete Tcl command, then additional lines are read until a complete command has been composed. The input is then evaluated in the BTCL interpreter (via Tcl's eval command).

listen

ipc varName listen ?port?

Establish a listen socket. If the optional port number is given, then an attempt to bind the listen socket to that port is made.

The result is an ordered list of length two. The first list element is a readable Tcl file handle which can be used to poll the listen socket. The second list element is the port number actually assigned to the listen socket. Typically, this is transmitted to an external program via an environmental variable or command line option in order to let the external program know where it should connect back to upon startup.

poll

ipc varName poll ?timeoutMilliseconds?

Determine if input is available or a connection is pending. The input source is either an IPC handle or a Tcl file handle contained in varName. If the optional timeout value is given, the poll command will wait up to the specified number of milliseconds before returning.

The result is 1 if data is available for reading or a connection request has appeared, and 0 if not.

send

ipc varName send ?-nonewline? ?args?

Write strings (usually BTCL/Tcl commands) to the IPC handle or Tcl file handle contained in varName. Normally, args is a Tcl list of strings. Each string is sent with a newline appended unless the -nonewline option is used.

There is no returned result (empty string).

The outgoing data is handled differently, depending on whether varName contains an IPC handle or a simple Tcl file handle:

IPC handle: The actual Tcl file handle used for output is obtained from the inFH member of the IPC object referred to. (Use ipc varName info outFH to obtain the output Tcl file handle for an IPC object.) The arguments are packed into a temporary BTCL string object, and the object varName write operation is used to send the data.
Tcl file handle: Each argument list element is sent as a separate line using the standard Tcl puts command.

server

ipc varName server

The server operation implements a simple loop using the eval command. Input is collected from the IPC handle or Tcl file handle and evaluated until the connection is closed.

Main access page

Advanced-Use access.