| <!-- source: socat-exec.html --> |
| <!-- Copyright Gerhard Rieger 2009 --> |
| <html><head> |
| <title>Socat address chains</title> |
| <link rel="stylesheet" type="text/css" href="dest-unreach.css"> |
| </head> |
| |
| <body> |
| |
| <h1>Socat address chains</h1> |
| |
| <a name="introduction"/> |
| <h2>Introduction</h2> |
| <p>Socat version 2 can concatenate multiple modules and transfer data between |
| them bidirectionally. |
| <p> |
| |
| |
| <a name="example1"/> |
| <h2>Example 1: OpenSSL via HTTP proxy</h2> |
| |
| <span class="frame"><span class="shell"> |
| socat - "OPENSSL,verify=0 | PROXY:secure.domain.com:443 | TCP:proxy.domain.com:8080" |
| </span></span> |
| |
| <p>This command does the following: socat connects to proxy.domain.com on port |
| 8080 and sends a proxy CONNECT request for secure.domain.com port 443; this is |
| similar to the proxy address available in version 1. Once the proxy server |
| acknowledges successful |
| connection to the target (SSL) server, socat starts SSL negotiation and then |
| transfers data between its stdio and the SSL server. |
| </p> |
| |
| |
| <a name="basics"/> |
| <h2>Address chain basics</h2> |
| |
| <p>socat version 1 was able to open two addresses and transfer data between |
| them. "Addresses" could be just sockets or other file descriptors, or could |
| be a little more complex like proxy client or OpenSSL server and client. It |
| was, though desirable, practically not possible to combine complex address |
| types, or to use other socket types than the predefined ones (usually TCP) |
| with complex addresses. |
| </p> |
| <p>socat version 2 has been designed to overcome these limitations. First, the |
| complex address types are now separated from the underlying file descriptor |
| types. Second, complex addresses that are now called <em>inter addresses</em> |
| can be concatenated to an <em>address chain</em>; however, an <em>endpoint |
| address</em> that just provides file descriptors must be the last component |
| of an address chain. |
| </p> |
| <p>The socat invocation takes two address chains, opens them, and transfers |
| data between them. |
| </p> |
| <p>An address chain consists of zero or more inter addresses and one endpoint |
| address, all separated by the pipe character '|'. When starting socat from |
| the command line these characters and the optional spaces must be protected |
| from the shell; it is recommended to put each address chain under double |
| quotes. |
| </p> |
| <p>The (bidirectional) inter addresses that are available with a socat |
| implementation can be listed with the following command: |
| </p> |
| <span class="frame"><span class="shell"> |
| socat -h |egrep 'b ..b groups='</span></span> |
| <p>A full socat 2.0.0-b3 program provides the following inter addresses: |
| </p> |
| <table border=1> |
| <tr><th>name</th><th>description</th></tr> |
| <tr><td>NOP</td><td>transfers data unmodified</td></tr> |
| <tr><td>OPENSSL-CLIENT</td><td>performs OpenSSL client negotiation, then |
| encrypts/decrypts data</td></tr> |
| <tr><td>OPENSSL-SERVER</td><td>performs OpenSSL server negotiation, then |
| encrypts/decrypts data </td></tr> |
| <tr><td>PROXY</td><td>performs proxy CONNECT client negotiation, then |
| transfers data unmodified</td></tr> |
| <tr><td>SOCKS4</td><td>performs socks 4 client negotiation, then |
| transfers data unmodified</td></tr> |
| <tr><td>SOCKS4A</td><td>performs socks 4a client negotiation, then |
| transfers data unmodified</td></tr> |
| <tr><td>SOCKS5</td><td>performs socks 5 TCP client negotiation, then |
| transfers data unmodified</td></tr> |
| <tr><td>TEST</td><td>appends > to forward, and < to reversely |
| transferred blocks</td></tr> |
| <tr><td>EXEC</td><td>invokes a program |
| (see <a href="socat-exec.html">socat-exec.html</a>), then transfers data unmodified</td></tr> |
| <tr><td>SYSTEM</td><td>invokes the shell (see <a href="socat-exec.html">socat-exec.html</a>), then transfers data unmodified</td></tr> |
| |
| </table> |
| |
| |
| <a name="reverse"/> |
| <h2>Reverse address use</h2> |
| |
| <p>Inter addresses have two interfaces. In most cases one of |
| these can be seen as a <em>data</em> interface, where arbitrary data |
| traffic may occur, and the other as <em>protocol</em> interface where the |
| transferred data has to follow some rules like socks and HTTP protocol, or |
| valid encryption. |
| </p> |
| <p>Bidirectional inter addresses are usually implemented such that their data |
| interface is on the "left" side, and the protocol interface on the "right" |
| side. |
| </p> |
| <p>It may be convenient to build an address chain where one or more inter |
| addresses work in the reverse direction, so their protocol side is connected |
| to left neighbor in the chain using the protocol, and the data side is |
| connected to the right neighbor for raw data transfer. socat allows to use |
| inter addresses in <em>reverse</em> direction by preceding their keyword with |
| ˆ. |
| </p> |
| |
| |
| <a name="example2"/> |
| <h2>Example 2:</h2> |
| |
| <p>Endpoint addresses that fork should usually build the first socat address |
| chain, without inter addresses. For creating an SSL to TCP gateway that |
| handles multiple connections the following command line does the job: |
| </p> |
| <span class="frame"><span class="shell"> |
| socat TCP-LISTEN:443,reuseaddr,fork "^OPENSSL-SERVER,cert=server.pem | TCP:somehost:80" |
| </span></span> |
| |
| <p>Without the reverse usage of the SSL server address, socat would "speak" |
| clear text with the clients that connected to its left address, and SSL to |
| somehost. |
| </p> |
| |
| |
| <a name="unidirectional"/> |
| <h2>Unidirectional data transfer</h2> |
| |
| <p>Like in socat version 1, it is possible to specify unidirectional transfers |
| with version 2. Use socat options <a href="socat.html#OPTION_u">-u</a> or |
| <a href="socat.html#OPTION_U">-U</a>. |
| </p> |
| <p>Unidirectional transfer must be supported by the involved inter addresses; |
| e.g., SSL requires a bidirectional channel for negotiation of encryption |
| parameters etc. |
| </p> |
| <p>It is possible to mix uni- and bidirectional transfers within one address |
| chain: Think of a simple file transfer over SSL. |
| </p> |
| <p>The socat help function can tell us which address types support which kinds |
| of transfer:</p> |
| <span class="frame"><span class="shell"> |
| socat -h |egrep 'openssl-server'</span></span> |
| <p>gives the following output: |
| </p> |
| <p><pre> openssl-server rwb b groups=CHILD,RETRY,OPENSSL |
| openssl-server:<port> rwb groups=FD,SOCKET,LISTEN,CHILD,RETRY,RANGE,IP4,IP6,TCP,OPENSSL</pre> |
| </p> |
| <p>The <tt>rwb b</tt> flags mean that this address type can handle readonly, |
| writeonly, and bidirectional transfers on its left (data) side, but only |
| bidirectional on its right (protocol) side. |
| </p> |
| <p>The second line describes the (version 1) endpoint form: no right side |
| traffic kinds are specified because this address type establishes its protocol |
| communication itself. |
| </p> |
| |
| <a name="dual"/> |
| <h2>Dual inter addresses</h2> |
| |
| <p>In socat version 1 it was already possible to combine two unidirectional |
| addresses to one bidirectional address. This idea has been extended in version |
| 2: Two unidirectional inter addresses can be combined to one bidirectional |
| transfer unit. |
| </p> |
| <p><em>Note: in version 1, the dual specification was like |
| </em><tt>righttoleft!!lefttoright</tt><em>. In version 2, it is: |
| </em><tt>lefttoright%righttoleft</tt><em>. This is the only major incompatibility |
| between versions 1 and 2.</em> |
| </p> |
| <p>With the few already available inter address types, this feature has no |
| practical use except with <a href="socat-exec.html">exec and system</a> type |
| addresses. However, the general function shall be described using the |
| hypothetical inter address types <tt>gzip</tt> and <tt>gunzip</tt>. |
| </p> |
| <p>Let us design these inter address types: <tt>gzip</tt> is a module that |
| reads arbitrary data on its left ("data") side, compresses it, and writes the |
| compressed data to its right (protocol side) neighbor. |
| <!-- Data that arrives onits right side is uncompressed and passed to the |
| left neighbor. --> |
| </p> |
| <p><tt>gunzip</tt> reads gzip compressed data on its left side and writes the |
| raw uncompressed data on its right side. |
| </p> |
| <p>socat can combine these to provide a bidirectional compress/decompress |
| function:<br> |
| <tt>gzip%gunzip</tt> |
| </p> |
| <p>Data coming from the left is passed through gzip and sent to the right; |
| data coming from the right is passed through gunzip and sent to the left. |
| </p> |
| <p>When the reverse functionality is desired this arrangement does the job:<br> |
| <tt>gunzip%gzip</tt> |
| </p> |
| |
| |
| <a name="fork"/> |
| <h2>fork</h2> |
| |
| <p>socat provides the <tt>fork</tt> address option for uses like network |
| servers where multiple clients can connect and are handled in parallel in |
| different socat sub processes. |
| </p> |
| <p>When the sub processes should work independently (share no socat file |
| descriptors) the fork option must be applied to the last component of the |
| first address chain. For better readability it is advisable to have only the |
| "left" endpoint address in the left chain and put all intermediate addresses |
| into the right chain. |
| </p> |
| |
| |
| <a name="understanding"/> |
| <h2>Understanding chain implementation</h2> |
| |
| <p>The idea of concatenated modules in socat is not new. But a few attempts to |
| completely rewrite and enhance the socat transfer engine |
| were never completed. At last, it was decided to choose an approach that |
| requires only moderate changes to socats transfer engine and the existing |
| address types. |
| </p> |
| <p>Think of several socat1 like processes somehow combined - with an abstract |
| operator || : |
| </p> |
| <span class="frame"><span class="shell"> |
| socat - openssl || socat - proxy:secure.domain.com || socat - tcp:proxy.domain.com:8080 |
| </span></span> |
| <p>The solution was to put all these into one process but have each socat engine |
| run in its own thread. The transfer between the engines goes over socket |
| pairs, so the engines see file descriptors as usual. The main work then was |
| to implement the functionality for opening address chains which includes |
| parsing, creating socket pairs and threads, combining the addresses, taking |
| care of unidirectional, dual, and reverse addresses etc. |
| </p> |
| <p>Here is the socat version 2 command line of example 1:<br> |
| <tt>socat - "OPENSSL,verify=0 | PROXY:secure.domain.com:443 | TCP:proxy.domain.com:8080"</tt> |
| <p>A schematic representation of how this is realized in socat:<br> |
| <tt>STDIO - engine[thread 0] - OPENSSL - socket pair - (FD) - engine[thread 1] |
| - PROXY - socket pair - (FD) - engine[thread 2] - TCP</tt> |
| </p> |
| <p>where FD means a trivial address similar to the FD (file descriptor) address |
| type. |
| </p> |
| <p>For debugging address chains it proved useful to write down two lines and to note the actual file descriptor numbers:</p> |
| <pre> STDIO ^ OPENSSL | ^ PROXY | ^ TCP |
| 0,1 ^ 6 | 7 ^ 4 | 5 ^ 3</pre> |
| <p>The symbol <b>ˆ</b> means a socat transfer engine. |
| </p> |
| |
| <p>Now the implementation of the reverse address feature should be easier to |
| understand. While a forward address is put to the right side of its |
| engine, a reverse address is just put to the left side. Example 2 can be |
| explained so: |
| </p> |
| <p>Example 2 command line:<br> |
| <tt>socat TCP-LISTEN:443,reuseaddr,fork "^OPENSSL-SERVER,cert=server.pem | |
| TCP:somehost:80"</tt> |
| </p> |
| <p>Schematic representation:<br> |
| <tt>TCP-LISTEN - engine[thread 0] - (FD) - socket pair - OPENSSL-SERVER - |
| engine[thread 1] - TCP</tt> |
| </p> |
| <p>Debug schema:<br> |
| <pre> |
| TCP-L ^ | SSL-SERV ^ TCP |
| 3 ^ 5 | 6 ^ 4</pre> |
| |
| |
| <a name="commtypes"/> |
| <h2>Communication types</h2> |
| |
| <p>For communication between the address modules of consecutive transfer |
| engines socat provides pairs (or quadruples) of file descriptors. You may |
| think about these as two normal UNIX pipes (fifos), one for left-to-right and |
| the other for right-to-left data transfer. |
| </p> |
| <p>There are a few requirements that these file descriptors should fulfill, |
| however they are different depending on the libraries used by the inter |
| address modules (e.g. libopenssl) or by external programs that are involved |
| (see <a href="socat-exec.html">socat-exec.html</a>). |
| </p> |
| <p>The factors to consider for these file dscriptors are: |
| </p> |
| <ul> |
| <li>Half close: when a module terminates communication on its write channel, |
| its read channel should still stay open.</li> |
| <li>Half close method: A module might half close a connection |
| using <tt>close()</tt> or <tt>shutdown()</tt> methods.</li> |
| <li>Buffering: The output buffering behaviour of some modules can be |
| influenced by the type of file descriptor</li> |
| <li>INET: Some external programs require a TCP/IPv4 file descriptor</li> |
| </ul> |
| <p>This table lists the available communication types and their |
| properties:</p> |
| <table border=1> |
| <tr><th>comm.type</th><th>half close with close()</th><th>allows shutdown</th><th>avoids buffering</th><th>TCP/IPv4</th></tr> |
| <tr><td>socketpairs</td><td>OK</td><td>OK</td><td>no</td><td>no</td></tr> |
| <tr><td>socketpair</td><td>no</td><td>OK</td><td>no</td><td>no</td></tr> |
| <tr><td>pipes</td><td>OK</td><td>no</td><td>no</td><td>no</td></tr> |
| <tr><td>ptys</td><td>OK</td><td>no</td><td>yes</td><td>no</td></tr> |
| <tr><td>tcp</td><td>no</td><td>yes</td><td>no</td><td>yes</td></tr> |
| </table> |
| |
| <p>The default is socketpairs. |
| </p> |
| <p>The overall communication type can be chosen using the <a href="socat.html#option_c"><tt>-c</tt></a> socat |
| option. With socat 2.0.0-b3 it is not possible to use different communication |
| types in one process (exception: right side of exec/system modules) |
| </p> |
| |
| <small>Copyright: Gerhard Rieger 2009</small><br> |
| <small>License: <a href="http://www.fsf.org/licensing/licenses/fdl.html">GNU Free Documentation License (FDL)</a></small> |
| </p> |
| |
| </body> |
| </html> |