UDT: Breaking the Data Transfer Bottleneck

http://udt.sourceforge.net/

DT is a reliable UDP based application level data transport protocol for distributed data intensive applications over wide area high-speed networks. UDT uses UDP to transfer bulk data with its own reliability control and congestion control mechanisms. The new protocol can transfer data at a much higher speed than TCP does. UDT is also a highly configurable framework that can accommodate various congestion control algorithms.

Key Features

Fast. UDT is designed for extremely high speed networks and it has been used to support global data transfer of terabyte sized data sets. UDT is the core technology in many commercial WAN acceleration products.

Fair and Friendly. Concurrent UDT flows can share the available bandwidth fairly, while UDT also leaves enough bandwidth for TCP.

Easy to Use. UDT resides completely at the application level. Users can simply download the software and start to use it. No kernel reconfiguration is needed. In addition, UDT's API is very similar to the traditional socket API so that existing applications can be easily modified.

Highly Configurable. UDT supports user defined congestion control algorithms with a simple configuration. Users may also modify UDT to suit various situations. This feature can also be used by students and researchers to investigate new control algorithms.

Firewall Friendly. UDT is completely based on UDP, which makes it easier to traverse the firewall. In addition, multiple UDT flows can share a single UDP port, thus a firewall can open only one UDP port for all UDT connections. UDT also supports rendezvous connection setup.

UDT is a protocol and library; it does not provide a tool/application for end users directly.
If you need a system software to support wide area data storage and distribution, please try Sector.

UDT Licensing

UDT is copyrighted software and the copyright holder is the Board of Trustees of the University of Illinois. The current UDT version 4 is under BSD license, while all previous versions were under GNU Library or Lessor GPL ( LGPL).

BSD license is one of the most flexible open source licenses. It allows you to redistribute (including to sell) and use UDT in either source or binary form, with or without modification. However, you MUST keep or reproduce the complete license in your redistribution.

For more information about UDT licensing, please contact us.

User Manual

• UDT version 4
• UDT version 3
• User manual is also available within the UDT release package.

http://www.douban.com/note/218239055/

UDT的简介@NowCode

2012-06-05 06:51:57

 

基于UDP的数据传输协议（UDP-based Data Transfer Protocol，简称UDT）是一种互联网数据传输协议。UDT的主要目的是支持高速广域网上的海量数据传输，而互联网上的标准数据传输协议TCP在高带宽长距离网络上性能很差。
顾名思义，UDT建于UDP之上，并引入新的拥塞控制和数据可靠性控制机制。UDT是面向连接的双向的应用层协议。它同时支持可靠的数据流传输和部分可靠的数据报传输。
由于UDT完全在UDP上实现，它也可以应用在除了高速数据传输之外的其它应用领域，例如点到点技术（P2P），防火墙穿透，多媒体数据传输等等。
UDT由开源软件作者谷云洪在美国伊利诺伊大学芝加哥分校攻读博士期间开发，并由他在毕业后继续维护和升级。UDT的开源软件可以在SourceForge上获取。

首先，先提前声明一下，我在这里并没有打算介绍UDT的详细设计实现（如果有兴趣可以参考：http://tools.ietf.org/search/draft-gg-udt-03），我只是在读完UDT的代码之后，总结一个大致的介绍，给准备使用该UDT协议的同仁们一个参考。

1. 基于UDP的可靠数据传输
学过网络的人都知道，在网络的网络层（Transport Layer）有两个比较重要的协议：UDP和TCP。
这两种协议的一个重要区别就是TCP提供可靠的数据传输，保证数据可以正确（内容正确和顺序正确）的发送到网络的另一端；而UDP则提供了“Best effort”的数据传输，不能保证数据会正确到达。但是UDP有一个优点，即UDP传输数据的时候不需要首先建立连接（TCP需要进行三次握手创建连接），这样UDP就可能能够提供较高的传输速率。
在某些情况下，UDP还有着TCP不能够完成的功能。很重要的一个就是做NAT打洞：在两台处于不同NAT网络的主机之间传输数据，首先就需要为建立起这两台主机之间的通道。这时，UDP便成了唯一的选择。
另外，由于TCP是一种优雅的（很有绅士风度）协议，该协议的AIMD设计最大程度的是网络带宽的利用公平（可以参考《计算机网络》这本书），这样在有些情况下，TCP的效率就显得较为低下，一个例子就是网格的分布式计算中，节点之间数据传输。
所以，UDP这种协议在有些情况下是必要的。但是，UDP只提供“Best effort”，并不保证数据传输的正确性。如此我们在使用UDP进行数据传输时就需要在网络应用层（Application Layer）为其提供可靠性保证。UDT正是这样的一种协议。

2. TCP式的数据传输
TCP较UDP有几个比较重要的特征：面向连接、拥塞控制、流量控制和重传机制等。这几个特性保证了TCP的可靠性数据传输。UDT这种协议同样也是通过这几个特性来实现基于UDP的可靠数据传输，不同点在于UDT的这几个特性是在网络的应用层实现的。
UDT的这几个特性在其文档中有详细的介绍：http://tools.ietf.org/search/draft-gg-udt-03

3. Berkeley Socket一样的应用程序接口（API）
UDT最终提供的API与Berkeley Socket的经典接口一样，提供了socket(), bind(), listen(), connect(), accept(), recv(), send()等接口，熟悉socket编程的人相信对这些已经非常熟悉了。除此之外，UDT还提供了用于异步通信机制的select()等接口。可以说，linux中为TCP实现的那些接口，在UDT中基本上（并不是全部，例如UDT不提供shutdown()函数）都有实现，我们可以像使用TCP socket一样使用UDT。
更详细的介绍可以参考用户手册：http://udt.sourceforge.net/udt4/index.htm

4. 不在RFC中
UDT并不属于RFC（Request For Comments），也就是说并不是作为一种标准出现的。但是，作为UDP可靠数据传输的实现，UDT有很高的使用价值。如果你要进行UDP的可靠数据传输，推荐使用UDT！

5、关于UDT P2P
UDT Connections

PowerFolder is able to connect computers via UDT network protocol. The UDT protocol is optimized for high speed data transfers. It also brings very handy features to run PowerFolder behind a Internet router or NAT device. UDT uses the UDP hole punching technique, which makes the configuration of port forwarding at the router obsolete. Many other popular peer-to-peer software (e.g. Skype) also use this technique to simplify the network setup process.

 

Supported platforms
UDT connections are supported on the following operating systems:

Windows Vista
Windows XP SP3
Windows 2000 SP4, Windows Server 2003 or Windows XP with Microsoft Visual C++ Redistributable Package

UDT is currently not supported on Mac or Linux. On these platforms PowerFolder is still able to connect with full relayed or HTTP tunneled connections. Since both methods requires the traffic to be relayed by the PowerFolder servers the transfer speed might not reach the maximum of your Internet connection. We strongly recommend tocorrectly setup any firewall and router on these platforms for optimal performance.

Using UDT (hole punching aka NAT traversal)
By default PowerFolder tries to connect via UDT (hole punching mode) if no direct TCP connection is possible. Before a hole punching UDT connection can be established it is required, that all computers are connected to the PowerFolder servers. These act as relay for the connection initialization handshake – afterwards the actual communication happens directly between the connected computers. PowerFolder automatically tries to connect to the PowerFolder servers and usually gets at least connected via HTTP-Tunnel.

UDT (hole punching) is enabled by default to make the setup process as easy as possible. You can disable UDT (hole punching) under Preferences/Networking/Use UDT connections

Using UDT (direct)
It is also possible to directly connect two computers via UDT without being necessarily connected to the PowerFolder servers. This option is disabled by default and only recommend when using PowerFolder over a high speed data network. It requires unrestricted communication between the computers without getting blocked by any firewall, router or NAT device.
You can enable UDT (direct) connections under Preferences/Pro Settings/Direct UDT connect. This option controls if PowerFolder should try to connect via UDT directly before trying to connect via TCP. When enabled PowerFolder also listens for incoming UDT connections on the configured port(s) under Preferences/Network/Advanced/Port. The default port is 1337. Attention: Due to conflicts of UDT on port 1337 (UDP) PowerFolder disables the network broadcasting in LAN, which also uses the the default port 1337 (UDP).

Troubleshooting

Check if UDT is available on your platform. The option “NAT traversal” should show up under Preferences/Network. If not please install the Microsoft Visual C++ Redistributable Package.
Try to manually connect the computers under Find Friends/Connect, enter “IP:port”, e.g. 192.168.0.10:1337
To check if a connection is established via UDT please turn on debug logging in PowerFolder. Then open a chat with the target computer – the info should show “ESUDTConnectionHandler” (instead of ESConnectionHandler).

Execute “netstat -a -p UDP” on the command line. There should be one line with “0.0.0.0:1337″ (Or a different port and network address if configured)
Check the debug logs for any problems with UDT components.
High CPU load: Disable ZIP compression in LAN under Preferences/Advanced and configure all computers to be detected on LAN via the LAN-IP-List.