UDP vs TCP (hands-on experience required)

Question

Dearest colleagues,
I've done a few hours of research learning what I can about TCP vs UDP. I could still use some input from those of you who have used both in real-world, scaleable, distributed, commercial, and successful projects.
I have a new position at a new company. My part of this project consists of 3 types of distributed Java server programs: desktop client, data server, and controller server. Thousands of desktop clients will be periodically transferring "megs" of data to several data servers. The (singular) controller will persist metadata into a database, and also coordinate data transfer between the desktop clients and the data servers.
TCP seems appropriate for all "control" and metadata communications between all these components. Everyone talks to the controller for everything, except for the periodic transfer of the "actual" data. The desktop clients send that data directly to the appropriate data server using either UDP or TCP.
One key project requirement is the ability to provide network throttling control, so as not to saturate the client's LAN or WAN. This product mustn't disrupt the client's real-time network data (like VoiP and video conferencing). I expect the controller server to manage this.
Someone with more seniority than me at this company has suggested TFTP (which uses UDP) as the best protocol for the data transfer protocol. This person also claims that UDP allows larger packet sizes and less protocol overhead, resulting in greater data transfer rates than TCP.
I have a number of concerns and questions:
1) Can't the packet sizes be set for both UDP and TCP? If so, can't one equal both (in terms of average overhead) just by adjusting the packet sizes?
2) I realize that one needs to accomodate UDP droppage in order to compare throughput with TCP. In a LAN environment is it reasonable to assume that droppage would be minimal or zero? )Otherwise wouldn't it be a LAN problem someone would be fixing?)
3) If #2 above allows me to assume minimal UDP droppage, does the underlying packet protocol overhead difference give UDP a substantial throughput advantage? If so, how much (roughly)?
4) I believe UDP packets are treated as "high priority" but "low reliability". For my application, doesn't that mean that UDP is more likely to cause problems by competing for VoiP and VidCon bandwith?
5) What about trying to write my servers to allow for either UDP or TCP to be
used for data transfer? If you have done this, would you recommend for or against it? Why?
6) If you had to recommend just one book for me for right now (given this project), which would it be:
6a) Java Distributed Computing (O'Reilly) by Jim Farley
6b) Internetworking with TCP/IP by Doug Comer's
6c) Unix Network Programming by Richard Stevens
Being the New Kid on the Block at this company, I need to know all I can as I navigate the politics with this more senior person. I'm no newbie - I've been programming professionally for 27 years, the last 6 in Java including networking, sockets, JDBC, etc.
Thank you very much for your assistance.

Answer 1

Dearest colleagues,
I've done a few hours of research learning what I can
about TCP vs UDP. I could still use some input from
those of you who have used both in real-world,
scaleable, distributed, commercial, and successful
projects.
I have a new position at a new company. My part of
this project consists of 3 types of distributed Java
server programs: desktop client, data server, and
controller server. Thousands of desktop clients will
be periodically transferring "megs" of data to
several data servers. The (singular) controller will
persist metadata into a database, and also coordinate
data transfer between the desktop clients and the
data servers.
TCP seems appropriate for all "control" and metadata
communications between all these components.
Everyone talks to the controller for everything,
, except for the periodic transfer of the "actual"
data. The desktop clients send that data directly to
the appropriate data server using either UDP or TCP.Key feature of TCP is connection orianted and reliablity. If you need long sessions with high reliability TCP is the way to go.
(Long sessions means the client do multiple transaction through same session)
But reliablity of TCP comes at a cost. (Less bandwith efficiency) but of course if you need to transfer data I say TCP is better.
But if you want your clients need to work in unreliable network links (Onse that breakes down often) TCP might cause problems since you have to re establish the connection every time there is a problem.
Also if the sessions are short then UDP is better becouse if you Use TCP for very short sessions Connection establishment and closing can take more time that the actual data transfer.
But if you use UDP you must deal with the unreliability.
One key project requirement is the ability to provide
network throttling control, so as not to saturate the
client's LAN or WAN. This product mustn't disrupt
the client's real-time network data (like VoiP and
video conferencing). I expect the controller server
to manage this.It is true that TCP add more owerhead than UDP but in most cases compaired to the amount of data you transfer through network that is nothing.
Key to save the bandwith is to designe a efficint communication protocol which minimize the amount of data that you have to transfer.
Also you can Implement input/output streams which can limit the number of bytes that can be written or read through them. (Even though that got nothing to do with networking that will give you full control over the bandwidth usage by the programs)
Someone with more seniority than me at this company
has suggested TFTP (which uses UDP) as the best
protocol for the data transfer protocol. This person
also claims that UDP allows larger packet sizes and
less protocol overhead, resulting in greater data
transfer rates than TCP.TFTP is good for transfering small files through networrks since there is less overhead but when it comes to large amounts of data there is no much difference.
But TFTP have a advantage over ureliable networks.
(If the networks fails TCP connection fails, but in UDP a network faliour is just a loss of several datagrams which the programs anyways should be able to handle)
I have a number of concerns and questions:
1) Can't the packet sizes be set for both UDP and
TCP? If so, can't one equal both (in terms of
average overhead) just by adjusting the packet
sizes?In tcp packet size is not a big issue since data are transfered through stream. Once the streams are open you can keep them opena as long as you want and send as many bytes as you want.
2) I realize that one needs to accomodate UDP
droppage in order to compare throughput with TCP. In
a LAN environment is it reasonable to assume that
droppage would be minimal or zero? )Otherwise
wouldn't it be a LAN problem someone would be
fixing?)Even though UPD is said to be unreliable if the programs are written well. In a LAN UDP is almost as areliable as TCP(This is depend on the reliability of LAN)
Thats why it says when you write UDP applications you must test them in slow unreliable Wan links.
3) If #2 above allows me to assume minimal UDP
droppage, does the underlying packet protocol
overhead difference give UDP a substantial throughput
advantage? If so, how much (roughly)?It depends
Ex:-
Lts assume UDB header is 25 bytes and TCP heade is 50 for simplicity
If your data size is 10 byte.
lets say UDP adds overhead of 5 bytes
now the efficiency is 10/(10+25) = 0.38
If TCP adds 10 overhead bytes
now the efficiency is 10/(10+50) = 0.16
lets doa the same calculation for 1000 bytes
UDP
1000/(1000 + 25) = 0.97
TCP
1000/(1000+ 50) = 0.95
So it depends on your application and data volume
4) I believe UDP packets are treated as "high
priority" but "low reliability". For my application,
doesn't that mean that UDP is more likely to cause
problems by competing for VoiP and VidCon bandwith?No Priority is set on IP level so it can be applied to both TCP and UDP
5) What about trying to write my servers to allow
for either UDP or TCP to be
used for data transfer? If you have done this, would
you recommend for or against it? Why?For data transfer I recomend TCP becouse reliability is important.
If you use UDP you have to handle reliability (Lots of programming)
6) If you had to recommend just one book for me for
right now (given this project), which would it be:
6a) Java Distributed Computing (O'Reilly) by Jim
Farley
6b) Internetworking with TCP/IP by Doug Comer's
6c) Unix Network Programming by Richard Stevens
Being the New Kid on the Block at this company, I
need to know all I can as I navigate the politics
with this more senior person. I'm no newbie - I've
been programming professionally for 27 years, the
last 6 in Java including networking, sockets, JDBC,
etc.
Thank you very much for your assistance.