![Screenshot - designview.jpg]()
Introduction
In this article, I hope to show a very useful architecture for building a
network server over TCP/IP communication. I know there are a lot of TCP and UDP
servers you can buy, and there are some more you can download for free, but the
problem is that those you can buy usually gives you a "black box". They also
cost money (the good ones cost a lot of money). Another problem is that they
were designed to work on heavy and resourceful machines. The others you can
download seem to go straight into a certain architecture which is usually
incorrect, or at the least, they use an architecture which I believe is not the
best we can use to milk a machine. One of my supervisors once told me that the
enemy of the "good" is the "excellent". Well, obviously, he was correct! But, I
still want the excellent, don't you feel the same?
Well, enough chit-chat, let's examine the problem
Sockets can use three modes for communicating:
- Synchronous communication - using the block mode which halts execution until
something happens in the socket.
- Synchronous communication - without using the block mode - but, this means,
we will get errors which we will have to catch.
- Asynchronous communication - which magically opens a thread per socket
activity, thus not halting execution.
And, of course, it is possible to mix synchronous calls with asynchronous
calls if you seriously feel you don't have enough problems to sustain a
tremendous amount of humor in your life.
The first examination is why and when would we want to use
asynchronous communication
Since async calls open a thread for every socket operation, it is a great
solution for client programs. It will manage the communication without halting
other procedures, it is tested, and it is absolutely not suitable for servers.
Why? The server using async calls will open a thread for every socket method,
which will mean our server is going to end up having way too many threads than
needed, and further, adding more threads at some point will make the entire list
of other threads to run slower and slower until the hardware will require an
upgrade. Now, that will happen no matter what logic we will implement; but, the
question is when is it going to happen? The more threads we open, the sooner we
need a hardware upgrade, so for now, we can determine that using async calls is
out of the question if we want a low resource waste with maximal optimization in
our server.
So, our goal is to use the smallest number of threads as long as our Latency
definition (a.k.a. Lag) is meeting our design requirements. Yes, I know some of
you would say something like: "what are you talking about, most servers work
with async communication!". They are correct - so if things are as I explained
in the previous paragraph, why do so many servers use the async architecture?
The answer is in an assumption that will not hold water, and in a non-elegant
workaround that brutally forces things to work, as I will explain. Behind this
architecture lays an assumption that clients keep communicating to the server,
making the threads behind close and open all the time since an operation like
receiving data or sending data is a short operation, and since it's short, it
takes resources only for few nano-secs and then they are released, and so many
threads can open and close without actually wasting resources in a constant way,
which is actually very true (so far).
However, if for example I connect to such a server from my client and I
decide not to send data to that server, it would mean that that server has
opened a BeginRecieve method that is waiting for my client to send
something, and as long as I don't send anything, the thread that waits for data
remains open, and does not close.. If we have 2000 clients doing that, the
server will be in trouble, and the architect who designed it will be either
unemployed, or possibly promoted to a division manager - that depends on where
you work. To workaround this problem, the architecture requires a non-idle
policy, so threads can close and malicious clients can not use that exploit. One
of those policies can be a timer that closes idle clients, another can be based
on a ping mechanism, and few would go for even smarter solutions that keep track
of resources and disconnect idle clients only when resources are getting low.
But, all those are workarounds which point out that they deal
with the symptoms and not the problem that causes them! For the async server
architecture, which as I explained, this is not suitable.
Another approach would be to use sync calls in block mode to our sockets -
this means that execution will halt, so to workaround this, we can open a thread
per client. But again, we shall have many threads running in our system, and
that is something we already killed as a server solution.
The last option remaining is to use the sync calls without block mode. The
problem is that 2000 clients connected to our machine will need to be scanned
repeatedly, and that would take too long to scan.
The remedy
So, the remedy is to use a few threads to scan our clients - for example,
every 100 clients will be scanned by a thread. However, we don't have the
slightest idea regarding how long would it take to scan 100 clients in different
machines. Another problem is that constraining to 100 is an arbitrary
definition, which means we will not be dynamic.
What I did at that point is create a class which I called
ServerNode. The ServerNode holds clients, and return
latency information of the scan (how long did it take to scan all the clients in
the node). In the server base, I created a property which I named
RequiredLatencyPerClient. From that point, things became much
clearer.
When the server clients list receives a new client, the server will scan all
nodes and decide which node to add the new client to. If no node latency meets
the RequiredLatencyPerClient, the server will create a new node and
will add the new client to that node.
Foreman Design Pattern - Is that a new design pattern ? (If not, I
got a new and very generic idea instead - I will call it the
"Wheel"!)
Well, so far so good, but while writing the code, I couldn't help thinking
that I somehow was doing something which is very generic in many situations. The
idea that we have a client that needs to be organized in nodes in order to
maintain a certain latency is something that can be useful in many situations.
So, I decided it would be even better to take that work pattern out of the
server and implement it independently so it can be used dynamically and in other
situations as well.
I seeked the books and the internet to see if there is a known design pattern
that actually does that - and found that there isn't, so if no one objects, I am
taking the right to call it the Foreman Design Pattern.
But... there is something that bothers me. What if some clients need a
different latency than others ? Hmm.. and what if I want to be able to change a
client latency dynamically?
So, scratching my head, I finally realized that a simple solution was staring
at me all the time! The Foreman will not have a
RequiredLatencyPerClient. Instead, each client, which from now on
we shall call a worker (since we are tinkering with a design pattern and not
just a communication server), will have a latency information, and the Foreman
will put the worker in a suitable node according to its latency
requirements.
Based on the Foreman Design Pattern, I can build a very cool communication
server.
So, after building the Foreman, I began writing a simple code to wrap the
socket communication options. I opened a project and called it
NetworkSocketManager - it will be used in two other projects, a server
and a client. The NetworkSocketManager will let us choose between a
socketing work methodology and it will wrap all the synchronous and asynchronous
communication options (including the block mode).
Having done with that part, I coded a simple client that contains the
NetworkSocketManager. The project is called NetworkClient.
Now, the client itself is also very nice and friendly since it already
implements a basic application level transmission control. Any data that is
going to be sent is being wrapped in an envelope which contains: |
prefix length = 1 digit | Prefix = length of data | data |. So, when
receiving, the event for receiving a message is going to happen only when the
full data accumulated through the socket. The situation for getting partial data
can happen if I don't implement the application level transmission control. If,
for example, the server out buffer is set to 1024 and the receive buffer is set
to 256 - it would cause partial data coming or full data coming in 4 application
level packets. But, as I said, this problem is already solved.
If you wish to have encryption on your data, make sure to encrypt before you
pass on the data to the send, and if you wish to manipulate your message before
sending it (for example, if you need integration with another server), you will
have to inherit the communication class and override the communication functions
to your needs.
The next task was to build a server that uses the already mentioned
ForemanDP. So, I opened another project and called it
NetworkServer. In it, I derived the ForemapDP worker class
to CommunicationWorker class and the server class derives from
Foreman BasicServer. And voila! If everything so far is clear, you
will be able to understand my code and why it is so powerful.
The fact that this is a powerful code implementation lies in the fact that
all the programmer needs to do is derive a worker and a server to have an out of
the box, running, and opened source client/server architecture which uses the
correct resource methodology.
Comparing to Classic IOCP
![]()
As we can see, the model based on the foreman will "milk" the machine better
since the definition of the required worker (client) latency has a meaning
regarding who needs more focus from the machine. To be even more exact, the
clients that require latency < 100 are treated in a different thread. This is
what makes the difference from the classic IOCP, where all clients are treated
in the same threadpool - there is a waste there since a client that requires a
latency of 1000 (1 sec) will cause the other queued clients to hold until it has
finished processing.
However, ForemanDP knows how to deal with such a case: in the
case a client latency is not met, a new thread (client's node) will be opened,
and that client will be moved to it to make sure it can receive the latency it
requires.
Using the Code
OK, so how do we get started?
Step 1
The first thing is to either add just references or the projects into your
solution. For those who need to have servers, you will need the
NetworkSocketManager, ForemanDP, and
NetworkServer. For those who need to have clients, you will need
the NetworkSocketManager and NetworkClient. For those
who need both, just add it all...
Step 2
Server developers: In your class, write your code to contain the
NetworkServerBase (or your class which derives it).
Collapse
Copy Codeprivate NetworkServerBase MyServer = null;
The next thing is to instantiate the server. You would want to put it in the
Form_Load or a constructor of your class:
Collapse Copy Code MyServer = new NetworkServerBase(new TimeSpan(1000));
MyServer.OnServerNotify +=
new ForemanDP.BasicServer.DelegateServerNotification(MyServer_OnServerNotify);
MyServer.AddListener(
new NetworkServerClientListener(new TimeSpan(2000),
new System.Net.IPEndPoint("replace this with the ip you want", 5500), 100));
The last line is interesting because what it actually does is add a listener
with a required latency of 2 seconds to the server.
The NetworkServerClientListener is derived from
NetworkClientWorker which derives from ForemanDP
worker. The line of code sets port 5500 to be a listening port, and puts a 100
backlog space for queuing clients which need to connect. You can add more
listeners - as many as you want! Run a loop on that line or just copy paste it,
whatever you need.
From that point forward, every communication event to that port is piped to
MyServer_OnServerNotify:
Collapse Copy Code void MyServer_OnServerNotify(ForemanDP.BasicWorker worker, object data)
{
switch (((NetworkServerClientWorker.NetworkClientNotification)data).EventType)
{
case NetworkClient.NetworkClientBase.ClientEventTypeEnum.Accepted:
connectedUsers++;
break;
case NetworkClientBase.ClientEventTypeEnum.Disconnected:
connectedUsers--;
break;
}
}
You can catch messages and anything else you need from that event.
Clients: In your class, write your code to contain
NetworkClient:
Collapse Copy Code private NetworkClientBase clientCommunication = null;
---- your constuctor or Form_Load -----
clientCommunication = new NetworkClientBase(
new CommunicationSettings(
CommunicationSettings.SocketOperationFlow.Asynchronic,
CommunicationSettings.SocketOperationFlow.Asynchronic,
CommunicationSettings.SocketOperationFlow.Asynchronic,
CommunicationSettings.SocketOperationFlow.Asynchronic,
CommunicationSettings.SocketOperationFlow.Asynchronic, false, 64));
clientCommunication.OnClientEvent +=
new NetworkClientBase.DelegateClientEventMethod(clientCommunication_OnClientEvent);
:
:
:
---------------------------------------
private void clientCommunication_OnClientEvent(
NetworkClientBase.ClientEventTypeEnum EventType, object EventData)
{
switch (EventType)
{
case NetworkClientBase.ClientEventTypeEnum.None:
break;
case NetworkClientBase.ClientEventTypeEnum.Accepted:
break;
case NetworkClientBase.ClientEventTypeEnum.Connected:
break;
case NetworkClientBase.ClientEventTypeEnum.RawDataRecieved:
break;
case NetworkClientBase.ClientEventTypeEnum.MessageRecieved:
break;
:
:
:
and so on to catch what ever you need or want.
Latest Version Additional Info
As an example, I added a simple chat implementation. In order to run it:
- From Visual Studio, click the green triangle button (F5) - this will run the
chat server.
- While the chat server is running, go back to Visual Studio and right click
on the chat client project - a popup menu will appear. From the popup-menu,
click Debug -> Start New Instance.
You can repeat step 2 to create and run as many instances as you want.
Note: The chat-client/server is just an example of how to use the
entire architecture. It is not the focus of this article, and it is not intended
to be bug-free.
Benchmark Program Addition
I have repaired much of the TCPSocket layer which was causing
problems in certain situations. If more bugs arise, please let me know so I can
fix them as well (in my machine, it works well).
The benchmark opens a server and multiple async clients to communicate with
the server. The server will echo each incoming packet back to the client who
sent it (much like ping!).
Since it was necessary to identify each async client, I created another
client which contains a NetworkClientBase and adds identity to it.
Normally, you will not need such a client unless you want to work with multiple
clients without a server or a main node to manage them inside it.
Bug fix (28.10.2008)
Thanks to BeAsT[Ru] from this forum, a bug was found and repaired in a queue
which is used inside the accepter to send data. An asymmetric handling was found
in the queue and a critical section was implemented in the code. After several
tests, I can conclude now that this bug is no more.
Bug fix (20.12.2008)
Thanks NapCrisis who fixed a minor bug in the chat example.
Finally
Most programmers believe that they have the best code that only God could
match in perfectness. And, I am no different! But, in order to really learn
something, we must put our ego very far away. So, please let me know what you
think, and feel free to criticize.
The connecter area in the server is not in use - I didn't finish it - but,
unless you want a server that can initiate communication to other servers like a
proxy, for example, it will not bother you. I promise to finish it if you
CodeProject folks want it.
Last thing: Some might ask why not use WCF instead of all this? WCF is a very
advanced request-response architecture, and as progressed as it is, it will not
give you "the always online in real time connection". Check this out: MSDN, if you don't believe me.
http://www.codeproject.com/KB/IP/Generic_TCP_IP_server.aspx