Asynchronous Named Pipe IPC and Messaging

December 4, 2013 at 9:10 AMAmer Gerzic

Over the past couple of weeks, I have been developing real time TCP/IP communication server. The server was implemented as .Net windows service. During the development phase, it was clear that the server will require configuration application, which I decided to implement as separate application. I wanted to avoid “desktop interaction” of my service. In addition, I wanted to experiment with IPC offered by .Net framework. I have considered following IPC mechanisms:

  1. Windows Communication Foundation;
  2. .NET Remoting;
  3. Named Pipes;

After some testing and experimentation, I decided to develop my own IPC mechanism. Below, you will find some reasoning behind the effort.

Windows Communication Foundation

WCF is a nice, flexible platform that contains very sophisticated IPC mechanism. WCF uses Client/Server concept to connect processes and exchange data. Communication channels are either pipes or sockets (http) and connection can be achieved with very little code.

However, during my testing process, I realized that WCF requires various underlying services to be installed (understandably) and running. This could cause issues if servers running my application did not have those services running. In addition, quick Google search discovered various annoyances like “endpoint could not be found” exception or Windows 7 security issues. I decided to investigate further.

.Net Remoting

Like WCF, .Net Remoting is a nice platform and it has been around for a while. However, .NET Remoting platform has been “replaced” by WCF. In addition previous experiences with .Net Remoting left me with “configuration heavy” feeling. It seemed to be overkill for what I wanted to accomplish. Again, I decided to investigate further (OK maybe deep down I really wanted to develop IPC on my own).

Named Pipes

Named pipes have been integral part of Windows OS since inception (well maybe not, but there were around relatively long time). Named pipes are similar to other communication channels like sockets or RS232, with respect to data transmission. Having a lot of experience with such communication channels, I decided to use named pipes. Quickly, I opened MSDN to look for .NET named pipes reference. After quick search, I found what I was looking for (NamedPipeServerStream and NamedPipeClientStream). I decided to write a test applications: Asynchronous pipe server and corresponding client.

However, asynchronous pipes are not as “asynchronous” as I hoped it to be. For example, in sockets, you can wait for connection, connect, send, receive, etc. in pure asynchronous way; Named pipes, well, not so much. The solution was clear: Multithreaded Client/Server.

During development process, I encountered couple of annoyances worth mentioning (as always):

1. Named Pipe Access/Security:

The intended use of my library was IPC between windows forms application and windows service. Knowing that windows services run in different session, it was obvious that a pipe created in a service may not be accessible by a desktop application. At first, I thought that I must enable windows service to interact with the Desktop (it’s a check box that must be checked manually in service configuration console). I did not like such solution because it requires manual user interaction/configuration after the installation. However, further investigation revealed that windows pipes provide access rights option via .Net code. I enabled access to “Everyone” and pipe server was visible to regular windows forms application. Access modification is not necessary if the client and the server are running under same privileges in the same session (for example two windows forms application executed by the same user on the same machine).

NOTE: Named pipe library is designed to run on single machine. However, minor modification can be made to establish connection over the network. 

2. Exception Managing:

Due to the fact that both pipe server and pipe client are running in separate threads, it is always challenging to properly handle exceptions. Besides the fact that the main thread must be notified of all errors/issues, it is also important to distinguish between exceptions that are handled by pipe class vs. exceptions that need to be passed further down the stream. I decided to use thread safe callback to notify the main thread of any exceptions that are not handled by pipe threads.

3. Thread Safety:

Thread safety discussion is described below.

Message Parsing

The decision to implement my own pipe client/server implied automatic decision to implement communication protocol. .NET serialization, combined with “dynamic” keyword and the ability of windows pipes to communicate in message mode provided me with all necessary ingredients to leverage C# compiler to perform all parsing for me. In other words:

1. Object is serialized;

2. Object is transmitted;

3. Object is received;

4. Object is deserialized;

5. “dynamic” keyword is used to automatically determine type and call proper method;

I used binary serialization, but any other serialization is possible (XML, JSON, etc.), even with encryption. Actually, simple modification can be made and similar messaging can be utilized with sockets, RS232, etc.

The obvious disadvantage of .NET binary serialization communication is the limitation to .NET environment. However, the intention is to demonstrate named pipes. Protocol can be implemented according to system requirements.

Thread Safety

During implementation phase, I decided to use SynchronsationContext for thread synchronization. At first, it was an obvious choice (every windows forms application has SynchronizationContext exposed through message pump). SynchronizationContext makes it very simple to “post” or “send” callbacks to be executed on specific (in many cases single) thread. This is especially important if we want to update GUI from separate thread.

However, I admit, I did not think the design (completely) through and later realized/remembered that windows services, much like console applications, do not provide a SynchronizationContext. The first thought was to abandon the context and implement different form of thread synchronization. However, I always wanted to know what happens under the hood of .NET SynchronizationContext, and what would it take to implement one.

Quick investigation showed that SynchronizationContext is nothing more than a glorified callback scheduler that places tasks in form of delegates onto a thread queue for execution. I know how to handle queues, threads, and delegates, so I implemented my own STASynchronzationContext fully compatible with .NET synchronization context.

The source code can be found in NPIPC class library.

Source Code

Source code includes following projects:

  1. NPIPC – Class library containing all named pipe code with threading tools.
  2. NPIPCMessaging – Class library containing messages exchanged between pipe client and pipe server.
  3. NPIPCTestClient – Windows Forms application demonstrating pipe client.
  4. NPIPCTestServer – Windows Forms application demonstrating pipe server.
  5. NPIPCTestService – Windows Service demonstrating pipe server.
  6. NPIPCTestServiceGUI – Windows Forms application demonstrating pipe client connecting to windows service server. This project is essentially identical to NPIPCTestClient project.

DISCLAIMER: While the code is working and it has been tested for demonstration purposes, it has NOT been tested in production. Please consider yourself warned that you should test before using as production code.

Download Source Code (73.97 kb)

Posted in: .NET | C#

Tags: , ,

Comments (1) -

Franjo Misetic
Franjo Misetic says:


I know that this is an old post, but I was wondering have you tried this code in a production environment?
Have you made any changes that would perhaps improve it?

I would really like to read up some more on named pipes, but everything that I can find googling are very simple examples, "hello world" like.

Thank you in advance.

Amer Gerzic
Amer Gerzic says:

Hello Franjo,

At this point this is as far as I went, so no new developments have been made with respect to this experiment. However, the example contains all the basics that can be expanded and utilized in production code. One issue that may be present is encryption, which may or may not play a role in your project.


Add comment