Implementing parallelism easily into a heavily multithreaded , virtually concurrent applications

[Raaz]

I am new to the world of multicore programming.I am right now working in a network monitoring software and one of the modules which is too heavily multithreaded is degrading in its performance once run in a multicore machine.All the objects in the application which will occupy a shared memory are made to be in a thread safe class. I would like to ask for expert suggestions for resolving this issue and to step into this new arena. This might also help others who would like to implement parallelism using TBB or OpenMP into their non parallel programs and there by improve the performance of their applications.

Thanks in advance to anyone who will lead this topic.
Raaz

[Gastón C. Hillar]

Hi Raaz,

You should provide more information about your application in order to allow others to help you.
Are you using C++? Which programming language are you using? Are you using OpenMP; TBB; .Net Parallel Extensions or manual threading? What is exactly the application doing?
You can benchmark your application using Intel Concurrency Checker and Process Explorer to find out the cores usage and collect parallelism and concurrency data from the appilcation. I've recently uploaded 2 videos as part of the exercises in my book "C# 2008 and 2005 Threaded Programming: Beginner's Guide", by Packt Publishing - www.packtpub.com.
You can watch these videos, I think they will help you:

"Using Intel Concurrency Checker 2.1 to benchmark threading and multicore metrics": http://www.youtube.com/watch?v=rhVbi8G0XeQ
"Running a parallelized C# threaded application using different number of cores in Windows Vista": http://www.youtube.com/watch?v=-Jfg3IHBNS0
"Running multiple processes to understand multicore CPUs power": http://www.youtube.com/watch?v=-bABM4fA_HA

If you work with C#, you can download code from my book from the following Web Page: http://www.packtpub.com/beginners-guide-for-C-sharp-2008-and-2005-t...

The code for Chapter 9: "Working with Parallelized Input/Output and Data Access" will help you, because your application uses I/O (network monitoring). This is the Chapter table of contents, you'll find the code and the chapter's contents useful:

Queuing threads with I/O operations
Time for action – Creating a class to run an algorithm in an
independent thread
Time for action – Putting the logic into methods to simplify multithreading
Avoiding Input/Output bottlenecks
Using concurrent streams
Controlling exceptions in threads
Time for action – Creating the methods for queuing requests
Using a pool of threads with the ThreadPool class
Managing the thread queue in the pool
Time for action – Running concurrent encryptions on demand using a
pool of threads
Converting single-threaded tasks to a multithreaded pool
Encapsulating scalability
Thread affinity in a pool of threads
Parallelizing database access
Summary


Cheers,

Gastón Hillar

[Raaz]

Hi Gaston,
I am using VC++(Visual studio 6).Right now I am using manual threading for parallelism(which actually was not intended for multiple core system). I have a queue which is being used as the common resources for a number of threads. This queue which is called as System cache need to be updated by all threads and the data in this queue is to be provided to the clients who request for the device status(device: an actual power supply,a node , amplifier etc),alarming of the device once the device gets faulted etc via the server.The system cache class is a thread safe class( implemented the synchronisation within the class). I am not facing any problem while running in a single core as there is no actual concurrency(i think CPU time slizing is used to give a feel for the concurrency... correct me if i am wrong) but while running the application in a dual core or quad core machine the application degrades in its performance and at times giving runtime errors. I need to know if this could be a problem with the threads while accessing the shared resource from different cores. Do I need to implement parallelism to avoid the issues or is there any way i could just fix this issue from the present code level itself? Is it anything related to STA and MTA(i am using STA model for COM)? Can you provide me with information on using the Intel Concurreny Checker? I have used vTune thread profiler and performance analyzer but could not find the real position related to the issue.

Thanks in advance,
Raaz

[Gastón C. Hillar]

Hi Raaz,

I see the problem. Manual threading combined with core affinity works great with multicore CPUs. However, the problem in your application seems to be the synchronization overheard. You can use Intel Concurrency Checker to test the level of parallelism your application is achieving (I've recently uploaded a video about it, as I mentioned in my previous response).
You can also use Process Explorer, to test the context switches (they'll give you an idea of locking problems).
When you use just one core, without Hyper-Threading, the CPU generates context switches when using more than 1 thread.
Taking into account the application you are using, and with the information you are providing me, I'd use .Net Parallel Extensions and I'd use the ConcurrentQueue to implement the System Cache. Why? Because the ConcurrentQueue is hyper-optimized for concurrency. I think the problem is that the application is locking, locking and locking.
Implementing a thread safe system cache with multicore high performance is as difficult as creating a new operating system kernel (something like that). This way, you can focus in other kinds of optimizations, like scalability, instead of running your application slower than when ran in single core CPUs.

Cheers,

Gastón