Why Multithreading?     {#multithreading}
===================

In the past, multithreading has been considered a powerful tool that
is hard to handle (some call it the work of the devil). While there may
be some truth to this, newer tools have made the job of the software
developer much easier when creating parallel implementations of
algorithms. At the same time, the necessity to use multiple threads to
create performant applications has become more and more
clear. Technologies like Hyperthreading and multiple core processors can
only be used if processors have to schedule processing power to
multiple, concurrently running processes or threads.

Event driven programs especially bear the issue of latency, which is
more important for the user's impression of application performance than
other factors. But the responsiveness of the user interface relies
mainly on the ability of the application to process events, an ability
that is much limited if the application is executing processing
power expensive, lengthy operations. This leads, for example, to delayed
or sluggish processing of necessary paint events. Even if this does not influence the total time necessary to perform the operation the
user requested, it is annoying and not state of the art.

There are different approaches to solve this issue. The crudest one
may be to process single or multiple events while performing a lengthy
operation, which may or may not work sufficiently, but is at least sure
to ruin all efforts of Separation of Concerns. Concerns can simply not
be separated if the developer has to intermingle instructions with event
handling where the kind of events that are
processed are not known in advance.

Another approach is to use event-controlled asynchronous
operations. This is sufficient in most cases, but still causes a number
of issues. Any operation that carries the possibility of taking a long
time or blocking may still stop event processing for a while. Such risks
are hard to assess, and especially hard to test in laboratory
environments where networks are fast and reliable and the system I/O
load is generally low. Network operations may fail. Hard disks may be
suspended. The I/O subsystem may be so busy that transferring 2 kByte may
take a couple of seconds.

Processing events in objects that are executed in other threads is
another approach. There are other issues that come with parallel
programming, but it does ensure the main event loop returns as soon as
possible. Usually this approach is combined with a state pattern to
synchronize the GUI with the threaded event processing.

Which one of these approaches is suitable for a specific case has to
be assessed by the application developers. There is no silver
bullet. All have specific strengths, weaknesses and issues. The
ThreadWeaver library provides the means to implement multithreaded job
oriented solutions.

To create performant applications, the application designers have to
leverage the functionality provided by the hardware platform as much as
possible. While code optimizations only lead to slight improvement,
application performance is usually determined by network and I/O
throughput. The CPU time needed is usually negligible. At the same time,
the different hardware subsystems usually are independent in modern
architectures. Network, I/O and memory interfaces can transfer data all
at the same time, and the CPU is able to process instructions while all
these subsystems are busy. The modern computer is not a traditional
uniprocessor (think of GPUs, too). This makes it necessary to use all
these parallel subsystems at the same time as much as possible to
actually use the possibilities modern hardware provides, which is very
hard to achieve in a single thread.

Another very important issue is application processing
flow. Especially GUI applications do not follow the traditional
imperative programming pattern. Execution flow is more network-like,
with chunks of code that depend on others to finish processing before
they can touch their data. Tools to represent those
networks to set up your applications order of execution are rare, and
usually leave it to the developers to code the execution order of the
instructions. This solutions are usually not flexible and do not adapt
to the actual usage of the CPU nodes and computer
subsystems. ThreadWeaver provides means to represent code execution
dependencies and relies on the operating systems scheduler to actually
distribute the work load. The result is an implementation that is very
close to the original application semantics, and usually improved
performance and scalability in different real-life scenarios.

The more tasks are handled in parallel, the more memory is
necessary. There is a permanent CPU - memory tradeoff which limits the
number of parallel operations to the extent where memory that needs to
be swapped in and out slows down the operations. Therefore memory usage
needs to be equalized to allow the processors to operate without being
slowed down. This means parallel operations need to be scheduled to a
limit to balance CPU and memory usage. ThreadWeaver provides the means
to do that.

In general, ThreadWeaver tries to make the task of creating
multithreaded, performant applications as simple as
possible. Programmers should be relieved of synchronization, execution
dependency and load balancing issues as much as possible. The API
tends to be clean, extensible and easy to understand.