Parallel Object-Oriented Programming with QPC++

Dietrich Boles

Fachbereich Informatik, Carl-von-Ossietzky Universität Oldenburg, D-26111 Oldenburg, Germany

e-mail: boles@informatik.uni-oldenburg.de

Abstract: QPC++ is an extension of the object-oriented programming language C++. It integrates mechanisms for specifying parallelism, communication, and synchronization into the base language. Processes are not added as an orthogonal concept to the language. Instead, the concepts of object-oriented and parallel programming are merged in such a way that processes are created as instances of specialized classes, called process classes. Processes can be regarded as active objects which can communicate via their member functions. Thus, QPC++ is a very useful and suitable tool for the modeling and implementation of applications which are characterized by activities taking place in parallel, e.g., simulations or interactive graphical user interfaces.

This article demonstrates the integration of mechanisms for expressing parallelism into existing object-oriented programming languages. It turns out that merging of concepts of object-oriented and parallel programming can be done in a very clear and natural way. QPC++ will serve as an example. Finally, other extensions of C++ will be presented and compared with QPC++.

Key Words: object-oriented programming, parallel programming, parallel object-oriented programming, active objects, rendezvous, multicasting, C++, QPC++

1. Introduction and Motivation

The use of object-oriented programming languages has strongly been increasing in the last few years. Especially the programming language C [1] has been replaced more and more by its object-oriented extension C++ [2]. First of all, the reason for that development is the raised reusability and easy extendibility of object-oriented software. Additionally, the modeling of large systems is supported by an object-oriented design. Problems of the "real world" can be modeled more naturally by objects than e.g. in a functional approach.

However, in a few areas of application, like simulations and the development of interactive graphical user interfaces, applications are often characterized by activities taking place in parallel. Processes in form of active objects are more suited to model and implement such applications than passive objects of sequential object-oriented programming languages. In contrast to a passive object which is always waiting quietly until it receives a message (in C++ in form of a call of a member function), an active object has an ongoing activity on its own. After being created, it executes a special function. The execution takes place in parallel with the execution of other functions by other active objects in the system. Only at certain explicitly indicated points an active object can communicate with other objects.

QPC++ supports such applications by offering the definition of processes in form of active objects besides the definition of passive C++-like objects. Processes are instances of specialized classes, called process classes. By the fact that processes are not added to the base language in an orthogonal way, the concepts of object-oriented programming, like inheritance or polymorphism, are valid for processes, too. The main goal of the definition of QPC++ was the integration of known, clear and easy to handle mechanisms for expressing parallelism into C++. The modeling and implementation of applications in the above sense shall be best supported by QPC++. It was not intended to support the definition of very fine-graduated processes. At the moment, the language is only implemented on a uni-processor. Processes are working quasi-parallel.

QPC++ is upward-compatible to C++. It is possible to use existing classes without any restrictions. Moreover, it is possible to define process classes by deriving them from usual C++-classes. Only a few new key words and syntactical constructs are added to the base language. Thus, it will be easy for a C++-programmer to switch from C++ to QPC++.

2. Objects and Processes

In the object-oriented programming-style a system can be regarded as a collection of objects. An object is a unit consisting of data and functions, called methods, acting on these data. The data are stored in so called instance variables. Usually, it is only allowed for the object itself to access its variables. Objects have no access to the variables of other objects, except they are explicitly allowed to do so. In C++, methods are called member functions. They form the interface of the objects.

Objects of the same type can be grouped in classes or that is to say, classes can be regarded as blueprints for the definition of objects. During execution of a program, objects can be created as instances of classes. Objects of the same class execute the same code for their methods. Instance variables of objects of the same class have the same names and types but each object has its own set of variables. Figure 1 illustrates the form of objects.

Figure 1. Objects and Processes

Objects of sequential object-oriented programming languages are characterized by their passivity. In C++, a program usually starts with the execution of a function, called main. During execution of the program, objects are created and initialized with the help of special member functions, the constructors. After being initialized, an object is waiting for a call of one of its member functions. Being called, it becomes active and executes the function. By doing that, it possibly calls member functions of other objects. After having finished the execution of the member function, the object becomes passive again.

An object which is not waiting quietly until one of its member functions is called but has an activity on its own is called active object. Only at certain explicitly indicated points its activities can be interrupted in order to execute a member function. Active objects represent autonomous processes. QPC++ offers the possibility to define so called process classes. They are similar to classes in C++, but additionally, they can contain a special member function, called body, in which such activities can be described. An instance of a process class is an active object. Syntactically, it can be created like an object in C++. After its initialization by executing a constructor, the body is explicitly activated. It is executed like a usual function. Its execution takes place in parallel with the execution of the statements following the creation (definition) of the object. Figure 1 illustrates the form of a process in QPC++.

A QPC++-program is a program consisting of active objects each executing its body. Besides the definition of process classes and processes the definition of object classes and passive objects is still allowed.

Figure 2 demonstrates the definition and creation of processes in QPC++. The program starts with the execution of the function main (called main process in the following). The main process creates two processes w1 and *w2. They are instances of process class Writer. After the initialization of w1 by executing the constructor of process class Writer, the body of the process (the member function $Writer()) is started automatically. Meanwhile, the main process creates process *w2 in a similar way. After the execution of the second statement of function main, there exist three active processes: the main process executing the for-loop and the two writer processes each executing the while-loop of its body. By calling the delete-operator or by leaving the scope of variable w1 respectively, the main process terminates the two writer processes. In both cases the main process has to wait, until the writer process has finished the execution of its body. Having left the scope of function main, the main process terminates as well. The termination of all processes causes the termination of the whole program.

#include <stream.h>

process class Writer { // process class

private:

int n, loops;

public:

Writer(int lo, int i){ // constructor

// initialization

loops = lo;

n = i;

}

$Writer() { // body

// after the creation and initializa-

// tion of a writer-process, this mem-

// ber function will be called auto-

// matically

while (loops--)

cout << n++ << endl;

// now the process is ready to termina-

// te

}

};

// the program starts with the main-func-

// tion

main() {

Writer w1(100, 1);

// process w1 is created and initia-

// lized; its body is called implizitly

Writer *w2 = new Writer(200, -24);

// process *w2 is created and initia-

// lized; its body is called implizitly

// now there exist three active proces-

// ses: the main-process executing this

// function and the writer-processes w1

// and *w2 each executing its body

for (int loops=0; loops<10; loops++)

cout << loops*loops << endl;

delete w2;

// the main-process wants to terminate

// process *w2

// the main-process will terminate pro-

// cess w1 because of the end of its

// scope

}

// the whole program terminates because all

// processes are terminated

Figure 2. Definition and Creation of Processes

3. Interprocess Communication

Usually, processes do not deal with a special problem in isolation from other processes. Processes have to interchange data in order to solve their special task. In general, the exchange of data between two or more processes can be achieved by offering the definition of shared variables and/or mechanisms for message passing.

3.1 Shared Variables

QPC++-processes can use shared variables in order to interact with other processes. Shared variables of two or more QPC++-processes are all those variables which belong to the scope of each of those processes. Processes can read values out of a shared variable and write data into it. The access to a shared variable by two or more processes has to be synchronized in order to avoid interference. In QPC++, synchronization can be achieved by means of semaphores (see figure 4).

3.2 Message Passing

An additional possibility for a QPC++-process to interact with another process deals with the call of one of its member functions. However, unlike the call of a member function of a C++-object, a call of a member function of a process does not lead to the immediate execution of the function. Usually, the called process is executing its body. Only at certain explicitly indicated points the body may be interrupted in order to execute a member function. Thus, the calling process (client) has to wait, until the called process (server) reaches such a point. QPC++ adds a new syntactical construct to C++ in order to be able to mark such a point of synchronization. It is called accept statement and includes two sorts of information, firstly the name of the member function which may be called and secondly the names of the processes which may call that member function.

Supposing a client calls a member function of a process (it makes a request) and the server accepts that call (it accepts the request), the two processes enter into a so called rendezvous: The client and the server synchronize, the actual parameters are passed to the member function, the function is executed in the thread of the server, and perhaps a result is passed back to the client by means of a return statement. After the execution of the member function the rendezvous is finished. Each of the two processes can execute its next statement independently. A rendezvous is a special form of interprocess communication via message passing.

Usually, if a client makes a request, it delays until the server accepts the request, executes the corresponding member function and finishes the rendezvous. Such a request is called a synchronous request.

An accept statement is blocking as well. That means, reaching an accept statement, the server delays until a suitable request is made. After the completion of the execution of the corresponding member function, the server continues with the execution of the statement following the accept statement.

A rendezvous is finished implicitly by reaching the end of a called member function or explicitly by calling a return statement. The semantics of a return statement of QPC++ is the same as in C++. Additionally, QPC++ offers another way to finish a rendezvous by means of a so called reply statement. A reply statement looks like a return statement. However, in contrast to it, it does not finish the execution of the member function. By means of a reply statement a server can pass back a result to the client as soon as possible without finishing the execution of the function. Thus, the client is only delayed as long as necessary.

QPC++-processes are sequential processes(see [4]). Member functions are executed in the thread of the server. Therefore, it is not possible to execute two or more member functions of the same process in parallel. At each time there exists at most one active member function per process. Intra-object concurrency is not supported. However, it is allowed to define accept statements not only in the body but in member functions of process classes as well. In the latter case, the execution of the outer request is interrupted, until the execution of the inner request is completed.

Figure 3 illustrates the interaction of processes by virtue of a rendezvous. After the creation of process b (an instance of process class Buffer) and the activation of its body, b reaches an accept statement expressing that b will accept a call of its member function put being made by any process (indicated by the key word all). Process b delays until a put-request is made. When a put-request occurs, it starts the execution of its member function put after having passed the actual argument to it. By virtue of the reply, b finishes the rendezvous and allows the client to continue immediately. The client does not have to wait, until the actual parameter is assigned to the instance variable buffer of b. Afterwards, the buffer process waits for a get-request. A get-request is performed by passing the actual value of the instance variable buffer back to the client. After the creation of process b, the main process makes a put-request to b. A rendezvous takes place. The member function put of process b is executed. Afterwards, the main process performs some actions. Finally, by means of a get-request, it asks b to pass the buffered value back and assigns that value to the local variable i.

process class Buffer {

private:

int buffer;

public:

void put(int value) {

reply; // reply-statement

// the reply finishes the rendez-

// vous; the client can continue

// immediately; the buffer-process

// executes the assignment before

// it returns to the body

buffer = value;

}

int get() {

return buffer;

}

$Buffer() { // body

all<.put; // accept-statement

// waiting for a put-request; when a

// process makes a put-request, the

// member function put is executed

all<.get; // accept-statement

// waiting for a get-request; when a

// process makes a get-request, the

// member function get is executed

}

};

main() {

Buffer b;

// a buffer-process b is created and

// initialized; its body is called

// automatically; b is waiting for

// a put-request

b.put(47); // synchronous request

// the main process makes a put-re-

// quest to b; because b is waiting for

// a put-request, a rendezvous can

// take place; the member function put

// of b is executed

// ...

int i = b.get(); // synchronous request

// the main process makes a get-re-

// quest to b; having finished the put-

// request, b is waiting for a get-re-

// quest; thus, a rendezvous can

// take place; the member function get

// of b is executed; it returns a

// value which is stored in i

}

Figure 3. Interprocess Communication

Often, it is desirable for a server to offer the possibility to accept two or more requests of a different type alternatively at a certain time. For that reason a new syntactical construct is added, the select statement. The select statement consists of one, two, or more accept statements and an optional otherwise statement. It is possible to attach an additional condition to the accept statements within a select statement (guarded accept statement). When a process reaches a select statement during run-time, the alternative accept statements are checked in a non-deterministic order whether they are holding or not. If an alternative is holding, the corresponding member function is executed and afterwards the select statement is finished. If no alternative is holding, the process is delayed, until a process makes a request that causes an alternative to be holding. In the latter case, the delay of the process can be avoided by means of an otherwise statement. The process executes the otherwise statement if no alternative is holding.

By defining a select statement only consisting of one accept statement and an empty otherwise statement, a non-blocking accept statement can be achieved.

On the other hand, a non-blocking call of a member function of a process, a so called asynchronous request, can be defined, too. Putting the key word asynchronous in front of a request, that request together with the actual parameters is stored in an internal mailbox of the addressed server. The client has not to wait until the server will execute the member function but can continue immediately.

The use of select statements and asynchronous requests is demonstrated in figure 4. A process class Semaphore is defined. Semaphores can be used to synchronize processes. Using semaphores, it is easy to protect critical sections, e.g., a statement accessing a shared variable. A semaphore is created and initialized with the number of processes that are allowed to stay in the critical section simultaneously. The condition "counter>0" in the second alternative of the select statement within the body of process class Semaphore indicates whether the critical section which is guarded by the semaphore is occupied. If the condition is not fulfilled, processes making a wait-request are delayed. At least they have to wait until one process leaves the critical section indicating that by virtue of a signal-request. A signal-request may be made asynchronously by the client, because the execution of the member function signal does not affect its activities.

#include <stream.h>

process class Semaphore {

private:

int counter;

public:

Semaphore(int value) {

(value<0)?(counter=0):(counter=value);

}

void wait() { counter--; }

void signal() { counter++; }

$Semaphore() {

while (1)

// waiting for a signal-request or

// for a wait-request if the condi-

// tion ´counter>0´ holds

select { // select-statement

when (all<.signal);

or when ((counter>0) => all<.wait);

}

}

};

Semaphore sem(1);

// creation of a semaphore-process which

// allows only one process to enter the

// critical section; at one time it is

// allowed for only one process to assign

// values to the global variable cout

// (declared in <stream.h>)

process class Writer {

private:

char *text;

public:

Writer(char *t) { text = t; }

$Writer() {

while (1) {

sem.wait();

// after a wait-request sem can

// only handle a signal-request

// because the condition ´counter

// > 0´ does not hold; so it is not

// possible for other writer-

// processes to pass the above

// statement; at least they are

// blocked, until a signal-request

// has been executed by sem

cout << text << endl;

// cout is a global variable;

// assignments to it must be syn-

// chronized

asynchronous sem.signal();

// a signal-request is put into the

// mailbox of sem; the process can

// continue immediately, it has not

// to wait until the request is

// accepted and executed

} // end while

}

};

main() {

Writer w1("hello america");

Writer w2("hello germany");

// two writer-processes are created;

// each of them executes its body

}

Figure 4. Select-Statement, Asynchronous Requests

Nevertheless, a client can receive a result from an asynchronously called member function adapting the wait-by-necessity principle [5] to QPC++: A client of an asynchronous request has to wait only when it attempts to use the result of the corresponding member function. Variables which are defined to receive the result of an asynchronous request are called future variables [6]. In QPC++ normal variables can be used as future variables.

A process is able to accept all those requests for which there exists a corresponding member function in its protocol. Additionally, a special accept statement is available to any process: all<.terminate. It has explicitly to be used like a normal accept statement and signals the willingness of the process to become terminated. A terminate-request is made implicitly by another process by means of the delete-operator or through leaving the scope of the process. A terminate-request is always synchronous. The destructor of a process class can be regarded as the member function being assigned to the termination accept statement. After a process has accepted a terminate-request and has executed its destructor, it is dead and cannot perform further actions. Note that in figure 4 those facts cause a deadlock of the main process. Leaving the scope of function main, the main process makes an implicit terminate-request to process w2. However, w2 will never accept that request. Thus, the main process is blocked forever.

Figure 5 illustrates the use of future variables and the termination accept statement. The main process creates process *c and makes the asynchronous request f to *c. The variable value1 is used as future variable. While the calculator process executes member function f the main process can perform some other operations in parallel. It has to wait for the result of f only when it evaluates the expression value2*value1. Note that the return statement in member function f does not force the calculator process to wait until the main process will use the result. The return value is automatically assigned to the future variable. The delete-operation of the main processes causes an implicit terminate-request to *c. The main process has to wait until *c has accepted that request and has executed the destructor as usual.

process class Calculator {

public:

Calculator() {...}; // constructor

~Calculator() {...} // destructor

float f(float x) {

float y = ...

return y;

// the return value is assigned to

// the corresponding future variable

// if f is called asynchronously

}

$Calculator() {

while (1)

// at each time a calculator process

// accepts either a f- or a

// terminate-request

select {

when (all<.f);

or when (all<.terminate);

}

}

};

main() {

Calculator *c = new Calculator();

float value1 = asynchronous c->f(2.3);

// value1 is used as future variable of

// the asynchronous request f;

// the main process has not to wait for

// the result of f but can execute the

// following statement immediately

float value2 = sin(30.7) * cos(93.0);

value2 = value2 * value1;

// now, during the evaluation of this

// expression the value of the future

// variable is used; the main process

// has to wait if f has not returned a

// result yet

delete c;

// by calling the delete-operator the

// main process makes an implicit

// terminate request to *c; it has to

// wait until *c has accepted that

// request and has executed its

// destructor

}

Figure 5. Future Variables, Termination

If no body is defined within a process class and no body is inherited (see the following chapter), a process of that class behaves just like a passive object. It is always waiting for any request. Receiving a request, it executes the corresponding member function. Afterwards it waits again. That means that in figure 5 the definition of the body of class Calculator is unnecessary and could have been left.

4. Inheritance and Polymorphism

The advantages of object-oriented programming over other programming-styles are the raised reusability and easy extendibility of existing software. These advantages are facilitated by the concepts of inheritance and polymorphism in conjunction with dynamic binding.

A class possessing some similarities with an already existing class can be derived from the existing one. That means that the new class automatically inherits the specification and implementation of the existing class. Parts of it can be extended or modified. The concept of polymorphism in conjunction with dynamic binding allows the definition of variables referring to objects that have the same interface but different implementations of their methods. The referred object and the executed method will be determined at run-time and not at compile-time.

C++ supports both concepts. It allows the definition of a class being derived from one or even two or more existing classes (multiple inheritance). Polymorphism is facilitated by means of so called virtual member functions.

In QPC++ both concepts are applied to process classes as well. Process classes can be derived from already defined process classes. They can even be derived from object classes. Member functions and variables are inherited just like for object classes. Furthermore, a body is inherited. Member functions of process classes can be defined virtual. Thus, the concept of polymorphism is valid for processes as well. It is even applied to accept-statements with corresponding virtual member functions. A call of a virtual member function of a process is said to be a virtual request. Receiving a virtual request is called virtual acceptance.

Figure 6 illustrates the merging of concepts of object-oriented programming and concepts of parallel programming in QPC++. Process class Calculator is an abstract process class with the pure virtual member function calculate. Process classes Sin and Cos are derived from Calculator. They inherit the body. Each of them defines the virtual member function calculate in a different way. The main process makes two virtual calculate-requests. They cause different results depending on the actual type of the process referred by c.

#include <math.h>

#include <stream.h>

process class Calculator {

// abstract process class

public:

$Calculator() {

all<.calculate;

// virtual acceptance

}

virtual float calculate(float) = 0;

};

process class Sin : public Calculator {

public:

// the body is inherited

float calculate(float x) {

return sin(x);

}

};

process class Cos : public Calculator {

public:

// the body is inherited

float calculate(float x) {

return cos(x);

}

};

main() {

Calculator *c;

c = new Sin();

cout << c->calculate(90.0) << endl;

// virtual request to *c;

// sin(90.0) is calculated

c = new Cos();

cout << c->calculate(90.0) << endl;

// virtual request to *c;

// cos(90.0) is calculated

}

Figure 6. Inheritance and Polymorphism.

5. Multicasting

Sometimes, it is necessary for a process to communicate not only with a single process but with a set of processes at the same time. For example, if a graphical object is erased from a drawing it might be necessary to make a refresh-request to all objects obscured by that object. The mechanism of requesting several processes simultaneously is called multicasting. QPC++ facilitates the multicasting of requests by defining so called process sets. Processes of determined classes may be inserted in a process set. Process sets can be regarded as instances of a predefined structure process set which is handled just like a C++-template. The definition of a process set includes the declaration of those process classes whose instances are allowed to be inserted into the set. Those classes are called CPSs (classes of the process set). Concerning the addressing, a process set is treated just like a normal process. It automatically possesses all those member functions which are defined in the public part of all of its CPSs. A call of a member function of a process set is said to be a multicast request. It causes a simultaneous request to all processes being included in the set. Requests to process sets may be synchronous or asynchronous. If a process makes a synchronous multicast request to a process set, it is delayed until all addressed processes have finished the rendezvous. It is not possible for the client to receive a result.

In line 25 of figure 7, a process set s with CPSs Box and Line is defined. Initially, processes b1 and l1 are included in s. The asynchronous multicast request in line 26 causes an asynchronous refresh-request to process b1 and process l1. In line 29 process b2 is inserted into set s. Thus, the synchronous multicast request in line 30 causes an simultaneous synchronous refresh-request to b1, b2 and l1.

There exists a special form of process sets, called dynamic process sets (dyn process set). Dynamic process sets differ from normal process sets in that way that at each time of program execution all currently existing processes of their CPSs are automatically included in the set. That means, a call of a member function of a dynamic process set causes the multicasting of the corresponding request to all currently existing processes of its CPSs.

In line 35 and 36 of figure 7, a (temporary) dynamic process set with CPSs Box and Line is defined and a multicast request of the member function refresh is initiated. The statement causes a refresh-request to all existing processes of classes Box and Line, that is to b1, b2, b3, l1 and *l2.

If a class is defined by deriving it publicly from an already existing class, all member functions of the public part are inherited. Referred to a process class that means that all requests which are allowed to be made to a process of the base class can also be handled by a process of the derived class. Following the implementation of the concept of polymorphism in C++, QPC++ facilitates the definition of process sets into which not only processes of the explicitly declared CPSs may be inserted but processes of classes being publicly derived from that class as well. Syntactically, that is expressed by suffixing an ampersand to the name of the CPS. Calling a virtual member function of such a process set causes a virtual multicast request, i.e. a virtual request to all processes which are included in the set.

/*01*/ process class Graphic {

/*02*/ public:

/*03*/ $Graphic() {

/*04*/ while (1) all<.refresh;

/*05*/ }

/*06*/ virtual void refresh() = 0;

/*07*/ };

/*08*/

/*09*/ process class Box

/*10*/ : public Graphic {

/*11*/ public:

/*12*/ void refresh() {...}

/*13*/ };

/*14*/

/*15*/ process class Line

/*16*/ : public Graphic {

/*17*/ public:

/*18*/ void refresh() {...}

/*19*/ };

/*20*/

/*21*/ main() {

/*22*/ Box b1, b2, b3;

/*23*/ Line l1, *l2;

/*24*/

/*25*/ process set<Box,Line> s1(b1, l1);

/*26*/ asynchronous s1.refresh();

/*27*/ // asynchronous refresh-request

/*28*/ // to b1 and l1

/*29*/ s1 = s1 + b2;

/*30*/ s1.refresh();

/*31*/ // synchronous refresh-request

/*32*/ // to b1, l1 and b2

/*33*/

/*34*/ l2 = new Line;

/*35*/ dyn process set<Box,Line>.

/*36*/ refresh();

/*37*/ // synchronous refresh-request

/*38*/ // to b1, b2, b3, l1 and *l2

/*39*/

/*40*/ dyn process set<Graphic&> d;

/*41*/ delete l2;

/*42*/ asynchronous d.refresh();

/*43*/ // asynchronous virtual refresh-

/*44*/ // request to b1, b2, b3 and l1

/*45*/ }

Figure 7. Multicasting Mechanisms

In line 42 of figure 7, a virtual multicast request is performed. Process classes Box and Line are derived from process class Graphic, both defining the virtual member function refresh. Because d is a dynamic process set of CPS-type Graphic&, the statement causes an asynchronous virtual refresh-request to all currently existing processes of those process classes which are publicly derived from process class Graphic, that is to b1, b2, b3 and l1.

6. An Example: The Producer-Consumer-Problem in QPC++

The purpose of this chapter is to illustrate the use of QPC++ by a bigger example. Therefore, the well known producer-consumer-problem is implemented in QPC++. The producer-consumer-problem is characterized in the following way: Given a buffer of limited size, some processes produce data and store them in the buffer. Some other processes fetch the data out of the buffer and consume them. Access to the buffer has to be synchronized. Data are only accepted by the buffer if it is not full. Data can only be taken out of the buffer if it is not empty.

const int BUFFER_SIZE = 10;

process class Buffer { // a FIFO-queue

protected:

int buff[BUFFER_SIZE]; // storing the

// data

int n; // number of values in the

// buffer

int in; // in-slot; next free slot for

// storing a value

int out; // out-slot; slot containing

// the oldest datum

public:

Buffer() { in = out = n = 0; }

void put(int num) {

// reply immediately, then put the

// argument into the next free in-slot;

// afterwards, reset the actual in-slot

reply;

buff[in] = num;

in = (in+1) % BUFFER_SIZE;

n++;

}

int get() {

// reply the value of the actual out-

// slot; afterwards, reset the actual

// out-slot

reply (buff[out]);

out = (out+1) % BUFFER_SIZE;

n--;

}

$Buffer() {

while (1)

// if the buffer is not full, accept

// put-requests; if the buffer is not

// empty, accept get-request; if

// there are some values in the buf-

// fer, accept both, put- and get-re-

// quests, in a non-deterministic

// order

select {

when (n<BUFFER_SIZE => all<.put);

or when (n>0 => all<.get);

} // end select

}

};

process class Producer {

protected:

Buffer &buffer;

int value;

virtual void produce_value() {

value = ...;

}

public:

Producer(Buffer &b) : buffer(b) { }

$Producer() {

while (1) {

// produce values and put it into the

// buffer

produce_value();

asynchronous buffer.put(value);

} // end while

}

};

process class Producer2 : public Producer {

protected:

virtual void produce_value() {

value = ...;

// other productions than Producer

}

public:

Producer2(Buffer &b) : Producer(b) { }

// the body is inherited

};

process class Consumer {

protected:

Buffer &buffer;

int value;

virtual void consume_value() {...}

public:

Consumer(Buffer &b) : buffer(b) {}

$Consumer() {

while (1) {

// fetch values out of the buffer

// and consume it

value = buffer.get();

consume_value();

}

};

main() {

Buffer buffer;

Producer prod1(buffer);

Producer2 prod2(buffer);

Consumer cons(buffer);

// three processes will use the buffer;

// two will produce values and put it

// into the buffer, one will fetch

// values out of the buffer and consume

// these values

}

Figure 8. Producer-Consumer-Problem

7. Implementation of QPC++

The language QPC++ is described in detail in [8]. A prototype is implemented on SUN-Sparc workstations. A compiler [9] and a run-time system [10] have been developed. The QPC++-compiler checks the correctness of a given QPC++-program. Further, it generates C++-code which is afterwards compiled by an existing C++-compiler. The generation of code includes the transformation of the new constructs of QPC++ into calls of routines which are defined by the run-time system of QPC++. Figure 9 illustrates the use of the compiler and the run-time system of QPC++.

Figure 9. Compiler and Run-Time System of QPC++

7.1 Run-Time System

The existing run-time system of QPC++ has been designed for uni-processors. It has been implemented in C++ on SUN-Sparc workstations on top of the UNIX operation system. The run-time system mainly deals with three tasks. Firstly, it provides mechanisms which facilitate the creation of processes. Secondly, it cares about the scheduling of processes, and thirdly, it implements the interprocess communication.

Processes are implemented as so called lightweight processes. They are working quasi-parallel in a common virtual address space of a (heavyweight) UNIX process. Each lightweight process has its own stack.

The scheduler is implemented as a lightweight process with special tasks. When it is called, it deactivates the running process by storing the actual contents of the hardware registers. Afterwards, it activates another process by either starting its execution or loading its earlier stored registers. Thus, a process switch is not much more expensive than a usual call of a function. The scheduler is either called explicitly by some routines of the run-time system or implicitly after a certain period of time has been elapsed (time slicing).

Lightweight processes are working in a common virtual address space. Thus, the interprocess communication could be implemented in a very efficient way. The call of a member function of a process is compiled into the definition of a special class. An object of that class is created which includes the member function and the actual arguments. The address of the object is appended to the mailbox of the server. Mailboxes are implemented as linked lists. When a server wants to accept a special request, it has to look for it in its mailbox. It fetches the object out of the mailbox and calls a special member function of it. Within the execution of that member function the corresponding member function of the server is called.

Figure 10. Hierarchy of Classes of the Run-Time System

The run-time system of QPC++ is implemented in C++ in form of a hierarchy of 29 classes. The main part of it is shown in figure 10. Class QP_Stackframe offers routines for storing and loading of hardware registers. Class QP_Task uses the inherited functions for the implementation of lightweight processes. Class QP_Communication adds member functions which facilitate the interprocess communication. At last, QPC++-processes are defined as instances of classes that are directly derived from class QP_Process. QP_Process changes some inherited member functions implementing the special semantics of processes of QPC++. The main process of a QPC++-program is created as an instance of class QP_Main_Process. Internally, the run-time system defines an object of class QP_Scheduler which handles the scheduling. The scheduler is implemented as a lightweight process, too. However, in contrast to other lightweight processes, it performs special tasks. The basic scheduling mechanisms are implemented in class QP_Simple_Scheduler. By deriving classes from QP_Simple_Scheduler and redefining a virtual member function of it, special scheduling algorithms can be implemented. Thus, it is very easy to compare different scheduling strategies.

7.2 Compiler

The QPC++-compiler checks the correctness of a given program. Using the routines of the run-time system of QPC++, it further generates C++-code which afterwards is compiled by an existing C++-compiler. Currently, the C++-compilers of AT&T and GNU are supported.

8. Related Work

8.1 Classification of Parallel Object-Oriented Programming Languages

Investigations in merging concepts of parallel and object-oriented programming have already been made for several years. Parallel object-oriented programming languages (POOPLs) combine the advantages of object-oriented programming - especially reusability and extendibility of software - with the characteristic of parallel programming, namely the concurrent execution of certain operations. On the one hand, that can be used for the implementation of applications on distributed systems. On the other hand, the development of software in certain areas of application (simulations, interactive graphical user interfaces, ...) is simplified, not only concerning multi-processor systems but uni-processors, too.

One way to classify POOPLs is described in [11]. Several languages are compared with respect to what objects stand for. Objects may be considered as processes, shared passive abstract data types, or as encapsulation of multiple processes and data. Several languages of each category are described. Another good overview about existing POOPLs is given in [12].

An alternative way to classify POOPLs is to distinguish, in which way the languages have been defined. A POOPL can be defined as a new language, as the extension of an existing object-oriented language with features of parallel programming, or as the integration of object-oriented concepts into an existing parallel language. QPC++ is a language of the second category. The advantage of languages of that category lies in the fact that existing class libraries in general can be used in the extended language without any problems.

In the following, some other languages of the second category will be described briefly. They are all extensions of C++. Afterwards, they will be compared with QPC++, with regard to the motivation and the aims underlying the definition of the extension. C++ is well suited as a base language because of its efficiency, its availability, and its wide spreading. C++ has also been chosen as the base language of QPC++ because an already existing user interface toolkit, implemented in C++, shall be reimplemented without much effort [13]. Other languages of the second category are, e.g., CEiffel [14], Eiffel|| [5] and ConcurrentSmalltalk [15]. They are extensions of the object-oriented languages Eiffel or Smalltalk. Conceptually, the extensions made in these languages could be made in C++ as well.

8.2 Parallel Extensions of C++

DROL

DROL [16] is an extension of C++ with the capability of describing distributed real-time systems. It supports the definition of sequential active objects. Timing constraints can be associated with the member functions of active objects. When an object misses a specified protocol during run-time, a special defined exception handling is initiated.

C_NET

C_NET [17] is a POOPL which adds concepts of the parallel programming language OCCAM to C++. The integration of mechanisms for expressing parallelism has been carried out orthogonally to the concept of classes of C++. That means, classes and objects keep the same properties as in C++; processes are created in OCCAM-like par-statements using objects as shared passive abstract data types.

C_NET consists of all existing syntactical constructs of C++, like functions, types, and classes. Additionally, a set of new declarations and statements has been introduced for process creation (par), communication and synchronisation (recv, send, alt), and communicating channel declaration (chan). All these programming primitives behave like the corresponding primitives in OCCAM. Along with C++-like functions, there exist so called communicating functions. Syntactically, they are similar to usual functions. However, they can take ports as arguments and can communicate through them with other communicating functions.

Concurrent C++

C++ is an upward-compatible extension of the C programming language providing data abstraction facilities. Concurrent C [18] is an upward-compatible extension of C providing parallel programming facilities. By merging C++ and Concurrent C, the POOPL Concurrent C++ [19] has been defined. It offers both: data abstraction and parallel programming facilities. They are offered in an orthogonal way. Classes and objects can be defined like in C++. Processes can be defined like in Concurrent C. Thus, objects do not have any parallel properties and processes do not have any object-oriented properties.

In Concurrent C++, the mechanism for expressing parallelism is similar to the tasking model of Ada [20]. Processes are instances of so called process types. They may be created dynamically. Their activities are described in bodies similar to bodies of QPC++. Processes interact with other processes by means of so called transactions. Transactions are associated with processes. They can be regarded as services that are called by other processes and must be accepted by the process itself. Transactions have the form of function declarations in C++. However, there is no code directly associated with transactions. They only serve as points of synchronization. Transactions may be defined in form of synchronous or asynchronous transactions.

C&&

C&& [21] adds mechanisms for expressing parallelism by means of cobegin-coend-blocks to C++. Cobegin-coend-blocks are well known from Concurrent Pascal [22]. C&& offers the possibility to define special classes (parclasses). Instances of parclasses are passive objects (parobjects). Calls of member functions of parobjects can be placed in cobegin-coend-blocks. The statements within an cobegin-coend-block are executed in parallel. It is not allowed to execute two or more member functions of the same parobject simultaneously. Member functions of parobjects can only be used exclusively.

Communication between parobjects is performed by means of special send/receive-procedures. Communication is data-oriented and always asynchronous. Besides the normal sending of messages by calling the send-procedure which have explicitly to be received by calling the receive-procedure, there exists a special send-construct, called telegram. A telegram-message causes an immediate interruption of the actual executed member function of the addressed parobject. An exception handling is initiated. After finishing the exception handling, the interrupted member function can continue.

ACT++

ACT++ [23] is a parallel extension of C++ which supports the actor model of concurrent computation [24]. An actor is a self-contained active object. Actors can be defined as instances of classes which are derived from a predefined class ACTOR. Interaction among actors can occur only through message passing. Each actor is associated with a unique mail queue whose address serves as the identifier of the actor. Member functions of actors are used for message passing. However, they are not called directly. Instead, their addresses and the actual arguments are packed in special send-constructs. When an actor receives a message, it decodes its information and calls the corresponding member function. Message passing is always done asynchronously. However, it is possible for the addressed actor to pass results back by means of so called Cbox-objects which are much like future variables in QPC++.

Within an actor class, a so called behaviour can be defined which determines how the actor reacts to the receiving of a message. In processing a message, the actor can change its behaviour using the become-statement. Executing a become-statement, a new actor of a different class is defined which becomes the deputy of the old actor. In general, the class of the deputy has the same interface as the class of the old actor. However, the implementation of some member functions may be different. When the old actor receives a message, it automatically forwards it to its deputy which then will process it.

KAROS

KAROS [25] is an exploratory language based on C++ which has been designed for reliable distributed applications. It has been implemented as a C++ class library, along the line of ACT++.

KAROS provides two kinds of objects: ActiveObjects and DataObjects. ActiveObjects are global logical units of distribution. Their references may be known and passed to other ActiveObjects in the system. On the other hand DataObjects are passive objects. They are always local to an ActiveObject. ActiveObjects communicate similar to actors in ACT++ by passing values in an asynchronous way using future variables to store reply values. A client may ask if the result has arrived or decide to wait for it anyhow or ask if the service has failed.

A so called ACS communication protocol has been adopted to KAROS to deal with concurrency control and failure recovery in distributed applications[26].

mC++

mC++ [27] is an extension of C++ which supports parallelism on several levels of abstraction. Besides the definition of C++-like objects, it is possible to define coroutines, monitors, coroutine-monitors, and tasks as instances of special kinds of classes.

Tasks are very similar to processes of QPC++. A task possesses a body which is implicitly activated after its creation and initialization. Tasks communicate via their member functions. Calls of member functions have to be accepted explicitly. Afterwards, they are executed just like member functions of processes of QPC++. At each time only one member function per task can be active. In contrast to QPC++, in mC++ a member function of a task is executed by the client, not by the server. During execution of a member function, the client can be postponed until some later time. The client is blocked. It is added to a condition queue. While the postponed task is deactivated, the server can accept and deal with other requests. Member functions of tasks can only be called synchronously.

In contrast to tasks the other kinds of objects of mC++ do not have a thread of control of their own. They are executed in the thread of a task. Coroutines are objects which can be activated repeatedly. However, a coroutine is not restarted at the beginning on each activation. A coroutine can explicitly suspend its execution. Its local variables are preserved. When control returns at some later time, the coroutine continues execution from the leaving point with the preserved values.

A monitor is an object with mutual exclusion. It cannot be accessed simultaneously by several tasks. It is particularly useful for managing shared resources.

A coroutine-monitor type has a combination of the properties of a coroutine and a monitor. Thus, a coroutine-monitor is a coroutine with mutual exclusion. It can be accessed by several tasks, whereas a coroutine can only be accessed by a single task. However, it cannot be accessed by several tasks simultaneously.

Concurrency Control Model for C++-Objects

In [28] a model of concurrency control for C++-objects is presented which supports concurrent execution of member functions of an objects. The implementation of the model is split into an extension to the language C++ and a class library. The class library supports asynchronous procedure calls and future variables.

The language extension concerns so called delay declarations as part of the definition of a class. They are used to associate so called conditional waits to member functions. Conditional waits are boolean expressions that will be evaluated when the member function is called. Execution is delayed and another request processed when the conditional expression evaluates to true. Furthermore, delay declarations can be used to specify synchronization constraints for the concurrent execution of member functions (intra-object concurrency) and to arrange activity priorities. Delay declarations are inherited by derived classes and may be overridden.

Task-Libraries

A different approach for the integration of mechanisms for expressing parallelism into object-oriented programming languages is offered by special libraries, e.g., in form of class libraries. Such libraries are often called task libraries. Predefined classes offer routines for the creation of processes and for the interaction among processes. Programmers can derive new classes from the predefined ones and use the inherited functions.

Such task libraries are described in [29], [30] and [31] for example. The advantage of class libraries to real language extensions is that no compiler has to be developed. Its disadvantage lies in the facts that the syntax is sometimes a little bit awkward, more errors can occur, and the use of some features can be limited (e.g., the feature of inheritance).

8.3 Comparison with QPC++

A classification of the described extensions of C++ in "good" approaches and "bad" approaches is not possible. Each language has been defined for a special class of application. Using it for the development of certain applications may have certain benefits, using it for the development of certain different applications may have certain disadvantages. Therefore, a comparison of the described languages with QPC++ can only be carried out with regard to the motivation and the aims which form the basis of the development of those languages.

As mentioned before, the main motivation of the definition of QPC++ was the harmonic integration of clear and easy to handle mechanisms for expressing parallelism into C++. QPC++ will be used particularly to support the development of interactive graphical user interfaces (GUIs). The use of object-oriented techniques and programming languages has already involved several advantages in this area compared with the use of traditional imperative programming languages (see [32] for example). However, GUIs are characterized by activities taking place in parallel. Several windows have to be managed. Occurring events, such as keyboard inputs and mouse clicks, have to be distributed to several objects. In principle, each object is allowed to communicate with each other object, often in a bi-directional way. The availability of autonomous active objects in conjunction with mechanisms facilitating synchronous and asynchronous communication and multicasting of messages makes the development of GUIs still easier than the use of passive objects of sequential programming languages (see [33] for further information).

DROL has been defined to support the development of distributed real-time applications. It mainly deals with timing constraints. KAROS has been defined to implement reliable distributed applications. Its ACS protocol mainly deals with failure recovery. In contrast to that, QPC++ has primarily not been defined to support the development of distributed applications. It focuses on the simplification of the modeling of large systems with inherent parallelism. QPC++ has been implemented and will be used in our projects only in form of a quasi-parallel language. Timing constraints and failure recovery are not needed in our applications.

As far as the motivation of the definition of the Concurrency Control Model for C++-Objects is concerned, the support of intra-object concurrency has been in the centre of attention. With the help of special delay declarations the concurrent execution of member functions of one object can be performed and synchronized. By that, very fine-graduated parallelism may be achieved. The aim of this approach is the maximization of the performance of distributed applications. However, the availability of intra-object concurrency does not imply any advantages as far as the modeling of GUIs is concerned.

The aim of the definition of mC++ has been the introduction of parallelism on several levels of abstraction into C++. A programmer can choose among objects, coroutines, monitors, coroutine-monitors, and tasks depending on the task which has to be performed. However, sometimes that leads to the development of programs that syntactically seem to be very confused. It is to be mentioned that tasks of mC++ are very similar to processes of QPC++. Nevertheless, QPC++-processes differ from mC++-tasks in that they support asynchronous requests. In mC++ it is not possible to call member functions of tasks asynchronously. In fact, asynchronous communication can always be performed by means of synchronous communication by defining buffer processes. However, such processes have explicitly to be defined by the programmer. In contrast to mC++, in QPC++ such buffer processes are automatically defined by the run-time system. The programmer has not to care about that. The missing of asynchronous requests is a great disadvantage of mC++ as far as the development of GUIs is concerned. In GUIs objects frequently communicate in a bi-directional way. Implementing bi-directional communication implies the availability of asynchronous requests in order to avoid deadlocks. In addition to the concept of asynchronous communication, the mechanism of multicasting of requests is a very useful mechanism. QPC++ is the only language of the presented ones which offers mechanisms that facilitate the multicasting of messages.

The motivation of the definition of C_NET has been the merging of the object-oriented concepts of C++ with the already known mechanisms of OCCAM to express parallelism. ACT++ adapts the actor model of concurrent computation to C++. In a similar way, Concurrent C++ uses the concepts of Ada and C&& uses the concepts of Concurrent Pascal to insert parallelism into C++. Programmers who are already familiar with the underlying parallel programming models and languages may have advantages using these extensions. However, QPC++ has been defined for programmers who have some experience with the object model of C++ but who do not need to have practical knowledge about parallel programming issues. In QPC++ the concept for expressing parallelism is fully based on the existing concept for defining objects. Interaction among processes is fully based on already known global variables and member functions. There are no additional send/receive-constructs. Without any restrictions, the features of object-oriented programming are available for processes as well. Thus, it might be very easy for a C++-programmer to switch to QPC++. And he/she can make use of the advantages of object-oriented software development just like before.

In summary, one can say that in comparison with the other described extensions of C++, QPC++ is the best suitable language for the modeling of large systems with inherent parallelism, like GUIs. QPC++ is the language with the minimal syntactical overhead. The added concepts for expressing parallelism are as simple as possible but as powerful as necessary. They are fully based on the class/object model of C++. Therefore, QPC++ is that language of the described ones that is the easiest to learn and the easiest to handle one. It might be very easy for a C++-programmer to switch to QPC++, even if he/she had no experience with parallel programming concepts before.

9. Conclusion

QPC++ is an extension of the programming language C++. It adds mechanisms for expressing parallelism to the base language. That is done in a non-orthogonal way by merging the new parallel concepts with the existing object-oriented concepts. In this way, QPC++ connects the advantages of object-oriented programming, i.e. modularity, reusability, extendibility of software, to the characteristic of parallel programming to execute activities concurrently. QPC++ is used in the XFantasy project. XFantasy is a project with the aim to design and implement an object-oriented user interface management system (UIMS) for multimedial information systems [13]. The UIMS will be implemented on top of a uni-processor. In principle, a parallel programming language is not necessary. However, it is - in form of a quasi-parallel implementation - very useful for the modeling of large applications and the transformation of the developed models into source code. Just as far as the modeling of interactive graphical user interfaces is concerned, often bi-directional relationships among autonomous objects occur which cause several difficulties in transferring such models into sequential source code. By offering autonomous sequential processes in form of active objects in conjunction with mechanisms that facilitate synchronous and asynchronous message passing and multicasting of messages, QPC++ supports a simple, clear and sufficient flexible transformation of such models into source code.

Acknowledgements

I would like to thank Prof. Dr. H.-J. Appelrath, R. Bruns, Dr. H. Eirund, R. Götze, H. Lorek, and B. Müller for reading and commenting on this work.

References

1. B.W.Kernighan, D.M.Ritchie (1978). The C Programming Language. Prentice Hall

2. B.Stroustrup (1991).The C++ Programming Language, Second Edition. Addison-Wesley

3. P.America (June 1987). Inheritance and Subtyping in a Parallel Object-Oriented Language. BIGRE(54), pp. 281-289

4. P.Wegner (1987). Dimensions of Object-Oriented Language Design. Proceedings OOPSLA´87, ACM Press, pp.168-182

5. D.Caromel (Sep./Oct. 1990). Concurrency And Reusability: From Sequential To Parallel, JOOP

6. A.Yonezawa, E.Shibayama, T.Takada, Y.Honda (1987). Modeling and Programming in an Object-Oriented Concurrent Language ABCL/1. In [12], pp. 55-89

7. D.G.Kafura, K.H.Lee (1989). Inheritance in Actor Based Concurrent Object-Oriented Languages. Proceedings ECOOP´89, S.Cook, Nottingham, Cambridge University Press, pp.131-145

8. D.Boles (1992). QPC++ - A Parallel Object-Oriented Programming Language, Syntax and Semantics, Version 4.0. Fachbereich Informatik, Universität Oldenburg, Germany, internal paper (in german)

9. D.Boles (1992). QPC++ - A Parallel Object-Oriented Programming Language, Compiler, Version 4.0.2. Fachbereich Informatik, Universität Oldenburg, Germany, internal paper (in german)

10. D.Boles (1992). QPC++ - A Parallel Object-Oriented Programming Language, Run-Time System (Uniprocessor), Version 4.0.2. Fachbereich Informatik, Universität Oldenburg, Germany, internal paper (in german)

11. M.Papathomas (July 1989). Concurrency Issues in Object-Oriented Programming Languages. Centre Universitaire d`Informatique, Genève

12. A.Yonezawa, M.Tokoro (1987). Object-Oriented Concurrent Programming. The MIT press, Cambridge, Massachusetts

13. H.-J.Appelrath, R.Götze (1991). XFantasy - an Object-Oriented UIMS for Multimedial Information Systems. Fachbereich Informatik, Universität Oldenburg, Germany (in german)

14. K.-P.Löhr (Oct. 1992). Concurrency Annotations. In OOPSLA`92 Conference Proceedings, ACM Sigplan Notices, Volume 27, Number 10

15. Y.Yokote, M.Tokoro (1987). Concurrent Programming in ConcurrentSmalltalk. In [12], pp. 129-158

16. K.Takashio, M.Tokoro (Oct. 1992). DROL: An Object-Oriented Programming Language for Distributed Real-Time Systems. In OOPSLA`92 Conference Proceedings, ACM Sigplan Notices, Volume 27, Number 10

17. J.-M.Adamo (1991). Extending C++ with Communicating Sequential Processes. Published in Transputing´91, P. Welch et al., Eds., IOS Press

18. N.H.Gehani, W.D.Roome (1986). Concurrent C. Software-Practice and Experience, 16, pp. 821-844

19. N.H.Gehani, W.D.Roome (Dec. 1988). Concurrent C++: Concurrent Programming with Class(es). Software-Practise and Experience, Vol.18(12), pp. 1157-1177

20. A.Burns, A.M.Lister, A.J.Wellings (1987). A Review of Ada Tasking. In Lecture Notes in Computer Science, 262, edited by G.Goos and J.Hartmanis, Springer Verlag

21. V.Hüsken (Feb. 1990). Object-Orientation and Parallelism In Operating Systems and Programming Languages. Thesis, RWTH Aachen, Germany (in german)

22. P.B.Hansen (July 1975). The Programming Language Concurrent Pascal. IEEE Trans. on Software Engineering 1(2), pp. 199-207

23. D.Kafura, Kung Hae Lee (May/June 1990). ACT++: Building a Concurrent C++ with Actors, JOOP

24. G.Agha (1986). ACTORS: A Model of Concurrent Computation in Distributed Systems. MIT Press, Cambridge, MA

25. R.Guerraoui, R.Capobianchi, A.Lanusse, P. Roux (Jan 1992). Une vue générale de KAROS: un langage à objets concurrents destiné à des applications distribuées. Technical Report CEA, CE Saclay DEIN/SIR (in french)

26. R.Guerraoui, R.Capobianchi, A.Lanusse, P. Roux (1992). Nesting Actions through Asynchronous Message Passing: the ACS Protocol. Proceedings ECOOP´92, O.Lehrmann Madsen, LNCS 615, Springer-Verlag, Utrecht, The Netherlands, pp.170-184

27. P.A.Buhr, et all (Feb. 1992). mC++: Concurrency in the Object-oriented Language C++. Software-Practice and Experience, Vol.22, Issue No.2

28. H.Saleh, P.Gautron (1992). A Concurrency Control Mechanism for C++ Objects. Proceedings of the ECOOP´91 Workshop on Object-Based Concurrent Computing, M.Tokoro, O.Nierstrasz, P.Wegner, LNCS 612, Springer-Verlag, pp.195-210

29. Sun (Feb. 1991). The Task Library. Sun 2.1 C++ Manual Set, AT&T C++ Language System, Library Manual

30. K.E.Gorlen, S.M.Orlow, P.S.Plexico (1990). Data Abstraction and Object-Oriented Programming in C++. Teubner Verlag, Stuttgart, Germany

31. D.Grunwald (Nov. 1991). A Users Guide to AWESIME: An Object-Oriented Parallel Programming and Simulation System. University of Colorado at Boulder, Technical Report CU-CS-552-91

32. A.Weinand, E.Gamma, R.Marty (1989). Design and Implementation of ET++, a Seamless Object-Oriented Application Framework. Structured Programming 10/2, Springer-Verlag New York Inc., pp.63-87

33. S.Gibbs (1991). Composite Multimedia and Active Objects. Proceedings OOPSLA´91, pp.97-112