IRIX 6.5 » Books » Developer »
OpenGL Optimizer Programmer's Guide: An Open API for Large-Model Visualization
(document number: 007-2852-002 / published: 1998-06-09)
table of contents | additional info | download find in page
Chapter 14. Managing Multiple Processors
Using all processors all the time on a multiprocessor machine is desirable but difficult. If you do not keep processors active, then you are not exploiting the advantages of the machine; you won't see execution speeds approach the ideal of a linear increase with the number of processors. Even on a single-processor machine, you may benefit from using multiple processes because, for example, the host can cull while the OpenGL process is blocked, waiting for the graphics first-in-first-out queue to clear.
The tools in this chapter help you manage multiple processes. They provide an infrastructure that simplifies the design of cooperative tasks. The tools fit into three groups:
General, high-level tools that schedule and manage tasks for multiprocess (MP) programs
Tools that guarantee the orderly execution of changes to a scene graph when several processes would make changes
Low-level multiprocess tools
This chapter has the following sections:
MP Control Tasks and Related Classes
The following tasks and related classes are discussed in this chapter:
Thread management: The class opThreadMgr provides a convenient mechanism to dispatch and synchronize tasks that run on a set of processes. opThreadMgr is a general purpose multiprocessing “harness” that can be used independently of your rendering needs.
Action objects to define multithreaded tasks: opFunctionAction, opMPFunListAction, and opMPFunAction provide callbacks to define the tasks.
MP-safe scene-graph modification: The opTransactionMgr class coordinates Cosmo3D function calls that alter the scene graph so that alterations attempted by contemporaneous threads do not interfere with each other.
Low-level MP operations: opTaskBlock, opLock, opSemaphore, opMutex, and opBlockingCounter provide basic tools for managing more complex MP software architectures in a manner consistent with the OpenGL Optimizer library.
Overview of the Thread Manager
The class opThreadMgr provides an environment for submitting tasks to a set of threads and monitoring and coordinating task execution.
Sequence of Events for Thread Management
To start a thread manager, supply an opThreadMgr with four parameters:
the number of new processes to start
the number of priority levels in the queue for each process
how to prioritize the queues
the maximum possible number of threads you can start
This is the sequence of events to specify and perform tasks managed by an opThreadMgr:
The application defines callbacks for instances of action objects.
The application then passes the action objects to scheduling methods.
The scheduling methods place the action objects in one or more queues.
When an object reaches the head of its queue, it executes its tasks.
Managing Interprocess Dependencies
To design effective MP programs that keep processors occupied, you have to know when tasks finish and you need tools to manage the order of their execution. For example, you are likely to have process interdependencies such as “do A after B,” “wait for C,” and so on. The opThreadMgr methods waitForRequests() and markRequests() allow you to manage interprocess dependencies.
 | Note: When you use multiple processors, you cannot know in advance the order in which tasks finish. opThreadMgr provides queueing and coordination tools, but be cautious with programming assumptions about completion times when you write MP programs.
|
Classes for Scheduling and Defining Tasks
Three action objects define tasks scheduled by opThreadMgr's three methods, which distribute one task to one process, one task to many processes, and many tasks to many processes. Table 14-1 summarizes the processing features of the three scheduling functions and their action objects.
Table 14-1. Modes of Executing Multithreaded Tasks and Their Action Objects
Function
| No. Tasks
| No. Processes
| Action Object
|
|---|
SchedSPFun()
| 1
| 1
| opFunctionAction
| SchedMPFun()
| 1
| many
| opMPFunAction
| SchedMPFunList()
| many
| many
| opMPFunListAction
|
The callbacks for action objects are discussed after the class opThreadMgr and its scheduling functions.
Thread Manager: opThreadMgr
The opThreadMgr methods are largely self-explanatory, except for methods that control scheduling action objects, which are discussed in “Scheduling Methods”. The action objects themselves are discussed in “Difference Between Interprocess Control Methods”.
Class Declaration for opThreadMgr
The class has the following main methods:
class opThreadMgr {
public:
// Constructor/Destructor
opThreadMgr( int initialNThreads = 2,
int prioritiesPerThread = 1,
opQDiscipline qd = opPreEmptive,
int maxNumberOfThreads = opThreadMgr::defaultMaxThreads);
~opThreadMgr( void );
/* Managing Threads */
// Thread parameter query and set
opTID addThread( int numberOfPriorities = 1,
opQDiscipline qd = opRoundRobin );
int getThreadCount( void ) const;
// The number of queues associated with a given thread.
int getPriorityCount( opTID tid ) const;
// Queue-discipline query and set.
void setQDiscipline( opTID tid, opQDiscipline qd );
opQDiscipline getQDiscipline( opTID tid ) const;
/* Scheduling Tasks */
// Enqueue a user function.
void schedMPFunList( opMPFunListAction* actions,
const opTIDSet& tids = opAllTIDs,
opPriority p = opDefaultPriority);
void schedMPFun( opMPFunAction* action,
const opTIDSet& tids = opAllTIDs,
opPriority p = opDefaultPriority);
void schedSPFun( opFunctionAction *action,
opTID tid = opDefaultTID,
opPriority priority = opDefaultPriority);
static void executeSPFun (opFunctionAction* action);
// Blocking calls that wait for queued requests to finish.
void waitForRequests(const opTIDSet& tids = opAllTIDs,
opPriority p = opAllLevels);
opBlockingCounter *markRequests(const opTIDSet& tids = opAllTIDs,
opPriority p = opAllLevels);
};
|
The main methods of opThreadMgr form two groups:
Methods that schedule tasks. These methods are discussed in “Scheduling Methods”.
Methods that manage interprocess dependencies. These methods allow you to guarantee that a task finishes before you start a second task that depends on the first. The methods are discussed in “Managing Interprocess Dependencies”.
Once you have created an opThreadMgr, you can queue tasks with calls to one of the three scheduling methods. Scheduling methods differ in the kind of action object they accept and, therefore, the execution mode of the action (see Table 14-1 for a summary of the basic processing features of the scheduling functions).
Callbacks of the action objects define the scheduled tasks. Action objects are discussed in “Difference Between Interprocess Control Methods”.
These are the scheduling functions:
| schedMPFun(opMPFunAction* actions, const opTIDSet& tids = opAllTIDs, opPriority p = opDefaultPriority) | |
Places a single task described by the action object opMPFunAction on a specified set of threads at a specified priority.
| | schedMPFunList(opMPFunListAction* actions, const opTIDSet& tids = opAllTIDs, opPriority p = opDefaultPriority) | |
Places a set of independent tasks described by the action object opMPFunListAction on a specified set of threads at a specified priority.
| schedSPFun(opFunctionAction *action, opTID tid=opDefaultTID,
opPriority priority = opDefaultPriority) | |
Places a single task described by the action object opFunctionAction on a single thread with a specified priority.
| | executeSPFun(opFunctionAction *action); | |
Executes action immediately on the calling thread. action, a user-defined subclass of opFunctionAction, provides the callback function and data for a single-process task.
|
Interprocess Control Methods
The opThreadMgr methods markRequests() and waitForRequests() allow you to control interprocess dependencies.
| markRequests(tids, p) | |
Marks tasks and allows you to have the calling process stop at some later time and await completion of the tasks. markRequests() allows you to submit subsequent tasks to the thread manager before you get verification that the marked tasks are finished.
When you call markRequests(), it returns an opBlockingCounter initialized to count down from the number of tasks currently active on the threads tids, and places in the queue of each thread an operator that decrements the counter when the current task(s) on the thread finish (see the section “Implementing a Condition Variable: opBlockingCounter”). Setting p to an integer value other than opAllLevels restricts the set of marked tasks to those at level p.
To make a process wait until the tasks finish, call the function opBlockingCounter::waitForZero(void).
| | waitForRequests(tids, p) | |
Marks tasks by placing flags in process queues and immediately stops the calling process until the tasks finish until all tasks finish that were active on the set of threads tids at the time you called waitForRequests().
Setting p to an integer value other than opAllLevels restricts the set of tasks waited on to those at level p. A thread waiting for itself will deadlock.
|
Difference Between Interprocess Control Methods
Here is an example of the difference between markRequests() and waitForRequests(). Suppose you have task B, which depends on the completion of task A, and you have a set of other tasks, Q1,...QN, which B does not depend on and which do not depend on A.
If you use markRequest(), you can do the following:
Submit A to the thread manager.
Call markRequests().
Pass the returned opBlockingCounter to B.
Submit the tasks Q1,...QN.
Have B wait for A.
If you use waitForRequests(), you could do either of the following:
First option:
Submit A, have B wait for A to complete.
Submit Q1,...QN, thus delaying Q1,...QN until both A and B finish.
Second option:
Submit A and Q1,...QN.
Have B wait on all the tasks.
The markRequests() method provides greater flexibility in developing an execution sequence, regardless of the number of processes.
Defining Tasks for a Thread Manager
To specify the tasks managed by an opThreadMgr, pass one of the three action objects opFunctionAction, opMPFunListAction, and opMPFunAction to the appropriate scheduling function.
The scheduling functions place the action objects in thread queues. When an action object reaches the head of the queue, it performs its tasks. You specify tasks by defining callbacks.
The following sections provide details about defining callbacks:
opActionInfo Holds Thread Information
The opActionInfo class is used as an argument for any action-object callback. It provides information about the callback's opThreadMgr, the thread on which the callback is running, and the execution priority of the callback.
Class Declaration for opActionInfo
The class has the following main methods:
class opActionInfo
{
public:
// Creating and destroying
opActionInfo(opThreadMgr *threadMgr, opTID tid, opPriority priority);
~opActionInfo() ;
// Accessors
opThreadMgr *getThreadManager() const;
opTID getTID() const;
opPriority getPriority() const;
};
|
opFunctionAction: One Task, One Process
opFunctionAction is the class for running one task on one thread in a multi-threaded environment. To schedule an opFunctionAction, pass it to schedSPFunction().
Class Declaration for opFunctionAction
The class has the following main methods:
class opFunctionAction : public opAction
{
public:
opFunctionAction() ;
virtual ~opFunctionAction() ;
virtual opActionDisp function(const opActionInfo&);
};
|
Methods in opFunctionAction
You specify the action object's task by defining the callback function() when you create an opFunctionAction. The default return value causes the deletion of the class on return from function(). The possible return values of the callback are discussed in “Controlling a Traversal With the Callback Return Value opTravDisp”.
opMPFunAction: One Task, Many Processes
opMPFunAction is the class for running one task on a set of threads. For example, you might submit a rendering action to four processes and divide the screen into four areas. You could submit one function to four processes and encode the portion of the screen actually drawn by the function by using the thread identification number. To schedule an opMPFunAction, pass it to schedMPFunction().
The thread manager processes an opMPFunAction in three steps:
A single thread applies the callback begin() to signal that processes are available for the task.
Once begin() returns, each of the scheduled threads processes the callback perThread().
The last thread to return from perThread() calls end() to signal that the action is completed.
Class Declaration for opMPFunAction
The class has the following main methods:
class opMPFunAction : public opAction
{
public:
opMPFunAction() ;
virtual ~opMPFunAction() ;
virtual void begin(const opActionInfo&);
virtual void perThread(const opActionInfo&);
virtual opActionDisp end(const opActionInfo&);
};
|
| begin(info)
| | Is applied by the first thread scheduled to process an opMPFunAction.info describes the calling thread and points to the controlling opThreadMgr. No thread executes the perThread() callback until begin() returns. The default for begin() does nothing.
| | end() | | Is applied after the last thread returns from perThread(). The default return value, opDeleteThis, deletes the opMPFunAction. See “Controlling a Traversal With the Callback Return Value opTravDisp”.
| | perThread()
| | Defines the task to be performed by the threads. Define this function when you derive from opMPFunAction; the default for perThread() does nothing.
|
opMPFunListAction: Many Tasks, Many Processes
The opMPFunListAction class runs several tasks on several threads. To schedule an opMPFunListAction, pass it to an schedMPFunctionList().
The tasks of an opMPFunListAction are defined by a list of opFunctionActions. The thread manager processes the list in three step:
A single thread applies the callback begin() to signal that processes are available for the list of actions.
Once begin() returns, several threads perform the actions on the list.
When every action on the list has been performed, a single thread calls end() to signal that the list of actions has been processed.
You may not always know the set of tasks you wish to implement when you construct an opMPFunListAction. For example, you might want to render only visible surfaces, for which you have an occlusion culling traverser. The methods setActionArray() and addAction() allow you to build the list of functions before you begin the action.
Class Declaration for opMPFunListAction
The class has the following main methods:
class opMPFunListAction : public opAction
{
public:
opMPFunListAction(int nActions,opFunctionAction **actions);
virtual ~opMPFunListAction();
virtual void begin(const opActionInfo&);
virtual opActionDisp end(const opActionInfo&);
void setNumberOfActions(int numberOfActions);
int getNumberOfActions(void) ;
void setActionArray(opFunctionAction **actions);
opFunctionAction **getActionArray(void) ;
void addAction(opFunctionAction *action);
};
|
Methods in opMPFunListAction
| addAction() | | Adds a new action to the end of the list of action objects and increments the number of actions. The function assumes there is sufficient storage in the action array for another element. A call to this function between calls to begin() and end() causes an error.
| | begin(info) | | Is applied by the first thread to process an opMPFunListAction. info describes the calling thread and points to the controlling opThreadMgr. None of the opFunctionActions is executed until begin() returns. The default for begin() does nothing.
| | end() | | Is applied after all the callbacks have been completed. The default return value, opDeleteThis causes the opMPFunListAction to be deleted after returning from end(). See the section “Controlling a Traversal With the Callback Return Value opTravDisp” for a discussion of opActionDisp return values.
| | opMPFunListAction(int nActions, opFunctionAction **actions) | |
Constructs the action object. You specify the number of members in an opFunctionAction array that you have previously defined and provide an array of pointers, thus defining the action array.
| | ~opMPFunListAction() | |
Deletes the action object and the action pointer array but not the opFunctionAction elements themselves. Delete each of the opFunctionActions by specifying opDeleteThis as the return value of each of the opFunctionAction::function() callbacks.
| | setActionArray() | |
Sets the action array with a pointer to the opFunctionAction objects. The class destructor deletes this array; to avoid this, set the array to NULL. A call to this function between calls to begin() and end() causes an error.
|
Coordinating Threads That Change a Scene Graph: opTransactionMgr
The class opTransactionMgr coordinates scene-graph–altering activities of several threads by providing a “clearinghouse” where threads submit requested alterations. Without an opTransactionMgr, or another process coordinating tool, threads could perform simultaneous accesses to scene-graph elements and corrupt the scene graph.
The principle of the opTransactionMgr class is that a single process, usually the one responsible for rendering, controls changes to the scene graph. Other processes read the graph but do not change it directly. These processes initiate a change to the scene graph by submitting to the transaction manager opTransaction objects, which consist of sequences of deferred Cosmo3D function calls. The process that controls the scene graph affects the queued changes by a call to a member function of opTransactionMgr.
The operations that send opTransaction objects to the queue are so common that you can perform them by calls that do not refer to an opTransactionMgr class scope. These functions are run by the default instance of opTransactionMgr, and you can call them simply as opSync(), opCommit(), and opBlockingCommit().
The following sections provide details about multiprocess scene graph manipulations:
Class Declaration for opTransactionMgr
The class has the following main methods:
class opTransactionMgr
{
public:
opTransactionMgr();
~opTransactionMgr();
void commit(opTransaction* transaction);
void blockingCommit(opTransaction *transaction);
void processTransactions(void);
// Sets the amount of time per frame that the main thread
// may spend processing pending transactions.
void setMergeTimeLimit(float seconds);
float getMergeTimeLimit(void);
void setMaxPending(int n);
int getMaxPending(void);
};
|
Methods in opTransactionMgr
| commit() | | Sends a transaction to the queue. The calling process is not blocked unless the queue is full. Queue size is set by setMaxPending().
| | blockingCommit() | |
Sends a transaction to the queue and blocks the calling process until the transaction has been executed.
| | processTransactions() | |
Processes the queued transactions until the queue is empty or until the merge time limit is reached. All transactions that are taken from the queue are fully executed before processTransactions() returns. If a process starts before the merge time limit, it finishes.
| | setMergeTimeLimit() | |
Sets the amount of time per frame that the main thread may spend processing pending transactions.
| | getMergeTimeLimit() | |
Returns the current transaction-processing time limit.
| | setMaxPending() | |
Sets the length of the transaction queue, that is, the number of pending transactions after which any process that commits a transaction to the queue will be blocked.
| | getMaxPending() | |
Returns the length of the transaction queue.
|
The opTransaction class holds Cosmo3D functions that you can submit to the transaction manager. Each of the opTransaction methods appends a token representing a Cosmo3D function to the list to be submitted to the transaction manager.
Class Declaration for opTransaction
The class has the following main methods:
class opTransaction : public MPQElement
{
public:
opTransaction();
~opTransaction();
// csObject operations
void setUserData(csContainer *container, csData *data );
void unrefDelete(csObject *object);
// csGroup operations
void addChild (csGroup *parent,csNode *child);
void insertChild(csGroup *parent,int idx,csNode *child);
void removeChild (csGroup *parent,csNode *child);
void replaceChild(csGroup *parent,csNode *oldChild,
csNode *newChild);
// csShape operations
void setGeometry(csShape *shape, int i, csGeometry *geometry);
void setAppearance(csShape *shape,csAppearance *appearance);
// csMaterial operations
void setDiffuseColor(csMaterial *material,float r,float g,float b);
};
|
The opTransaction methods correspond to methods of a Cosmo3D class according to the following rules:
The name of the opTransaction method corresponds to a method of the Cosmo3D class.
The Cosmo3D class is the first argument of each opTransaction method.
The remaining arguments of the opTransaction method are the same as those for the Cosmo3D class method.
For example, setUserData( base, data ) appends a token for the function base->setUserData(data) to the list of transactions.
opCommit(), opBlockingCommit(), and opSync()
These functions correspond to the most commonly used opTransactionMgr methods. They are defined so that you can use them without referring to a specific opTransactionMgr scope; they are executed by the default instance of opTransactionMgr, _opTransactionMgr, which is initialized by opInit.
The functions opCommit() and opBlockingCommit() have actions that correspond to the like-named opTransactionMgr methods. The function opSync() calls an opTransactionMgr::processTransactions() and returns a value of 1.
Low-Level Multiprocess Tools
In addition to the high-level tools presented so far in this chapter, there are five OpenGL Optimizer tools that you can use to spawn processes and coordinate their activities. These tools typically use libc calls with similar names, but, to be consistent with the rest of the library, use the OpenGL Optimizer versions. Do not use the libc functions fork() and sproc() in an OpenGL Optimizer application.
The following sections provide details on low-level multiprocess tools:
This class implements a simple locking mechanism.
Class Declaration for opLock
The class has the following main methods:
class opLock
{
public:
// Allocates the lock from the arena that the opLock structure was
// allocated from.
opLock();
~opLock();
bool lock(void);
bool unlock();
};
|
The methods in opLock use the functions in ulocks.h; however, use opLock to be compatible with the rest of the OpenGL Optimizer library. These are the essential features of the two member functions:
| lock() | | Blocks until a process acquires the lock. lock() returns true unless an error occurs.
| | unlock() | | Releases a lock. unlock() returns false unless an error occurs.
|
Mutual Exclusion Within a Code Block: opMutex
The opMutex class provides a mechanism to simplify the control of mutual exclusion within a block of code. An opMutex acquires and holds the lock passed to its constructor until control exits the current scope. The lock is released when the destructor is called.
A typical use for opMutex is in conjunction with normal C++ scoping to make sure that a lock is released when control leaves a block. This is particularly useful when an exception could be thrown from within a block, or to guard against returning from the middle of a locked block. See the reference page opLock(3in) for more details and a code example. The file opMutex.h also contains a code example.
 | Note: The maximum number of locks in the system is 4096. No more than 65 processes may share a single lock.
|
To be compatible with the OpenGL Optimizer library, use the class opSemaphore to control semaphores.
Class Declaration for opSemaphore
The class has the following main methods:
class opSemaphore
{
public:
// Allocates the lock from the arena that the opLock structure was
// allocated from.
opSemaphore(int count);
~opSemaphore();
opBool p(void);
opBool v(void);
void init(int count);
};
|
| opSemaphore(count) | |
Constructs an opSemaphore with the counter initialized to count. The value of count reflects the number of resources available:
If count is greater than zero, count resources are available.
If count is negative, the absolute value of count is the number of waiting processes.
| | p()
| | Decrements the semaphore counter. If the count becomes negative, the semaphore will block the calling process until the count is incremented by a call to v() by another process. p() always returns a value of true.
| | v() | | Increments the semaphore counter. If any processes have been blocked and are waiting for the semaphore, the first process in the queue begins execution.
|
The method names p() and v() were introduced by Edsgar Dijkstra based on the signalling strategy used by Dutch trains; the names of the methods derive from the Dutch words “passern,” to pass (a train is passing); and “vrijgeven,” to give free (the track is free). See http://www.kzoo.edu/~k087023/algor/bio/.
Making Processes Wait on a Task: opTaskBlock
The class opTaskBlock controls interprocess dependencies by making any number of processes wait for the completion of a task.
These are the steps involved when an opTaskBlock is used:
A blocking task establishes a block by creating an instance of opTaskBlock and calling start().
Other processes wait until the blocking task finishes if they call the member function waitUntilFinished().
When the blocking task finishes, it calls finish() and all the waiting processes begin execution.
Class Declaration for opTaskBlock
The class has the following main methods:
class opTaskBlock
{
public:
opTaskBlock();
~opTaskBlock();
void start();
void finish();
void waitUntilFinished();
};
|
| finish() | | Is called by the blocking task when it finishes, thus allowing waiting processes to begin execution.
| | start() | | Is called by the blocking task to establish a block.
| | waitUntilFinished() | |
Is called by processes that should wait for the completion of the blocking task.
|
Implementing a Condition Variable: opBlockingCounter
This class implements the basic operation of opThreadMgr::markRequests(). It uses opMutex and opSemaphore to implement a condition variable and to provide more refined control over execution dependency between processes than you have with opTaskBlock.
The application creates an opBlockingCounter initialized to count down from x: opBlockingCounter C(x). After that, a process will block on a call to C.waitForZero() until C.decrement() has been called x times. Naturally, calls to C.decrement() should correspond to the completion of tasks the application wants to wait for.
Class Declaration for opBlockingCounter
The class has the following main methods:
class opBlockingCounter
{
public:
opBlockingCounter(int count);
~opBlockingCounter();
void decrement(void);
void waitForZero(void);
};
|
Methods in opBlockingCounter
Once a process starts after a call to waitForZero(), the opBlockingCounter reinitializes itself and is ready to receive waitForZero() calls from any process.
If process P is blocked by a call to waitForZero(), a call to waitForZero() by a second process R will block R until a call to decrement() after P starts.
OpenGL Optimizer Programmer's Guide: An Open API for Large-Model Visualization
(document number: 007-2852-002 / published: 1998-06-09)
table of contents | additional info | download
Front Matter
About This Guide
Part I. Getting Started
Part II. High-Level Strategic Tools for Fast Rendering
Part III. Specific Tools for Fast Rendering
Part IV. Managing and Rendering Higher-Order Geometric Primitives
Part V. Traversers, Low-Level Geometry Processing, and Multiprocessing
Part VI. Utilities and Troubleshooting
Part VII. Appendices
Glossary
Index
home/search |
what's new |
help
|