Optimizing Game Performance With Job Queues And Worker Threads
Games require processing many complex tasks like physics, animation, and artificial intelligence in addition to rendering graphics. Trying to perform all these computations on the main game thread can overload it and reduce the game’s frames per second (FPS). By offloading expensive jobs to background threads that run asynchronously, the main thread is freed up to focus on time-sensitive operations like rendering, preventing FPS drops.
Understanding Game Loop Limitations
The game loop on the main thread manages input, updates game state, and renders the scene. This loop must iterate rapidly in order to deliver a high, consistent FPS. Computationally intensive tasks like physics, AI, and animation calculations can stall the game loop and cause FPS drops or instability when run synchronously on this thread. Specifically, running physics to calculate collisions, trajectories, and object interactions concurrently with AI systems that govern entity behaviors and decision making can easily overload the main thread.
Main Thread Overload Reduces FPS
Performing too many synchronous operations on the main thread creates resource contention and causes frame rate decreases. Every frame, key systems like animation, physics, input, and rendering compete for a slice of the main thread’s CPU time. When too many concurrent processes fight for the thread’s attention each frame, it takes longer to finish a loop iteration. This translates directly to a lower FPS.
Physics and AI Compete for Resources
Game physics and artificial intelligence are two of the most computationally expensive subsystems. Both utilize advanced math and algorithms to simulate real-world behavior. Physics systems simulate laws of motion, collisions, joint constraints, and other properties to animate objects realistically. AI systems govern entities based on knowledge representation, planning, learning, and other heuristics to mimic intelligent behavior. Together, these systems do a lot of numeric processing just within one game loop update. When physics and AI run synchronously on the main thread, they can easily dominate CPU time and reduce frame rates, diminishing overall performance.
Implementing a Job Queue
A common solution to offload expensive computation from the main thread is to introduce asynchronous job queues and worker threads. A queue stores separate jobs as data packages read by background threads. This divides processing into discrete jobs and spreads tasks across threads running in parallel. This prevents the main thread from being overloaded with intensive simulations and calculations.
Defining Jobs and Job Data
A job encapsulates a unit of work submitted to a worker thread for asynchronous processing. The job object defines the operation and batch of data to compute. For physics, this could be solving a constraint system over an array of simulated bodies and joints. For AI, it could be making decisions on a group of entities based on knowledge representation. Jobs package everything a worker thread needs to independently perform an atomic chunk of processing off the main thread. Carefully scoping atomic jobs is key to maximizing throughput.
Creating Thread-Safe Queues
Job queues mediate communication between the main thread adding jobs and worker threads removing jobs for execution. The queue stores job objects as they’re submitted, then hands them off on a first-in-first out basis to available workers. Queues must be thread-safe to prevent data corruption when modifying the queue across threads. This is usually achieved via a mutex lock governing all read/write access to the underlying container.
Scheduling Job Execution
Thoughtfully scheduling job execution maximizes performance gains. Often jobs have dependencies requiring them to run sequentially or in priority order. Physics collision detection may need to finish before resolving contacts and applying forces for example. Scheduling policies that respect dependencies and priorities avoid workers wasting cycles. There are also strategies like task pipelining that continually feed jobs to maintain a steady backlog, keeping worker threads highly utilized.
Creating Worker Threads
Dedicated worker threads run independently to chew through queued jobs without blocking the main thread. Care must be taken to properly synchronize access and avoid race conditions between threads.
Spawning Background Threads
Worker threads are spawned as background threads when initializing the job system, often equal to the number of available CPU cores. This parallelizes processing across the machine for maximal efficiency. Some frameworks offer threaded contexts to abstract OS thread management. Workers then enter an infinite loop awaiting jobs from the queues.
Assigning Jobs from the Queue
A common model assigns a single worker thread to each job queue. The worker continually pulls the top job from its queue and executes the enclosed operation. Key to this flow is that no two threads ever manipulate the same job data. Workers have ownership over jobs as they pop them for processing before marking them completed.
Synchronizing Thread Access
Though workers run independently, carefully controlling access to data is crucial to avoid race conditions. Locks prevent concurrent modification of game state like physics properties. Signaling avoids wasted cycles having workers poll for jobs, instead waiting until notified. Completion events let the main thread know when asynchronous results are ready. These constructs synchronize threaded execution.
Updating the Main Loop
While the main thread is liberated from intensive job processing, it takes on the role of dispatcher to integrate results. The game loop updates become focused on preparing jobs submissions and consuming their output.
Submitted Jobs to Queue
Each main loop tick identifies expensive computations, packages them into job data, and submits them to the appropriate queue for background execution. This keeps worker threads stocked with jobs each frame. Keys are scoping work into discrete units while maximizing batching to reduce overhead.
Processing Completed Jobs
As threads complete enqueued jobs, callbacks execute on the main thread to ingest results. For physics this means consuming updated transforms, velocities, and collision data to influence gameplay. For AI this involves decisions enacted on-screen by entities. Processing results each frame integrates background changes into the game state before rendering.
Example Job System Code
Here is some example C++ code demonstrating key components of a job queue threading system as described.
Job Queue Class Definition
class ThreadSafeQueue {
public:
void Push(Job* job);
Job* Pop();
int Count();
private:
mutex lock_;
queue jobs_;
};
Worker Thread Implementation
void WorkerThread() {
while(true) {
Job* job = jobQueue->Pop();
// Process job data
job->Complete();
}
}
Dispatching Jobs In-Game
void GameUpdate() {
// Build physics jobs
Job *job = CreatePhysicsJob();
jobQueue->Submit(job);
// Submit AI jobs
// Consume finished job results
SceneData->Render();
}
Conclusion
Implementing job queues and worker threads for expensive game tasks provides tangible optimization benefits. Offloading physics and AI frees up CPU cycles for the main thread to prepare scene rendering each frame. This stabilizes frame rates for silky animation with no fluctuation in game performance. Beyond improving raw frames per second, properly architecting asynchronous processing also makes games more scalable. Job systems scale well to take advantage of machines with more CPU cores by adding extra worker threads. With the foundation covered here, more advanced techniques like data-oriented design can further optimize how jobs access and transform data for lower cache misses and memory bandwidth. Just by following principles of scoped tasks, threaded parallelism and synchronization, major speedups are realizable in any gameplay application.
