Glossary
Diminishing Returns (in Parallel Computing)
The principle that adding more processors to a parallel computing task eventually provides less and less additional speedup due to inherent sequential portions of the program and communication overhead.
Example:
While adding a second chef to a kitchen might double cooking speed, adding a tenth chef might not significantly improve overall meal preparation time due to limited counter space and coordination, illustrating diminishing returns.
Distributed Computing
A computational model where multiple independent devices or computers work together over a network to solve a single problem or run a program.
Example:
When you stream a movie, the content might be delivered from multiple servers across the globe, showcasing distributed computing.
Parallel Computing
A computational model where a program is broken into smaller operations that are performed simultaneously using multiple processors or cores within the same machine.
Example:
A modern video game console uses parallel computing to render graphics, process AI, and handle physics simulations all at once across its multiple CPU and GPU cores.
Parallel Execution Time
The total time it takes for a program or set of tasks to complete when processed simultaneously across multiple processors, determined by the longest sequence of tasks.
Example:
If two students work on three assignments (10, 20, 30 minutes), and one student takes the 30-minute task while the other takes the 10 and 20-minute tasks, the parallel execution time is 30 minutes.
Sequential Computing
A computational model where instructions are processed one after another in a single, linear sequence.
Example:
When a single-core processor runs a program, it executes each instruction in order, demonstrating sequential computing.
Sequential Execution Time
The total time it takes for a program or set of tasks to complete when processed one after another without any concurrent operations.
Example:
If a student completes three homework assignments taking 10, 20, and 30 minutes respectively, the sequential execution time for all assignments is 60 minutes.
Speedup
A metric that quantifies how much faster a parallel solution is compared to its sequential counterpart, calculated as (Sequential Execution Time) / (Parallel Execution Time).
Example:
If a complex calculation takes 100 seconds sequentially but only 20 seconds in parallel, the speedup achieved is 5.