Parallelism
The simultaneous execution of multiple tasks or the different parts of the same task.
Parallelism involves dividing a large problem into smaller sub-problems, which are solved concurrently on multiple processors or cores. Unlike concurrency (which deals with managing multiple tasks at once), parallelism is about doing multiple tasks at the same time. It is key to high-performance computing and leveraging modern multi-core CPUs.
graph LR
Center["Parallelism"]:::main
Rel_concurrency["concurrency"]:::related -.-> Center
click Rel_concurrency "/terms/concurrency"
Rel_continuous_integration_ci["continuous-integration-ci"]:::related -.-> Center
click Rel_continuous_integration_ci "/terms/continuous-integration-ci"
Rel_multiprocessing["multiprocessing"]:::related -.-> Center
click Rel_multiprocessing "/terms/multiprocessing"
classDef main fill:#7c3aed,stroke:#8b5cf6,stroke-width:2px,color:white,font-weight:bold,rx:5,ry:5;
classDef pre fill:#0f172a,stroke:#3b82f6,color:#94a3b8,rx:5,ry:5;
classDef child fill:#0f172a,stroke:#10b981,color:#94a3b8,rx:5,ry:5;
classDef related fill:#0f172a,stroke:#8b5cf6,stroke-dasharray: 5 5,color:#94a3b8,rx:5,ry:5;
linkStyle default stroke:#4b5563,stroke-width:2px;
🧒 Explain Like I'm 5
Imagine you have to peel 100 potatoes. Doing it alone takes a long time. If you invite 4 friends and you all peel at the same time, it's finished much faster. That's parallelism—using multiple 'hands' (processors) to do work simultaneously.
🤓 Expert Deep Dive
Data Parallelism distributes data across different nodes, while Task Parallelism distributes different sub-functions. Amdahl's Law limits the maximum speedup achievable through parallelism based on the sequential fraction of the code. Massive parallelism is achieved via GPUs (thousands of cores) and MPI clusters (thousands of nodes).