Data scientists everywhere are delving more deeply into deep learning (DL).
Developers of convolutional, recurrent, and other DL models use Spark in their projects for the following reasons:
- Available platforms, libraries, and tools: Spark lets DL developers quickly train and deploy multi-layered neural nets using libraries and compute clusters that are already at their disposal.
- Familiar computing model and development framework: Spark allows DL developers to get up to speed on distributed architectures without having to master an unfamiliar DL-specific computing model such as NVIDIA’s CUDA.
- Flexible execution and deployment options: Spark facilitates developer experimentation with DL architectures that incorporate model training on Spark clusters (horizontally scalable, CPU-based, multi-node, in-memory) alongside DL-optimized accelerators that do fast matrix manipulations with DL-optimized algorithm libraries on DL-optimized, GPU-based single-node co-processers.