A (Dead End?) Arrowized Dataflow Parallelism Interface Attempt
So I’ve been having several ‘dead end’ projects in Haskell recently that I’ve sort of just scrapped and move from, but I decided that it might be nice to document some of them :) For reading back on it later, for people to possibly help/offer suggestions, for people to peruse and possibly learn something, or for people to laugh at me. Here is my most recent semi-failure — implicit dataflow parallelism through an Arrow interface.
tl;dr:
- Compose parallelizable computations using expressive proc notation.
- Consolidate and join forks to maintain maximum parallelization.
- All data dependencies implicit; allows for nice succinct direct translations of normal functions.
- All “parallelizable” functions can also trivially be typechecked and run as normal functions, due to arrow polymorphism.
The main problem:
- Consider
ParArrow a c,ParArrow b d,ParArrow (c,d) (e,f),ParArrow e g, andParArrow f h. We execute the first two in parallel, apply the third, and execute the second two in parallel. Basically, we want two independentParArrow a gandParArrow c hthat we can fork. And this is possible, as long as the “middle” arrow does not “cross-talk” — that is, it can’t be something likearr (\(x,y) -> (y,x)).