Project Description

NPipeline is a .NET port of the Apache Commons Pipeline components. It is a lightweight set of utilities that make it simple to implement parallelized data processing systems.

Data objects flowing through the pipeline are processed by a series of independent user-defined components called Stages . A pipeline may have a number of different branches of execution, each of which is a fully qualified Pipeline in its own right.

Project Status

This project is in Pre-Alpha state and not ready for production use. I essentially needed a place to host the project :-)

• Component design - Done
• Pipeline construction and Lifecycle events - Done
• Base class implementations for Interfaces - Done
• Synchronous StageDriver - Done
• Unit Tests for Framework - Done

Roadmap

• Multithreaded State Drivers (Dedicated and Pooled)
• Functional Stages
  • File Locator
  • Text File Reader
  • CSV File Reader
  • XML File Reader
  • Data-Key branch routing stage
  • Branch Join
  • Pool able Object stage
  • Dynamic Language Runtime stage (Iron Python)
  • File Writer stages
• Add samples to show a simple data feed processing stage
• Update documentation

Dataflow and Components