Project Description

NPipeline is a .NET port of the Apache Commons Pipeline components. It is a lightweight set of utilities that make it simple to implement parallelized data processing systems.

Data objects flowing through the pipeline are processed by a series of independent user-defined components called Stages . A pipeline may have a number of different branches of execution, each of which is a fully qualified Pipeline in its own right.

Project Status

This project is in Pre-Alpha state and not ready for production use. I essentially needed a place to host the project :-)
  • Component design - Done
  • Pipeline construction and Lifecycle events - Done
  • Base class implementations for Interfaces - Done
  • Synchronous StageDriver - Done
  • Unit Tests for Framework - Done

Roadmap

  • Multithreaded State Drivers (Dedicated and Pooled)
  • Functional Stages
    • File Locator
    • Text File Reader
    • CSV File Reader
    • XML File Reader
    • Data-Key branch routing stage
    • Branch Join
    • Pool able Object stage
    • Dynamic Language Runtime stage (Iron Python)
    • File Writer stages
  • Add samples to show a simple data feed processing stage
  • Update documentation

Dataflow and Components

PipelineTutorial_1.png

Last edited Mar 24, 2010 at 3:08 AM by bscholze, version 8