DataCROP (Data Collection Routing & Processing) is a configurable framework for real-time data collection, transformation, filtering, and management across IoT and cybersecurity domains. It emphasizes interoperability through a specialized data model for sources, processors, and results, enabling flexible workflow-driven analytics.
- Barley (v1.0): MongoDB, Apache Kafka, RabbitMQ, Kafka Streams, Node.js, React, optional Hyperledger Fabric.
- Farro (v2.0): Builds on Barley; MongoDB, Apache Kafka, RabbitMQ, Node.js, React, and algorithm support (Java, Python, R).
- Maize (v3.0, in progress): MongoDB, Apache Kafka, ELK stack; expanding observability and data services.
Deployable Farro demo: https://github.com/datacrop/farro-demo-deployment-scripts.
The documentation has been migrated to Docusaurus v3 and lives in the root directory. To run the site locally:
-
Ensure Node.js 20+ is installed.
-
Install dependencies:
npm install
-
Start the local development server:
npm start
This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
-
Build the static site (to verify production build):
npm run build
This command generates static content into the
builddirectory and can be served using any static contents hosting service.
All markdown content files are now located in the docs/ directory.
docs/home/- Framework overview and roadmap.docs/setup/- Maize setup instructions (MVP and manual per-component guides).docs/user-guide/- End-user workflows, data models, and Pipeline configurations.docs/dev-guide/- Developer guidance, including processor integration.