koi_net.components.pipeline

Classes

KnowledgePipeline(log, cache, ...)

The knowledge pipeline, which ties together many systems to process knowldge objects.
class koi_net.components.pipeline.KnowledgePipeline(log, cache, request_handler, event_queue, graph)[source]

Bases: object

The knowledge pipeline, which ties together many systems to process knowldge objects.

Parameters:
cache: Cache
call_handler_chain(handler_type, kobj)[source]

Calls handlers of provided type, chaining their inputs and outputs together.

The knowledge object provided when this function is called will be passed to the first handler. A handler may return one of three types:

  • KnowledgeObject - to modify the knowledge object for the next handler in the chain

  • None - to keep the same knowledge object for the next handler in the chain

  • STOP_CHAIN - to stop the handler chain and immediately exit the processing pipeline

Handlers will only be called in the chain if their handler, RID, and event type match that of the inputted knowledge object.

Return type:

KnowledgeObject | StopChain

Parameters:
event_queue: EventQueue
graph: NetworkGraph
knowledge_handlers: list[KnowledgeHandler]
log: Logger
process(kobj)[source]

Sends a knowledge object through knowledge processing pipeline.

Handler chains are called in between major events in the pipeline, indicated by their handler type. Each handler type is guaranteed to have access to certain knowledge, and may affect a subsequent action in the pipeline. The five handler types are as follows:

RID

  • Provided kobj.rid.

  • If event type is FORGET, this handler decides whether to delete the knowledge from the cache by setting the normalized event type to FORGET. If the specified knowledge cannot be found in cache, the pipeline will exit.

  • Otherwise this handler decides whether to validate the manifest (and fetch it if not provided). If the manifest cannot be found or fetched from the source node, the pipeline will exit.

  • After processing, if event type is FORGET, the manifest and contents will be retrieved from the local cache, and indicate the last state of the knowledge before it was deleted.

Manifest

  • Provided kobj.rid, kobj.manifest.

  • This handler type is not called when the event type is FORGET.

  • Decides whether to validate the bundle (and fetch it if not provided). If the bundle cannot be found or fetched from the source node, the pipeline will exit.

Bundle

  • Provided kobj.rid, kobj.manifest, kobj.bundle.

  • Decides whether to write knowledge to the cache by setting the normalized event type to NEW or UPDATE. If the normalized event type remains unset, the pipeline will exit.

Network

  • Provided kobj.rid, kobj.manifest, kobj.bundle.

  • Decides which nodes (if any) to broadcast an event about this knowledge to by adding node RIDs to the kobj.network_targets set. If no network targets are set, final handlers are still called. An event formed from the knowledge object bundle and normalized event will be pushed into the event queue for each target node.

Final

  • Provided kobj.rid, kobj.manifest, kobj.bundle.

  • Final action taken after network broadcast.

The pipeline may be stopped by any point by a single handler returning the STOP_CHAIN sentinel. In that case, the process will exit immediately. Further handlers of that type and later handler chains will not be called.

Parameters:

kobj (KnowledgeObject)

register_handler(handler)[source]
Parameters:

handler (KnowledgeHandler)

request_handler: RequestHandler