Federated Network Components
Federated Network (FN) was developed in collaboration with MIDRC, ARPA-H, NIBIB, and ACR to enable the creation of distributed medical imaging and clinical data registries. FN is an application currently hosted on ACR Connect, free software that is installed behind health facility firewalls, providing secure and seamless data handling. With FN, facilities can build patient cohorts and extract demographic information into an index. While the imaging and clinical data remain at the facility, the index is then sent to a central data repository, such as ACR Data Analysis and Research Toolkit (DART), where central analysts can create cohorts for federated jobs. In addition, the data repository can launch federated jobs (e.g. AI algorithms) that run at facilities and return results to the central repository.
Though the libraries below are extensible to any platform, they are currently being utilized on a tech stack including ACR Connect, ACR DART, and AI-LAB.
The ACR is committed to open science and all components developed under this collaboration are available as open source software described in the table below:
Component | Description | Source |
|---|---|---|
Index-Creation | This C#.NET library source code takes DICOM files, parses them, and generates a metadata index in JSON format that conforms to the ARPA-H-BDF Metadata Specification. | Coming soon… |
Index-Ingestion | This Nifi job script source code parses a file indexed according the standard described in ARPA-H-BDF Metadata Specification and packaged as a JSON file. This script parses the JSON index and stores it in a JSON format for use in Elastic Search or other JSON repositories such as Mongo DB. |
|
Federated-Central-Orchestration | This C#.NET library source code handles the central business logic for federated jobs. When a job is “triggered” for a given multi-site cohort based on the standard index (ARPA-H-BDF Metadata Specification), it creates site-specific jobs that are triggered from each site’s client. It also receives updates from the clients so that the status of the federated job is understood by the central user. Finally, results of the federated algorithm are also received and stored. |
|
Federated-data-contracts | This C#.NET library source code contains the data contracts used between the local client orchestrator (AI-LAB) and the central orchestrator (DART). This facilitates processes such as how to retrieve the job from the central orchestrator (DART) and provide continual updates of job status. It is also responsible for the definition of the final payload that is returned to the central orchestrator. | acrdart-federated-job-api-contract.zip
|
Federated Client Processing | This C#.NET library source code performs the communication tasks associated with the local AI performer (AI-LAB) interacting with the central orchestrator (DART). This includes polling to retrieve the jobs from the orchestrator and begin local execution, updating status for the central orchestrator and returning the payload. | acrdart-federated-job-api-client.zip
|
This python-based Connect utility source code wraps the MIDRC Stanford Penn De-identifier so that it accepts CSV, XLSX or JSON inputs, adds to the data files for the hide-in-plain-sight categories, and has improved logging. A Docker file and Windows executable version (Pyinstaller) of this utility is also provided. For these implementations, the model files are included. | MIDRC-ACR_report_de-id.zip |
Definitions:
DART (Data Archive and Research Toolkit) - is a resource managed by the ACR to conduct research. DART hosts a variety of open source and commercial tools for multi-modal research. In the context of creating federated jobs, DART is used as the central orchestrator.
ACR CONNECT – ACR connect is an infrastructure managed by the ACR that includes a variety of cloud and local components designed to facilitate bi-directional communication including local data ingestion, data processing, data upload, etc. In the context of federated learning, ACR Connect serves as an interface to local systems and a “host” for the relevant workflow applications (e.g. AI-Lab, Federated Learning).
AI-LAB – Is a local application hosted on ACR-Connect in the context of Federated Learning. It serves as the local performer that receives and executed federated jobs on behalf of a central orchestrator.
Federated Network – A local application designed to index and manage local data repositories for use in federated workflow.
Terms and Conditions
The code in the above table is shared under the Apache 2.0 License: LICENSE .