Submit Public Comments on Supplement 223: Repository Query, Inventory IOD, and Related Services, by January 7, 2022

This Supplement is being released for a second round of Public Comment to account for substantial edits made after the first round of Public Comment.

Scope and Field of Application
This Supplement introduces a new Repository Query SOP Class to obtain an inventory of a repository system, a composite Inventory IOD that is the equivalent persistent object instantiation of such an inventory, an Inventory Creation SOP Class to initiate asynchronous creation of Inventory SOP Instances, and SOP Classes to transfer/query/retrieve Inventory SOP Instances.

A use case of steadily increasing significance is porting large DICOM repositories from one image management system (PACS or VNA) to another. Users typically replace their PACS after ~12-15 years, often with change of vendor. Replacement requires migrating historical data to the new system. Thus, every year, 5-10% of user organizations may be doing a PACS data migration.

Old images are routinely retained “forever”, and data set sizes are increasing with 3D/4D and multimodality studies. Archives in many institutions store over a billion instances with data volumes over one petabyte. Migration approaches need to operate at large scales, and handle both on-premises and remote (e.g., cloud-based) storage. Migration often occurs while either the source system or the destination, or both, are in clinical operation, but systems designed and configured to handle the throughput of regular operations might not have capacity for the additional massive input/output requirements of migration. With a data transfer rate of 1 terabyte / day (quite high for even the most advanced PACS), the time to transfer a petabyte archive is 3 years. Performance constraints exist on both the source and destination systems.

Similar needs arise when healthcare institutions merge previously disparate repositories into an enterprise repository - the old archives need to be migrated. This is an increasing need with the accelerated pace of healthcare organization consolidations. Conversely, large sets of archived data may sometimes need to be migrated out of a repository to support business divestment or realignment in a healthcare organization.

There are also research use cases, including artificial intelligence and machine learning, where bulk access to the archive is desirable, and such uses might leverage some of the same mechanisms developed for migration. PACS audit and quality control may also utilize some of the standardized functionality developed for migration, such as an archive inventory and metadata to identify the data produced by a particular unit or by a particular modality.

The current DICOM Standard does not address the use case and technical interoperability requirements for migration of a full enterprise repository data set, and it is currently ill-suited for the major performance issues of migration.

The current Standard is designed for routine daily department operational workflows – acquisition, storage, analysis, and reading of imaging studies associated with individual patients. The Standard is optimized for identifying and transferring the objects for, at most, just a few studies or patients at a time. Network Query/Retrieve operations are synchronous, and the network connection must remain open until the operation is complete. The number of items in a response therefore is typically restricted to an implementation-specified number appropriate for a human interface, and the Standard is silent on behavior when that number is exceeded. Even media-based data exchange is specified only for the use cases of limited file sets, basically what can fit on a DVD.

A key requirement for migration (and other use cases) is the ability to have an inventory of all studies, series, and instances in an archive. While the current Query Service (DIMSE or equivalent DICOMweb) could be used, limitations on number of responses and the synchronous protocol require the use of a possibly very large number of partial Query requests, with undefined behavior when query limits are exceeded. This makes producing such an inventory difficult with the current Query Service.

While the current Standard Retrieve and Storage Services make moving data possible, they require significant overhead for the transfer of each object. A standardized method of direct filesystem access to stored object files is needed. Recognizing the varying technical implementations and business needs of repository manufacturers and of inventory users, this Supplement specifies two mechanisms to achieve a robust inventory production.

This Supplement specifies a new Repository Query SOP Class that includes features supporting a sequential set of Queries intended to produce a complete repository inventory. These features include defined behavior for Queries that reach a system limit for number of responses, and an ability to resume at the next record in a subsequent Query. All key attribute selectors available in the current Query/Retrieve Service are also available in the Repository Query SOP Class, and some new selection mechanisms are defined that are useful for inventory production.

This Supplement also specifies an Information Object Definition capable of encoding an inventory of all studies, series, and instances in a repository. This is functionally equivalent to a Query response that returns an inventory of the entire repository database, or a subset thereof as specified by key attributes. SOP Instances of this Inventory IOD could be produced upon local initiation by the repository administrative controls. The Supplement further defines a mechanism to remotely initiate the production of the inventory through a DICOM network service, and allow production to proceed asynchronously. The inventory SOP Instance(s) would be available for transfer when production is complete through Store/Query/Retrieve mechanisms similar to those specified for other DICOM Non-Patient Object classes (such as Color Palettes).

Only inventory of patient-related studies, series and instances is defined. Inventory of non-patient objects is out of scope for this Supplement.

Since archive systems may optionally support direct filesystem access to DICOM Part 10 compliant files, for all or some of their stored instances, the Repository Query and the Inventory IOD allow a URI link to such accessible files. To support a common PACS implementation design, wherein the archive may retain metadata updates (e.g., changed patient IDs) in its database and not propagate them to the stored instances, the inventory provides the current metadata, which may differ from the values in the stored instances.
While this Supplement defines an IOD and Services to support migration, it is not in itself a complete standard for migration. Migration involves many other processes, some of which may be supported by other standards such as HL7, and some of which are not (yet) supported by any standards.

Please submit comments by January 5th so that I can consolidate and submit comments to DICOM by the January 7th deadline.