Abstract
AbstractAs neuroscience datasets continue to grow in size, the complexity of data analyses can require a detailed understanding and implementation of systems computer science for storage, access, processing, and sharing. Currently, several general data standards (e.g., Zarr, HDF5, precompute, tensorstore) and purpose-built ecosystems (e.g., BossDB, CloudVolume, DVID, and Knossos) exist. Each of these systems has advantages and limitations and is most appropriate for different use cases. Using datasets that don’t fit into RAM in this heterogeneous environment is challenging, and significant barriers exist to leverage underlying research investments. In this manuscript, we outline our perspective for how to approach this challenge through the use of community provided, standardized interfaces that unify various computational backends and abstract computer science challenges from the scientist. We introduce desirable design patterns and our reference implementation called intern.
Publisher
Cold Spring Harbor Laboratory
Reference24 articles.
1. D. Kleissas , R. Hider , D. Pryor , T. Gion , P. Manavalan , J. Matelsky , A. Baden , K. Lillaney , R. Burns , D. D’Angelo et al., “The block object storage service (bossDB): A cloud-native approach for petascale neuroscience discovery,” bioRxiv, p. 217745, 2017.
2. W. T. Katz and S. M. Plaza , “DVID: Distributed Versioned Image-Oriented Dataservice,” 2019.
3. S. Plaza and W. Katz , “DVID,” Retrieved June 2018, https://github.com/janelia-flyem/dvid.
4. High-accuracy neurite reconstruction for high-throughput neuroanatomy
5. CATMAID: collaborative annotation toolkit for massive amounts of image data
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献