Latest EMI Data evolutions
Access to distributed storage systems is becoming increasingly important for applications, consequentially to the wide diffusion of mobile devices accessing the network and facilitated by the increasing availability of broadband access, both traditional and mobile, fostering crossdevice access to multimedia data. The different applications, running over several platforms, demand standardized, transparent and shared access to distributed data.
In these regards, grids have been limited by their use of proprietary, non standard protocols.
This has restricted support from the mainstream end-user technologies, that has prevented wide usage of grid systems. Standardization and simplification of grid middleware are overall EMI objectives. For data movements such goals are driving EMI storage solutions (dCache, DPM, StoRM) into adopting standard protocols allowing seamless and device-independent access.
The WebDAV (Web-based Distributed Authoring and Versioning) protocol is a set of http
based methods enabling users to access, edit and manage remote files through a web interface. Latest versions of the most common operating systems (Mac OS X, Windows 7, Linux) implement native support for WebDAV, which means that files stored on EMI storage element will be easily accessed either from a Desktop file manager or a web browser.
User authentication supports both X509 certificates and user/password over http/ https. These authentication methods are commonly used, and their support guarantees accessibility from a wide number of devices.
WebDAV and http/s support are not the only EMI advances on data management.
There is also the enhancement of EMI storage products to support NFS 4.1/ pNFS, which allows the adoption of parallel NFS. NFS is another well known standard for network access to remote file systems. Several features, as parallel data access, and separation between filesystem metadata and file data location are implemented in version 4.1, contributing to a significant improvement of performance.
For dCache, all these features are already available in the EMI 1 Kebnekaise release. EMI 2 Matterhorn, next major release due in fall 2012, will see also DPM supporting WebDAV and pNFS. StoRM's release carried out with Matterhorn will not support WebDAV, but will provide equivalent methods for web based management of remote files. StoRM will also provide different protocols for the remote filesystem mount, with pNFS support planned for next year.
The upcoming Matterhorn release will feature EMI data lib, which is a common interface to the different EMI storage solutions, that, beside implementing a consistent layer for applications, will also simplify code maintenance reducing lines of code. EMI data lib will enable a transparent access to the same functionalities, independent from the underlying storage technology, and jointly with the implementation of widely diffused standards, will prefigure exploitation of grid solutions by any general purpose application where access to large storage capabilities is essential.
Contributed by Emidio Giorgio, Patrick Fuhrmann, January 2012