Personal tools

The European Grid Infrastructure: opportunities and successes for the e-Research South community and the UK

The European Grid Infrastructure is a federation of National e-Infrastructures which are, in turn, federations of regional resource centres providing compute and data services for the research community at large.

egiLOGO
 

The European Grid Infrastructure (EGI) is, in many ways, a large jigsaw with many small and detailed pieces that have been gradually falling into place over the last two years. This picture of a pan-European e-Infrastructure supporting everything from traditional Research Infrastructures to distributed multi-disciplinary teams has always been clear, but the panorama is getting broader and the detail more clearly rendered. If you turn over all of these jigsaw pieces you will find that a significant number of them have ‘Made in the UK’ stamped on the back. The picture that is emerging is changing too; the federated sites running resources for the Large Hadron Collider’s physicists and other scientists are being transformed (through increased virtualisation and a fast emerging cloud model) into a dynamic and versatile infrastructure that is reaching out to more communities large and small. UK research communities are part of this expanded user base, from astronomy to structural biology. In many cases, researchers will be unaware that the distributed computational resources on which they depend are integrated into an infrastructure that benefits from broader support and coordination at a European level. Furthermore, many institutions are contributing to the development and innovation of the infrastructure itself. Teams in many parts of the UK are contributing to middleware components, job management and diagnostic tools, technical support services, training and communication.

 

The European Grid Infrastructure

Before reviewing the myriad of UK-originated contributions it is worth painting the big picture of how EGI fits into this landscape. The European Grid Infrastructure is a federation of National e-Infrastructures which are, in turn, federations of regional resource centres providing compute and data services for the research community at large. The Amsterdam-based not-for-profit organisation EGI.eu plays a coordinating role across this collaboration, both in terms of the community and the technology required to manage and maintain the infrastructure. EGI’s technical coordination role consists of three strands. Firstly, a resilient operation function that brings together the relevant managers from all sites that comprise the infrastructure. Secondly, a technical roadmap that coordinates the developmental direction of the components making up the grid platform itself. This involves defining a broad Unified Middleware Distribution (UMD) based upon requirements captured from across the communities. Thirdly, formal communication with a growing number of self-organised research communities through which requirements are captured and developed. The focal point of the community coordination is the biannual EGI Forums that bring researchers, developers, managers and operation teams together.

 

 EGI Image 1d

The European Grid Infrastructure enables access to computing resources for European researchers from all fields of science, from high energy physics to humanities. Credit: "Image: EGI.eu"

 

The most recent of these big events was the Community Forum held at the Garching Research Centre just outside of Munich at the end of March. Nearly 500 researchers, developers, operators, managers, policy makers and trainers attended agenda-setting keynote talks, presentations, demonstrations, lively workshops and training sessions. The spring sunshine pouring through the impressive glass roof of the Bavarian research centre throughout the week was a very welcome addition to the demonstrations and poster sessions that took place in the atrium. The Forum culminated in an inspiring talk by Professor Horst Zuse on the history of the computer, which was further embellished by the presence of his reconstruction of his father’s Z3, generally acknowledged as the first functional programmable computer in the world. The community will re-group in Prague in September where the next Forum will have a more technical focus. Then, in spring 2013, the next Community Forum will be held in Manchester. This event will provide an opportunity for the UK e-Research community to showcase their work, learn more about activities in the rest of Europe and beyond, and most importantly, participate in a range of dynamic sessions to share ideas and plans. A number of other technology projects and research communities have already indicated a desire to co-locate with future EGI forums in order to benefit from the vibrancy clearly seen and experienced in previous events. We look forward to seeing you in Manchester next spring, if not before, either as individual researchers, project teams or technology suppliers.

 

UK’s involvement in the European grid

 

So how does all of this affect the UK? National representation of the UK in EGI is handled by the UK National Grid Infrastructure (NGI) which comprises the National Grid Service (NGS), GridPP and other infrastructure projects. The NGS is led by the STFC e-Science Department, which is also a major partner in the EGI-InSPIRE project (part-funded by the European Commission) designed to establish EGI as a sustainable, long-term platform for distributed research computing. In addition to its coordination role, STFC is also responsible for the Training Marketplace, one of a suite of customisable technical services EGI provides for partners and user communities.

In addition to the work done by the e-Science Department at STFC, many other institutions and projects are playing their part in the EGI ‘ecosystem’. The following list is not exhaustive but shows the breadth of involvement from across the UK. GridPP is of course a key player, supporting the High Energy Physics (HEP) community as well as offering services to non-HEP users. Communication activities are organised by the NGS and Queen Mary College at the University of London (the latter through the e-Science Talk project). The STFC Daresbury Laboratory contributes to the development of the Grid Configuration Database (GOCDB), a portal providing information on the resource centres that make up the grid. The Oxford e-Science Research Centre (OERC) has been particularly productive over the last year in leading the Federated Cloud Task Force, a collaborative endeavour involving 23 institutions from 13 countries. It recently delivered an impressive demonstration of how customised virtual machine images from multiple European-wide sites can be integrated to deliver flexible tailor-made solutions for specific research needs. OERC is also involved in the European Middleware Initiative (EMI) project which is a major supplier of components for the infrastructure.

Over the last couple of years EGI has extended its scope by forming partnerships with other strategically important projects which can contribute to the longevity of the e-Infrastructure ‘ecosystem’. The Initiative for Globus in Europe (IGE) is one such project and it has chosen to co-locate its meetings with EGI’s forums. The University of Southampton is a partner in IGE and contributes regularly to the EGI Technical Coordination Board. This represents important work built upon the legacy of the OMII-Europe project and enables EGI to mix Globus services into its portfolio of components collectively distributed as UMD. The University of Westminster is involved in the EDGI and SHIWA projects, both of which have signed Memorandums of Understanding with EGI. EDGI is working to deliver dynamic, on-demand extensions of the connected Desktop Grids with Cloud resources and SHIWA has developed a simulation platform that runs as an online service. This enables researchers to run workflows developed on a diverse range of workflow platforms. One such platform is the Taverna workflow management system, developed by Professor Carole Goble’s team at the University of Manchester. Taverna is an important gateway for many grid users, such as the GAIA space astronomy scientists. The University of Edinburgh is involved in EU-DAT, a project that seeks to deliver a collaborative data infrastructure, as well as VERCE, which is looking to deliver a data-intensive e-Science environment for seismology researchers. EGI will be supporting both of these projects at the European level.

EGI, therefore, supports and provides opportunities for the UK research community in variety of areas. The NGS and GridPP are the primary channels of communication and these organisations can both provide support and guidance at various levels; from individuals to research groups, or from projects to institutions. In addition to providing services and resources, EGI and its partners run small, short-term projects (up to six months) that take on and resolve specific challenges. These challenges can range from virtualisation solutions for specific communities, to coordination activities to help research teams coalesce into articulate communities. Another thrust of EGI’s outreach is to establish stronger links with the ESFRI Roadmap projects and support their e-infrastructure ambitions wherever existing solutions can integrated. Like all creative endeavours, research is a judicious mix of perspiration and inspiration. The role of EGI in conjunction with its partners is to manage the hard work: the management, monitoring and development of the infrastructure so that researchers can focus on the innovation, inspiration and impact of their research.

Related links:

 

by Steve Brewer, Chief Community Officer, EGI.eu

 

 

Document Actions