EASInet - IBM's Contribution to Scientific Networking in Europe
27th of June, 1992
Document Number STR92-SJ01
AbstractIn November 1987, IBM launched the EASI programme which establishes partnerships with leading European academic and research organisations to foster supercomputing education and research.
EASInet was a logical development of the EASI programme to permit the EASI partners to exchange information on supercomputing techniques and to share in joint research where appropriate.
This paper will describe the multiprotocol environment, connectivity and routing, the concept used for central management and operation, and the convergence with other existing networks.
We are addressing the issues in integrated wide area networks, local and metropolitan networks and high performance attachment of computing systems, and demonstrate that EASInet is an important component towards a future european high speed backbone network.
Table of ContentsAbstract
Data NetworkingEASInet Collaboration with other networks
- Geographical map of EASInet
- The IDNX backbone of EASInet
- The german EASInet connections
- Details of the T1-Link
- Infrastructure at EASIgate
- Some statistical data
- Configuration for ISO CLNP support
- EASInet as of 06-03-92
IntroductionThe vision of a pervasive high performance international network operating at magabit speeds and serving a wide range of research interests is becoming a reality, however, the European international connectivity is lagging behind. The main reason for the slow progress is the high cost of international circuits in Europe. Other barriers are technical, commercial, organisational and political, like the absence of pan-European suppliers of leased lines, the absence of wide collaboration between industry, government, PTT's and the research and educational community.
We will give a short introduction on some networking related highlightes in America and Europe, before we demonstrate EASInet, an important component towards a multiprotocol backbone for Europe.
American InternetWide Area Networking (WAN) is considered to have started in the U.S. in approximately 1975 with the ARPAnet. This was a network set up by the Defense Advanced Research Projects Agency (DARPA) and linked sites involed in advanced research funded by the American Department of Defense.
The technology used in DARPA includes a set of network standards that specify the details of how computers communicate, as well as a set of conventions for interconnecting networks. Commonly referred to as TCP/IP it supports the interconnection of many disparate physical networks and makes them function as a coordinated unit, called an internetwork, or internet.
Most of the domestic Internet is supported and coordinated by Federal agencies, in principal the Defense Advanced Research Projects Agency (DARPA), the Department of Energy (DOE), the National Aeronautics and Space Agency (NASA) and the National Science Foundation (NSF).
The National Research and Education Network (NREN) component of the High Performance Computing and Communications (HPCC) program dramatically expands and enhances the U.S. portion of an existing worldwide infrastructure of interconnected computer networks. In particular the NSF backbone network (NSFNET) is running at 45 Megabits per second already.
Advanced Network Service, a consortium of IBM Corporation, MCI Communications Corporation and MERIT Inc., has been founded to enter the business of data networking, initially for the academic and the research community.
The new not-for-profit organization manages and operates the federally-funded National Science Foundation Network (NSFNET) backbone, under subcontract to MERIT Inc., as well as provides a broad spectrum of networking services to researchers and educators in universities, federal laboratories and the private sector.
European NetworkingMany European countries have national data networks for use by their academic and research community. These networks are supplied by the national PTTïs and do not exceed 64kbps since the operational costs can not be provided by the universities and research institutes attached to the network.
The European Academic and Research Network (EARN) which was sponsored by IBM in 1984, was proving to be a success. Even old-fashioned, EARN was very robust and made it relatively easy for other countries to join and quickly obtain a reliable connection at a reasonable cost. About 50% non-IBM machines attached to EARN and their users participated in the mail and file transfer services, these older technology provided.
The Cooperation for Open Systems Interconnection Networking in Europe (COSINE) was responsible for setting up an Interim X.25 Infrastructure (IXI), which has been providing an X.25 service via 64kbps lines, connecting various countries together which have poorly developed infrastructure.
RARE, is the Association of European Research Networks and the contract holder for the COSINE implementation phase. At its Council of Administration (CoA) meeting in Blois in May 1991, it established a Task Force to examine the possibility of creating an Operational Unit to take responsibility for the provision of pan-European backbone and value added networking services.
The TCP/IP (Transmission Control Protocol / Internet Protocol) protocol suite, as used in the American Internet, has a substantial influence on the growth of the European Internet. Since this growth has not been officially planned, it has to rely on very informal management agreements. The IP coordination across Europe is handled by RIPE (Reseaux IP Europeens).
The EASI programmeThe EASI (European Academic Supercomputer Initiative) programme which IBM launched in November 1987, established partnerships with 18 leading European academic and research organisations in 9 countries to foster supercomputing education and research.
The program contained the following four major components:
- sale of an IBM 3090-600 with Vector Facility for a special price reduction
- support of cooperations on supercomputer applications using IBM mainframes
- provision of a managed network with connections of 64kbps among the EASI partners
- at least a 56kbps connection from Europe to the U.S.EASI Installations Country City Institution _____________________________________________________________________ Austria Vienna University Belgium Leuven KUL France Lyon IN2P3 France Montpellier CNUSC France Paris CEA France Paris CIRCE Germany Aachen RWTH Germany Braunschweig Technical University Germany Darmstadt GSI Germany Hamburg DESY Germany Karlsruhe KfK Italy Bologna CINECA Italy Rome University Netherlands Amsterdam SARA Spain Barcelona FCR Sweden Skelleftea UMEA Switzerland Geneva CERN Switzerland Zuerich ETHEASInet was a logical development of the EASI programme to permit the EASI partners to exchange information on supercomputing techniques and to share in joint research where appropriate.
Figure 1. Geographical map of EASInet
To coordinate the interests of the EASI sites and to decide on policy and general rules for the network an "EASInet Project Committee" (EPC) was formed. It consists of the representatives of five EASI sites, one representative of IBM, one of GMD and to intensify cooperation with other European networking organisations, one representative of the RARE Council of Administration (RARE - CoA). The EPC has established a lot of cooperative agreements with national and international European networking organisations. For many of them EASInet transports US-traffic or establishes European connectivity. Also many line sharing arrangements were made to create a backbone with higher speeds than 64Kbps.
*** Please see graphic ***
To avoid a bottleneck on the line between America and Europe, IBM ordered a T1-link (1.544Mbps) from the begining. Since this doubled the existing bandwidth for scientific networks between Europe and the U.S. it is a very attractive facility for other networks to collaborate with EASInet and improve the efficiency of scientific networking, EPC established a recommendation for the use of EASInet: "whoever applies for access to the T1-link should try to contribute to existing bandwidth by either increasing the bandwidth of used lines or by merging existing lines with EASInet-lines". This led to a network with much better international connectivity and much more bandwidth than was available in Europe before. In fact, EASInet established the first international European backbone for research and science that extended to multiples of 64Kbps in bandwidth.
ObjectivesIn the context of the considerations for the realization of the network it was obvious that no EASI partner could be interested in enlarging the number of its already existing network connections by one more. This would unnecessarily increase the complexity and the effort to operate the network infrastructure. Instead there was a tremendous demand to increase existing bandwidth or at least consider the integration in a future networking concept.
The efforts undertaken by IBM, to find a European carrier or any other vendor who would collaborate with EASInet to contribute to a "European Broadband Link" at that point in time, failed. So the European networking service providers were left alone in trying to set up a simpler, more cost-effective and more efficient networking infrastructure for research and education.
It was also clear from the begining, that it would not be sufficient to provide the EASI partners with physical lines only. To guarantee connectivity it was required that IBM established a central network management and started a well coordinated and careful planning of EASInet as an additional offer. GMD, the German National Research Laboratory for Computer Science, was subcontracted to help design, implement and operate the network.
Networking EnvironmentThe goal of EASInet is to provide different applications to the end user using different protocol stacks. To support these different protocols in one backbone scenario EASInet made a lot of investigations with representatives of the 18 EASI sites to understand the special needs and requirements of each particular site. The outcome of this was the following:
- Each site will not tolerate any degradation of connectivity or service compared to the current status.
- Each site rely on the service in relation to the share they pay in terms of bandwidth and/or equipment.
- The main issue is 'connectivity'. Which means EASInet has to provide connectivity in the following technical areas.
- TCP/IP (Transport Control Protocol / Internet Protocol)
- SNA (IBM Systems Network Architecture)
- X.25 (CCITT / ISO / OSI Connection Oriented Network Service (CONS))
- Additionally there should be a technical solution to support also CLNP (ISO / OSI Connection Less Network Service) and DECnet using the same infrastructure.
- The new backbone should be a step forward to an integration of different physical connections throughout Europe utilized in terms of different communities, services and applications.
Technical SolutionIn general EASInet is designed to support different communication protocols (like TCP/IP, X.25, SNA, CLNP, DECnet, ...) on the same underlying infrastructure. Concept and design of this infrastructure depends partly on the facilities and services European PTTs provide to customers in the scientific community. Besides these technical issues the political conditions in the European Networking Communities led to some special agreements and solutions. Reflecting this circumstances EASInet implemented mainly two different kinds of multiplexing.
- Multiplexing using dynamic time division multiplexer IBM 973x (IDNX), manufactured by N.E.T. (Network Equipment Technology).
- Multiplexing using X.25 ISO/OSI connection oriented protocol as a carrier for different protocol stacks (TCP/IP, IBM SNA, ...).
The IBM 973x backboneCurrently the IDNX backbone consists of fourteen IDNX boxes distributed throughout Europe.
Figure 2. The IDNX backbone of EASInet
Each IDNX box is used to separate communication protocols on international communication links. This enables the network provider to guarantee bandwidth for different purposes.
*** Please see graphic ***
In general, time division multiplexing can be done in three different ways. First, using a kind of split modems (DSU/CSU) which allocate bandwidth on a permanent basis using fix granulation (i.e. 9600 baud or 64000 baud channels). Fix granulation leads usually to a certain amount of internal fragmentation, which can not be tolerated because of European communication tariffs. Second, using a statistical multiplexing technique, which utilizes the bandwidth on demand. This is the optimal way to communicate but equipment supporting these multiplexing technique is not available for such line speeds at a reasonable price.
- Guarantee productional bandwidth for a specific protocol stack
- Guarantee productional bandwidth for a specific community
- Allocate bandwidth for test scenarios (i.e. CLNP)
Finally, the IDNX box uses a third kind of bandwidth allocation scheme called 'dynamic bandwidth allocation'. In general, during the lifetime of a connection (using one protocol, between two transmission facilities) the amount of bandwidth is fixed. The interesting point here is, that the granulation of the allocated channels is not fix. That means, we can assign in a flexible way bandwidth to different protocols and/or communities as requested, using 'dynamic bandwidth allocation'.
This feature and the ability to manage the resulting IDNX backbone remotely in an efficient and sophisticated way were the main points in choosing IBM 973x multiplexer. Last but not least the IDNX boxes are able to support digital voice transmission, which become more and more importance in Europe.
The fourteen IDNX boxes build up one network. This means, each IDNX has knowledge about all available (connected) IDNX's somewhere in the network. This leads to a point of view as a whole integrated level 1 network. It is assumed that each IDNX has a unique number assigned, so that we can identify each backbone EASI site by an IDNX Node number (IN).
Here the actual list of IDNX nodes:IS IN connected to managed by ----------------------------------------------------------------------- CEA Fontenay N1 N2, N3 CEA CEA Paris N2 N1, N3 CEA CEA Saclay N3 N1, N2, N4, N5 CEA CEA Grenoble N4 N3, N5, N6, N141 CEA CEA Cadarache N5 N3, N4 CEA CEA Pierrelatte N6 N4, N7 CEA CEA Marcoule N7 N6 CEA CIRCE Paris N33 N133 CIRCE/GMD SARA Amsterdam N131 N141 SARA/GMD CNUSC Montpellier N133 N33, N141 CNUSC/GMD CERN Geneva N141 N4, N131, N133, CERN/GMD N149, N233, N249 CERN/GMD GMD Bonn N149 N141 GMD IN2P3 Lyon N233 N141 IN2P3 DESY Hamburg N249 N141 DESY/GMDTo allocate bandwidth between two (data) ports of two (usual) different IDNX boxes the network manager just defines the amount of bandwidth together with the source port and destination port of the involved IDNXs. Each IDNX on the way from source to destination determines the amount of resources needed (internal bus capacity, spare bandwidth pool, ...) and if possible made the necessary allocations. To communicate with each other for internal administration purposes the IDNX uses its own protocol, called SCLP (Signal Channel Link Protocol). SCLP based on the recommendation SS#7, but its specification is not published. The amount of bandwidth used for IDNX internal communication varies between 6400 and 16000 baud, depending on the kind of physical link leased from the local carrier.
To manage the IDNX backbone it is necessary to have some detailed information about the actual status of the backbone nodes and the occurring events. Each IDNX node has two types of event logs:
Consequently there is one designated IDNX node which collects all network significant events in the EASInet IDNX backbone. In our case it is the IDNX node N149 at GMD Bonn, the Network Operation Center (NOC). At the NOC we have installed additional equipment, namely IBM Netview/PC, IBM NetView R3 and N.E.T. ALERTMON to monitor the IDNX backbone automatically, convert the IDNX events to SNA NMVTs (Network Management Vector Transport) and display them on a centralized NetView NPDA (Network Problem Determination Application) panel on the NOC host.
- node event log, which logs all events dealing mainly with node internal conditions
- network event log, which logs all network significant events coming out of the network
Additionally GMD NOC has installed the EOC, Enhanced Operator Console, an IBM PS/2 system providing reporting and statistical data gathering tools to analyse the IDNX backbone event and status information in a detailed way. This helps to recognize as soon as possible physical link problems in correlation with reliability and availability degradations of specific backbone connections.
>SNA/SNI ConnectivityThose EASI sites who need connectivity for their mainframes running SNA (IBM's Systems Networking Archtecture) are connected via the SNI technique. SNI (SNA Network Interconnection) allows to manage large SNA networks by splitting them into many small networks or to connect other autonomous networks. Within EASInet the 'back-to-back' technique of SNI has been implemented. With 'back-to-back' the participating SNA networks remain nearly autonomous. They only need to agree to a few definitions of a common 'Null-net' to which they are connected via one or more gateways. The gateway function is part of the SNA access methods on the IBM mainframes.
The SNI network of EASInet itself has gateways to other SNI networks within Europe e.g. EARN (European Acadamic Research Network) or AGFnet (Network of the German National Research Centers and Universities).
In Germany the SNI network of EASInet is implemented on top of X.25, this means that all the German EASI sites are connected to WIN and via SNA ports in the IBM 3745 of DESY, Hamburg, to the international IDNX-backbone.
X.25 ISO/OSI support within EASInetAll German EASI sites are connected to WIN. WIN (WIssenchaftsNetz) is an X.25 network, based on '84 standard, offered by DFN (Deutsches Forschungsnetz) with speeds up to 64KBit/s. Dependent on the need and the availability, the German EASI sites are running OSI applications via WIN. Mostly used OSI applications are those supporting the ISO/CCITT-standards X.400 for electronic mail and X.29 (X.3/X.28) for line oriented dialogue.
Primarily the X.25 infrastructure of WIN is used to get connectivity for SNA/SNI, TCP/IP and DECnet. International X.25 connectivity for the EASI sites - if needed - is provided via the X.25 ports of the IDNX backbone, e.g. on the link between DESY and CERN.
Figure 3. The german EASInet connections
*** Please see graphic ***
The NSFNET T1-TechnologyThe IBM designed packet switch, also known as the Nodal Switching System (NSS), provides access to the NSFNET backbone, "switching" between backbone nodes, intelligent routing and network management. The NSS is a high-throughput, low-delay packet switch that combines IBM products in a modular design that is extendable, serviceable and fault tolerant.
This concept has proven its quaility and rich functionality now for more than three years in the U.S. and nearly two years in Europe.
To achieve this, the NSS design connects multiple RISC-processors based on IBM RT/PC's, (IBM 6151 and IBM 6150) connected to a dual token ring. This dual token ring guarantees the requested fair access method to the common network media. Although a NSS consists of more that one processor it looks like as one functional unit from the outside.
The processors attached to the dual token ring are divided in the following categories:
For a detailed description of the NSFNET technology the reader may look at the NSFNET proposal dated November 1987 'MANAGEMENT AND OPERATION OF THE NSFNET BACKBONE NETWORK' which shows the original concept of the NSFNET T1 backbone network in the U.S.
- Packet Switching Processor (PSP): Processor with serial line interface, currently at T1 speed.
- Exterior Packet Switching Processor (E-PSP): Processor which provides connections via Local Area Networks between the NSFNET backbone and the Regional Networks. These are usually Ethernet LANs.
- Routing Control Processor (RCP): Processor that gathers, calculates and distributes routing information.
It controls the flow of routing information within the NSFNET backbone using the dual-IS-IS-routing protocol and between the NSFNET backbone and the regional networks using the EGP (Exterior Gateway Protocol) and BGP (Boundary Gateway Protocol) routing protocols. The RCP is used to gain a consistent view of the routing tables for processors belonging to a NSS.
Other processors attached to the dual token ring are used to manage the token ring via IBM Netview, monitor traffic and provide backup.
The T1-LinkOn march 15, 1990 IBM Europe announced the largest and fastest data communication link between universities and research organisations in the United States and Europe.
Information coming to Europe from the U.S. enter EASInet at the European Laboratory for Particle Physics (CERN) in Geneva, Switzerland, which is an EASI partner. The American gateway to NSFNET is the Cornell National Supercomputer Facility at Cornell University in Ithaca, State of New York.
The line ends at EASIgate, a token ring infrastructure with all kinds of processors to provide support for EASInet connections and the T1-Link to NSFNET.
The CYLINK 4201 ACSUThe connection of the physical T1-Link at CERN and Cornell is done via the 4201 Advanced Channel Service Unit (ACSU), a stand-alone D4 or Extended SuperFrame (ESF) channel service unit. It performs the following functions:
The machines at CERN and Cornell accommodate ESF framed input, using Alternate Mark Inversion (AMI). On the network side, ESF framing and AMI network coding are selected.
- serves as an interface between a user's data terminal equipment (DTE) and a 1.544 Mbps digital line.
- monitors and ensures that the signal input from the DTE complies with AT&T T1 transmission requirements.
- terminates the T1 line and regenerates the received network data stream.
- provides extensive diagnostic and performance monitoring.
The NOC can remotely configure and diagnose the ACSU and obtain internally stored alarms and performance data by means of the CYLINK Netwoek Managment System (CNMS). The available RS232C port operates asynchronously at 1200 bps and is a dial-up connection.
A detailed view of EASIgate and the physical T1 circuit is shown.
Figure 4. Details of the T1-Link
Figure 5. Infrastructure at EASIgate
*** Please see graphic ***
*** Please see graphic ***
EASInet Router TechnologyAt CERN one processor called Split Exterior Packet Switching Processor (Split-E-PSP) was placed as an extension of NSS-10 in Cornell to offer an access point to Europe with the same functionality as any other E-PSP processor direct connected to a NSS. This split-E-PSP called psp-10-s1-2 is fully managed by MERIT Inc., the NOC which manages NSFNET, and controlled by the RCP Routing Control Processor of NSS-10. The psp-10-s1-2 is used as the access point between EASInet and NSFNET to offer packet switching functionality as well as the exterior gateway protocol EGP. EGP is used to exchange routing information between NSFNET and EASInet.
The EASInet engineers have chosen the token ring as the media to interconnect NSFNET and EASInet. Attached to this token ring are routers managed by EASInet.
These routers offer connections to the CERN infrastructure and to other regional networks that are connected via serial lines to EASInet routers directly.
To benefit from the NSFNET technology, EASInet uses IBM RT/PC's as routers as well.
Due to different user requests and different line speeds that are feasible, the NSFNET model has been slightly modified to achieve the best for the European situation:
Currently a single token ring is sufficient for our needs, because EASInet has less traffic than NSFNET.
- IBM RT/PC's are used as autonomous routers with full functionality.
Each IBM RT/PC combines functions of the NSFNET RCP, PSP and E-PSPs. They are not controlled by a Routing Control Processor. Each EASInet router performs its own calculation for routing protocols. This distributed routing sheme offers much better fault tolerance than the NSFNET model because it does not depent on a single Routing Control Processor. We have chosen to use the public domain GATED routing software which now offers a large variety of interior and exterior routing protocols (i.e.: RIP, HELLO, OSPF, EGP, BGP).
- A single IBM RT/PC is used to monitor all packets on the token ring and collect statistical data like protocol usage, traffic by networks and other data. It is also used as an TCP/IP protocol analysing tool for problem determination.
- All system software and configuration is held on the same level on all routers. Only a few configuration files on the routers differ. All configuration files are also available in the EASInet management center at GMD, Bonn. Therefore the NOC has the possibility to configure any router with the same configuration as any other router in EASInet.
- All routers offer the same hardware functionality. This means that in case of faulty routers any router may be substituted by another router connected to that token ring infrastructure.
Figure 6. Some statistical data
*** Please see graphic ***
The Network Management ToolsMonitoring and managing complex networks involves acquiring and processing information from a large number of different sources.
XGMON is a network management facility, developed for the NSFNET network backbone. Our NOC is using XGMON to manage the IP part of EASInet, since the program offers a Simple Network Management protocol (SNMP) client. In the IP world, SNMP is available with most vendor products and therefore allows the management of a heterogenous networking environment.
The SNA related network management information in EASInet is collected and processed via IBM Netview. The program has been discussed in detail in a former Systems Journal (Volume 27, Number 1, 1988).
The network management service for X.25 is basically provided by the PTTs so that GMD is not much involved. The problem with X.25 network management information is that the data which are provided is not unique and therefore very hard to process. The reason is the different vendor equipment being used.
Within EASInet we currently develop a network management platform for a multiprotocol environment based on the ISO standards Common Management Information Services and Common Management Information Protocol (CMIS/CMIP). This prototype will collect information from the IDNX boxes, IP routers, 37xx's and X.25 switches to perform the actual processing.
Transmission of OSI Data TrafficThe international exchange of ISO CLNP packets was demonstrated between end systems at the NSFNET network operations center in Ann Arbor, Michigan and the EASInet network operations center in Bonn, Germany, in August 1990.
Figure 7. Configuration for ISO CLNP support
The current NSFNET prototype OSI implementation is intended to provide wide area connectivity between OSI networks, including networks utilizing the DECnet Phase V protocols.
*** Please see graphic ***
As NSFNET today is primarily a TCP/IP network, the OSI implementation is designed to coexist with the operational IP NSFNET. Both DoD IP and ISO CLNP packets are forwarded together through the same packet switches and links. CLNP packets are transmitted "native" on the links; they are not encapsulated within DoD IP. Dynamic routing is supported for both protocols, providing automatic rerouting in case of an outage.
The NSFNET OSI implementation supports the ISO Connectionless Network Protocol (CLNP, ISO 8473). This is a datagram protocol, similar to the DoD Internet Protocol (IP), providing best effort delivery of packets. Each CLNP packet is examined by the backbone and forwarded along the best path toward its destination.
The implementation supports the transmission of CLNP packets over T1 serial links, IEEE 802.3 CSMA/CD LANs, and IEEE 802.5 Token Ring LANs.
Routing and address resolution between NSFNET, End Systems and other Intermediate Systems is provided by the End System-Intermediate System routing protocol (ES-IS, ISO 9542). This protocol provides functionality similar to the Address Resolution Protocol (ARP) and the Internet Control Message Protocol (ICMP) Redirect functions used in the TCP/IP protocol suite. Routing internal to the NSFNET backbone is based on the ISO IS-IS intra-domain routing protocol (ISO DIS 10589).
Interdomain routing between the NSFNET backbone and other routing domains is currently static, awaiting the development of OSI interdomain routing protocols.
Collaboration with other networksAs mentioned before, EASInet has reached some agreements with other networking organisations in Europe by sharing lines which provide a multiple of 64Kbps. A number of mutual backup agreements exist for the T1-Link as well.
Figure 8. EASInet as of 06-02-92
The following table shows the line sharing agreements which exist today:
*** Please see graphic ***
Remarks: This foil has a multiple purpose:
1) The red frames (boxes) are the EASI sites
2) The black solid lines demonstrate our 3-year
commitment for 64Kbps connectivity
3) The green lines show the agreements in
bandwidth we have reached with other organisations
4) A description of bandwidth splitting in terms
of protocols multiplexed via the IDNX could be
found as well
EASInet Cooperations Line from Line to Bandwidth Partner Network _____________________________________________________________________ Stockholm Amsterdam 192Kbps NORDUnet Amsterdam Geneva 256Kbps HEPnet, SURFnet, EUNET Hamburg Geneva 768Kbps HEPnet, DESY Montpellier Geneva 256Kbps CNUSC, EARN Lausanne Geneva 2048Kbps SWITCH Paris Geneva 256Kbps CEA Lyon Geneva 1024Kbps IN2P3
ConclusionEASInet, the second IBM initiative in data networking for Europe, has improved the national and international networking connections for the EASI partners and other European networks. The authors believe that EASInet has stimulated the sort of collaboration in european data networking towards a pan-European backbone. The heterogenous environment we have implemented and in operation today for the research and education community could be extended to support commercial customers. We have proven that Europe has the skills, the industrial strength, and, we hope, also the will, to continue to make progress with a solid pan-European infrastructure for data networking. There is still a lot of work to be done to improve the understanding of the "problems" of european data networking among Europe's politicans, PTT's, network providers, industrialists and network users.